Yes, but requires decent hardware and energy to do so. If the cost to host keeps dropping, people will self host and the ai companies won’t make money. If the cost remains high, the subscriptions won’t provide value and they won’t make money.
I dunno about that… Very small models (2-8B) sure but if you want more than a handful of tokens per second on a large model (R1 is 671B) you’re looking at some very expensive hardware that also comes with a power bill.
Even a 20-70B model needs a big chunky new graphics card or something fancy like those new AMD AI max guys and a crapload of ram.
Granted you don’t need a whole datacenter, but the price is far from zero.
Price is essentially zero if you just run it locally
Yes, but requires decent hardware and energy to do so. If the cost to host keeps dropping, people will self host and the ai companies won’t make money. If the cost remains high, the subscriptions won’t provide value and they won’t make money.
I dunno about that… Very small models (2-8B) sure but if you want more than a handful of tokens per second on a large model (R1 is 671B) you’re looking at some very expensive hardware that also comes with a power bill.
Even a 20-70B model needs a big chunky new graphics card or something fancy like those new AMD AI max guys and a crapload of ram.
Granted you don’t need a whole datacenter, but the price is far from zero.