I suspect at some point LLM in its current form will be deemed good enough for general research and coding tasks. I don't get why we need to continue with a de-facto cloud-based approach. Cloud in my opinion solves operational complexity, which is worth paying a premium for. But it seems it isn't quite all that complex to get an open source model running locally as long as you have the hardware. Over time I suspect the models get better and cheaper.
Is there a future where we can expect people to just buy "AI" from BestBuy, like a TV set? It ll probably come with some model preloaded - cheaper if open-source, premium pricing for frontier lab models. The hardware is basically a bunch of GPUs enough for local inference.
Take it home and plug it into your home network and you can open a chat instance by going to the IP on any local device. You can give it access to internet if you want. Maybe it can also receive OTA updates.
Curious how others think about this - does local-first AI feel like a possibility? What are the economic and social challenges with this?
Today a local AI box (Strix Halo, DGX Spark, Mac Studio) is $2,500+. Even if it comes down, almost no one will pay upfront when they could pay $20/month instead.
Small models can run on a normal PC but similar or better quality models are free in the cloud.
Is there a future where we can expect people to just buy "AI" from BestBuy, like a TV set? It ll probably come with some model preloaded - cheaper if open-source, premium pricing for frontier lab models. The hardware is basically a bunch of GPUs enough for local inference.
Take it home and plug it into your home network and you can open a chat instance by going to the IP on any local device. You can give it access to internet if you want. Maybe it can also receive OTA updates.
Curious how others think about this - does local-first AI feel like a possibility? What are the economic and social challenges with this?