> If the M5 generation gets this GPU upgrade, which I don't see why not, then the era of viable local LLM inferencing is upon us.
I don't think local LLMs will ever be a thing except for very specific use cases.
Servers will always have way more compute power than edge nodes. As server power increases, people will expect more and more of the LLMs and edge node compute will stay irrelevant since their relative power will stay the same.
Compared to an encyclopedia, Wikipedia is unreliable. That doesn’t mean that Wikipedia isn’t useful. But there is a hidden danger with Wikipedia being mostly reliable - people lower their guard and end up consuming misinformation without realizing.
I think there is a difference between ideas that are based on a fundamental misunderstanding of how the world works and ideas that we do not currently have the capabilities to solve.
I would argue that most of the value of LLMs comes from structuring your own thought process as you work through a problem, rather than providing blackbox answers.
Using AI as an oracle is bound to cause frustration since this is attempts to outsource the understanding of a problem. This creates a fundamental misalignment, similar to hiring a consultant.
The consultant will never have the entire context or exact same values as you have and therefore will never generate an answer that is as good as if you understand the problem deeply yourself.
Prompt engineers will try to create a more and more detailed spec and throw it over the wall to the AI oracle in hope of the perfect result, just like companies that tried to outsource software development.
As a dev that often writes UI text, which simple rules do you recommend that I should follow?