> Recently someone messaged me on Reddit about my post. I replied. They wrote again, I replied again. After a few messages I realized I was talking to an AI agent.
My exact experience. The irony was that we were talking about AI agents
14B even at Q4 isn't realistic for coding on a single 12GB RTX 3060. Token speed is too slow. After all they are dense models. You aren't getting a good MoE model under 30B. You can do OCR, STT, TTS really well and for LLMs, good use cases are classification, summarization and extraction with <10B models.
Interesting. Won't stuff like entity extraction suffer? Especially in multilingual use cases. My worry is that a smaller model might not realize some text is actually a persons name because it is very unusual.
Is it possible for such a small model to outperform gemini 3 or is this a case of benchmarks not showing the reality? I would love to be hopeful, but so far an open source model was never better than a closed one even when benchmarks were showing that.
This is very interesting. Especially the last part where it shows gpt-5.2 and gpt-oss and their very similar and unique outcome of being 90%+ Serious.
I tested this locally and got the same result with gpt-oss 120b. But only on the default 'medium' reasoning effort. When I used 'low' I kept getting more playful responses with emojis and when I used 'high' I kept getting more guessing responses.
I had a lot of fun with this and it provided me with more insight than I would have thought.
Yes. I mostly work on Quarkus microservices and use cursor with auto agent mode.
> we wouldn't give an AI some vague requirements and ask it to build something
> we would discuss as a team
seems like a reasonable workflow. It's the polar opposite of what was written in the blog post. That is the usual, easy way people use agents and what I think is the wrong path. May I also ask what language and/or framework you work with where so much context works good enough?
> Asking AI to explain code and help me learn how it works means I can pick up new systems significantly quicker.
LLM's are good at making stuff from scratch and perfect when you don't have to worry about the codes future. 'Research' can be a great tool. But LLMs are horrible in big codebases and multiple micro services. Also at making decision, never let it make a decision for you. You need to know what's happening and you can't ship straight AI code. It can save time, but it's not a lot and it won't replace anyone.
I don't want to sound rude, but what was your reason to go from scratch instead of joining an already established, open source effort? The likes of Cline, Roo, Continue, ...