My biggest gripe with Java has always been the tooling ecosystem, almost every java engineer I know solves this with the attitude of "just use intellij".
No. I won't, I refuse to use that, and I refuse to give a company money just to use a language as efficiently as other languages.
I am very happy to see that there's progress being made here. I wish you the best of luck!
Since I started running my own inference server, I've had zero degradation that I didn't do myself. Basically the only time I see it get worse is if I drop one of the quants.
Which is what I suspect the providers are doing to fit more inference on the same amount of hardware over time.
I think the majority here have stated the same... That CLAUDE.md or AGENTS.md effectively do this. Either that or the readme.
The only tip I can give is that your skill that builds or wraps up work. You should have it update those files if anything has changed.
Claude/Agents files shouldn't be bloated, but should imho act as a basic amount of context on the project so your agent and skills can pick up and go, with even the most basic initial prompt.
Well. Right now buying hardware to run your own models tops off at about 32gb VRAM at any price point that's not insane. Sure you can get a Mac mini, or a PC equivalent. But the problem is RAM.
More RAM means bigger models, which means smarter models.
Which is why Qwen and Gemma have been so interesting to a lot of us who run our own... Now 32gb VRAM isn't so bad, as these models can be run on that with decent results.
Where this gets interesting is in a couple years, when all the A100, etc, all the Enterprise hardware hits eBay.
Sure. It's just an old I7 8700 (non-k), 64gb ram. Running proxmox. But recently I put an AMD R9700 AI Pro, in there which is a 32gb inference focused card, think of it as a 32gb version of a 9070xt.
All the inference happens on that card, so the CPU/RAM is there for the other containers.
I'll eventually swap the motherboard and CPU for something better, so I can fit 1 or 3 more of those cards.
Why not NVIDIA? 32gb on team green means spending crazy money. And I can get 4 R9700s for the cost of one 32gb 5090.
Yes. I run local models, Qwen3.6-27B and IMHO the massive level up was the agents and skills files that I've worked on.
Basically I run a flow
Brainstorming > Create Spec > Review Spec* > Create Plans > Review Plan* > Execute Plan (in subagents) > Review Against Plan > Code Review* > Open PR > Finish Plan (marks plan files done)
* Each review step marked with an asterisk uses a paid larger LLM, right now Deepseek V4 Pro. Having it do this catches a lot of small things, and now I'm effectively one shotting any task I give it.
And it's not costing me much at all, just those three reviews. I could use a free model like Gemini but I'm happy with what I've got.
I've spent the last month bringing in a small demo of what the future could be like, running Qwen, Gemma, and Deepseek, behind LiteLLM so we can monitor token usage, and instead of some dumb ass "tokenmaxxing" we're actively trying to get the cost of inference both down, and in-house.
Boss is happy, very happy. We're rolling it out more widely now.
Same. Having experienced the growth of computing in those eras, the show itself had a very well researched yet very nostalgic sense of "oh yes. I'd forgotten about that".
The best part of Silicon Valley was that it had a very south park quality to it.. in that things that were actually happening at the time were parodied on the show.
Well considering right now MTP support is being developed, there was a conversation in that that seemed to throw around the idea of separating the MTP model out of the main GGUF, like with Mmproj. This was rejected.
Which I'm happy for. So given that decision, I don't think it's unreasonable to think that they might be open to including Mmproj files in the GGUF.
Only issue I can think of is, which one? BF16, F16? Etc
There's a percentage of people who love to question how the open models were trained.. they are almost always going to try and make some argument about using the closed frontier models for distillation as some form of theft.
Just totally forgetting that the frontier models themselves stole an insane amount to get to where they are.
It's theft all the way across the board, and when someone tries to make the argument that open models theft is bad, but Altman or Amodei's theft is good.. they are revealing a lot about themselves
Basically small and medium models that are crazy well trained for their sizes.
Then we have a lot of specular decoding stuff like MTP and others coming to speed up responses, and finally better quantisation to use less memory.
Local LLM is the future, and the larger labs know that the open models will eat their lunch once people realise that the gap is only a few months. If we were good with LLMs a couple months ago, we're good with the open models now.