When I found Tesla P100s on ebay for $75 I thought I was getting a cheap server, but 1/3 of the total cost was RAM. Sorry, first time I've built a computer since the shortage.
I don't know why we're even talking about Qwen3.6 for writing code when qwen3-coder exists. My experience is there's no contest. I'm using 30b with 96k context on a dedicated server.
Qwen3.6 was the first model I ran locally that seemed smart, but qwen3-coder:30b is way, way more responsive and more accurate for writing code according to my tests, including human-eval. If you can run one than you can almost certainly run the other. If you haven't tried qwen3-coder I would definitely recommend it. It feels positively snappy compared to every other local model I've tried. All you need is 32G VRAM and some heat dissipation.
I know it uses electricity, but part of the benefit of a local model has to be that you can let it do this while you sleep, and not pay Anthropic for an unknown number of tokens.
Exactly. I have sort of a fetish for trying to make things smaller by trimming out things that aren't needed. Unfortunately this skill has been largely useless since forever, because hardware improves to the point that these optimizations are trivial:
Network Bandwidth, Storage space and speed, memory capacity. While all of these were worth optimizing for at a point in history, that point is behind us. It's probably a reasonable expectation that it will eventually be true for VRAM.
I'm really curious about this, not because I disagree, but because I want to avoid agents going whack. Are you running vllm for yourself only, or a for a team, or for an application, etc? And do you feel there is a minimum hardware requirement for vllm to be useful in this way?
My weekend project is going to be building a home inference server (from ancient datacenter parts) and I'm still massaging in my head what the end result will be.
The fact that every job application wants a link to my profile on a platform that tries to push "brain training puzzle and games" on me just makes me angry every single time. I really hate LinkedIn and my active rebellion against it is hurting my ability to find a new job.
I know there has been other LinkedIn hate on HN this week. I know they have some good tools for job searching and hiring. I still wish we as a society could move on and leave this one with MySpace.
Sure. Even a history of safety success contributes to this. We haven't had an accident in 3000 days, what was dangerous about this job again? Also what's this stupid policy for anyway, I've never seen anybody even come close to (non-dangerous-sounding fate) while working here.
But probably the policy is in place because it used to happen before the policy was in place. It's just not obvious to people who have never seen the consequences before.
This is really accurate to my experience learning FreeCAD earlier this year. I am a former professional CAD user (of a lesser software than AutoCAD) and I don't think I would have gotten far without being able to ask ChatGPT for help understanding some of the quirks of FreeCAD.
For free and open it's truly impressive though. Actually I think my time building iOS UIs in Storyboard was at least as useful as previous CAD experience, since constraints are the foundation of (at least one approach to) designing parts.
To be fair, I think human nature is probably a bigger culprit here than the quote. Yes, it was one of the first things told to me as a new programmer. No, I don't think it influenced very heavily how I approach my work. It's just another small (probably reasonable) voice in the back of my head.
DRY is not to avoid writing code (of any amount). DRY is a maintainability feature. "Unless you're very familiar with the code" you probably won't remember that you have to make this change in two places instead of one. DRY makes life easier for future you, and anyone else unfortunate to encounter (y)our mess.
I think this is a reasonable rule of thumb, but there are also times that the code you are about to write a second time is extremely portable and can easily be made reusable (say less than 5 minutes of extra time to make the abstraction). In these cases I think it's worth it to go ahead and do it.
Having identical logic in multiple places (even only 2) is a big contributor to technical debt, since if you're searching for something and you find it and fix it /once/ we often thing of the job as done. Then the "there is still a bug and I already fixed that" confusion is avoided by staying DRY.
I'm sorry that the author got bitten by this. But .com purism is funny to me. I only buy GTLDs for personal projects, and I've never had a problem before. But then, I've never bought .online.
Right, I'd say this is the best value I've gotten out of it so far: I'm planning to build this thing in this way, does that seem like a good idea to you? Sometimes I get good feedback that something else would be better.