Correct. We have open-weight models from OpenAI, Facebook, Mistral, DeepSeek, Z.ai, MiniMax, and all sorts of other companies. Most of them have fantastic and open licensing terms.
If we can't build the weights, then we don't have the source. I'm not entirely sure what an open-source model would even look like, but I am confident that these binary blobs that we are loading into llama.cpp and vllm aren't the equivalent of source code. We have absolutely no idea what sort of data went into them.
This is fine. It isn't slanderous. It is what we have, and it is awesome. Just because it is awesome doesn't make it open source.
No, but yes? OmniCoder 9B at Q6 fits on my 9070 XT with 200k+ tokens of context, and it works pretty well with OpenCode. It is for sure the best local model that I've managed to squeeze onto my GPU, and it even works at 120k context at Q3 on an 8GB RX 580 GPU.
I can't imagine trying to using this model on either GPU for real work. I can use much bigger and faster models on the $3 Chutes subscription or $10 OpenCode Go subscription.
Even so, I am still excited. I don't feel like there was even a model worth using with a tool like OpenCode 6 to 9 months ago. I like the way things are heading, and I am looking forward to seeing how capable coding models of this size are in another 6 to 9 months!
This can be such a complex problem. Maxing out 8 cores is rarely twice as fast as maxing out 4 cores, but it will likely make the CPU draw around twice as many watts. That makes it seem like it might be a better use of power to keep cores idle during a big compile job, but it also takes watts to drive the display and keep the backlight on.
I can't imagine this would be easy to get right, and it is probably much harder to set a good default. Even so, the background compile job taking twice as long might not be a waste of time as long as I get to play Oxygen Not Included twice as long!
The Crucial MX500 is a rather old piece of hardware now.
Tom's Hardware NVMe benchmarks include a "Sustained Write Performance and Cache Recovery" component. Whenever there's a good sale price on an NVMe, that is just about the only metric I hunt down now. Most of the worst drives they test will always beat a mechanical disk, but the worst drives Tom's Hardware ever tests are still decent drives.
I grabbed one of the cheapest SATA SSDs last week to replacing a failing lvmcache drive. It is a NETAC 1 TB that might still be on sale on eBay for $34. I expected the worst, and I did want to test its sustained write performance, but I wasn't as nearly scientific as you!
I just ran dd for a while and watched it stay between 420 and 470 megabytes per second for about 120 gigabytes straight before I stopped the test. The meanest I am to this cache is dropping 50 GB of video on two different days each month, so that was all the data I needed.
Had I known that I would be reading your blog four days later I would have let the dd finish so I could take better notes! Thank you for taking the time to do the science for us!
I never have much trust in the time on page or time on site numbers. In your screenshot, the number sure looks like it is meant to be a total.
I use Matomo for my analytics. I think their little Javascript doodad attempts to report back time information after 15 seconds. Anyone who clicks away before that timer goes off is recorded as a zero.
I get a sort of warm fuzzy feeling when I see high time on page numbers, but zeroes never bother me. There are a huge number of reasons why a time can't be captured, but when a time is captured, it is almost definitely someone who had eyeballs on your page.
You're doing a good job, Simon! I would be thrilled to see a million clicks from Twitter on one of my posts. I am just more excited about your 50k clicks from Hacker News!
The views are nice, but the time on page stats from social media tend to be horrible. One of Simon's screenshots shows over a million pageviews with 43 minutes of time on page. This isn't a long blog post, but it is fairly information dense. Out of 1 million unique views, this is only enough minutes for a couple dozen people to have actually read the entire post.
Most of my blog's traffic comes in from Google, and I most of my posts that see a reasonable amount of traffic will have at least average 1 or 2 minutes of time on page.
If I get a spike in traffic to a page from Twitter, the average time on page will drop to 1 or 2 seconds.
When someone comes in from search, they are seeking information about something specific. The folks dropping in from Twitter are infinitely more likely to immediately click on something else.
I don't know that I write much that would be of broad interest to the Hacker News audience, but I would be much happier to see traffic from here than from Twitter. I would bet Simon's 49.5k clicks from here got him way more engagement than 712k clicks from Twitter.
If we can't build the weights, then we don't have the source. I'm not entirely sure what an open-source model would even look like, but I am confident that these binary blobs that we are loading into llama.cpp and vllm aren't the equivalent of source code. We have absolutely no idea what sort of data went into them.
This is fine. It isn't slanderous. It is what we have, and it is awesome. Just because it is awesome doesn't make it open source.