Show HN: Reame – a CPU inference server that gets faster as it runs(github.com)
github.com
Show HN: Reame – a CPU inference server that gets faster as it runs
https://github.com/swellweb/reame
16 comments
Seems the entire repo and documentation is all ai-generated?
Yes, I use AI to help me build and write docs — solo dev, I'd be stupid not to. The benchmarks are real though: free Oracle ARM box, my laptop, a cheap VPS. I ran them all myself, including the ones that went badly (they're in the README too).
Thanks for sharing your work!
Please how to select the model? I downloaded tinyLlama, put it in ./models, changed reame.conf but I get:
(No such file or directory)
Otherwise putting the model in /opt does not please me much, I fear to forget a model is there, if it is in reame folder its much easier to notice and manage.
Please how to select the model? I downloaded tinyLlama, put it in ./models, changed reame.conf but I get:
(No such file or directory)
Otherwise putting the model in /opt does not please me much, I fear to forget a model is there, if it is in reame folder its much easier to notice and manage.
Sick that you can get 2 arm cores and 12 GB ram for free at Oracle cloud, did not know that
You can actually get more. You have an amount of resources for ARM servers, if you allocate all of that to a single one you ger 24GB RAM and 4 vCPU.
This was reduced recently to 2/12. It's also next to impossible to allocate the free stuff unless you put in a CC. If you do, you get the servers right away, though. Just be careful not to spend. You can set up a budget trigger to shut down the vms.
Old servers may have been grandfathered, but mine was set to 1/1 and couldn't be reshaped.
Old servers may have been grandfathered, but mine was set to 1/1 and couldn't be reshaped.
Yeah but it's Oracle
If you have good luck perhaps. That tier of server is virtually always sold out
If a prefix cache is good, why not keep it in primary mem?
Why qwen 2.5 everywhere? Why not 3.5?
Because llms reflect qwen 2.5 heavily in training data, and that's who is giving the recommendations.
looks really interesting
Interesting. But I'm kind of hard getting past the AI-written README:
"What Reame is NOT for — said plainly, because trust is built here: a general-purpose ChatGPT replacement (frontier reasoning and broad knowledge need frontier parameter counts), agentic coding assistants, or creative long-form writing at scale. If your task needs a 100B-class brain, buy one; if it needs your documents processed privately, forever, at zero marginal cost — that's a realm you can own."
The realm you can own. How did these things learn to write that way? Oh, yeah, lots of marketing and advertising copy.
"What Reame is NOT for — said plainly, because trust is built here: a general-purpose ChatGPT replacement (frontier reasoning and broad knowledge need frontier parameter counts), agentic coding assistants, or creative long-form writing at scale. If your task needs a 100B-class brain, buy one; if it needs your documents processed privately, forever, at zero marginal cost — that's a realm you can own."
The realm you can own. How did these things learn to write that way? Oh, yeah, lots of marketing and advertising copy.
It is so weird (or, used to be) to see an LLM's internal thought process pop up this way. Like imagine how strange it would be to read human writing that accidentally included thoughts undercutting the ongoing sentence. It's the moment you know that nothing you're reading has necessarily been seen by a human before or relates to reality.