HackerLangs
TopNewTrendsCommentsPastAskShowJobs

Zetaphor

1,037 karmajoined 11 anni fa
Software Engineer

zetaphor at hey dot com

Submissions

The local LLM ecosystem doesn’t need Ollama

sleepingrobots.com
648 points·by Zetaphor·3 mesi fa·207 comments

comments

Zetaphor
·l’altro ieri·discuss
It's far less obvious how choosing Bun results in more revenue for Anthropic. Unlike Claude Code, Bun doesn't require you to pay for tokens to use it
Zetaphor
·5 giorni fa·discuss
https://qntm.org/mmacevedo
Zetaphor
·11 giorni fa·discuss
Alternatively you could run it on Strix Halo for $1,000 less, and while it may be slightly slower you won't have to deal with NVIDIA's shit on Linux and worrying about having to use their custom kernels or Ubuntu.
Zetaphor
·13 giorni fa·discuss
Hello from the US! I'll never not be amazed by the fact that we live in a point in human history where language and distance are no longer barriers to the exchange of ideas, despite the efforts of our governments.
Zetaphor
·14 giorni fa·discuss
Please use the upvote button instead of doing this.
Zetaphor
·14 giorni fa·discuss
Because that has worked so well for:

* Drugs

* Media piracy

* Alcohol

* Sex work

* Unlicensed gambling

The government is not an all powerful entity with absolute control over its people. Even in countries under past and present dictatorship there are examples of people getting access to what the government deemed as illegal.
Zetaphor
·18 giorni fa·discuss
There's no game worth installing a rootkit for. The market is far more diverse and interesting than the handful of popular AAA titles , most of which are just shooters
Zetaphor
·18 giorni fa·discuss
On Linux I can play games going back multiple generations as well as emulating other consoles
Zetaphor
·18 giorni fa·discuss
Piracy and alternative storefronts don't exist on PS5. I could buy a steam machine and then exclusively buy my games from EA. They're also not charging you for the multiplayer access
Zetaphor
·18 giorni fa·discuss
Exactly, no game is worth installing a rootkit for
Zetaphor
·19 giorni fa·discuss
Good news!

https://9to5linux.com/kdes-new-css-based-style-engine-union-...
Zetaphor
·26 giorni fa·discuss
I'm someone who uses tens of millions of tokens each month, almost exclusively with open weight models that I run on my own hardware. That said you are taking the wrong approach here, this type of mentality is only going to further radicalize those who have decided they're against this technology.

Additionally when the finally bubble bursts and the executives wake up from psychosis and look to distance themselves from this because it's become a dirty word, you'll be one of the first to go. The nail that sticks out gets hammered down and all that.

I do think there are real benefits and productivity gains with this technology, but it does not benefit everyone equally. It's great for the programming parts of my job, but useless in the other 40% of the work. I have coworkers for whom generative AI has no obvious practical application, and yet management is trying to find a way to shoehorn it in anyway. No doubt because they've also drank the kool-aid and are eager to reduce headcount.

This attitude of it making everything more productive and anyone who doesn't follow will be left behind is not just false, it's cruel and myopic. You're talking about people's livelihood being taken away because a handful of executives decided this is how things should work despite the MASSIVE number of shortcomings and poor product market fit.

Edit: I also almost missed where you're seemingly celebrating the devaluation of human labor as a result of this. Please stop and reflect on how your position may read to someone who is just trying to put food on the table.
Zetaphor
·28 giorni fa·discuss
That's not really how the experts in an MoE work. They activate on token probabilities and are activated on every token. You don't necessarily have a discrete math expert and a discrete physics expert. And if it were you would still need a router that is trained on all of those domains.
Zetaphor
·mese scorso·discuss
I am unfamiliar with his work, can you provide sources for his past incorrect predictions?
Zetaphor
·mese scorso·discuss
Something powered by an LLM is going to end up being the tool that makes this accessible in the way it always should have been, and that gives me complex feelings.
Zetaphor
·mese scorso·discuss
This was covered in the issue itself, in fact the issue is pretty well documented with regards to the packaging:

> Publish an official Claude Desktop build for Linux, targeting the two current Ubuntu LTS releases (and Debian) as a signed .deb via an Anthropic-operated apt repository, using the same distribution pipeline Claude Code already uses for Linux.

Also Flatpak or AppImage would make this accessible to every other distro. Alternatively you could run the deb with a Podman Toolbox.

Your point about backwards compatibility with Windows goes both ways, I have old games that I can _only_ run on Linux as they don't work on modern versions of Windows.
Zetaphor
·mese scorso·discuss
Qwen 3.6 35B-A3B and 27B both at Q8 on a Strix Halo machine
Zetaphor
·mese scorso·discuss
You're supporting the developers original work at that price. There's plenty of cheaper devices that take that original work and just throw it on some chips
Zetaphor
·mese scorso·discuss
I want to reduce my dependency on companies like Google, OpenAI, and Anthropic. Aside from the concerns of data sharing I'm also not a fan of how they run their operations, for example Anthropic now using xAI's Colossus data center which is poisoning a marginalized community, or OpenAI getting in bed with the military.

Not everything I want to use an LLM for requires "PhD level intelligence", and increasingly I'm finding more uses that involve sharing my personal data.

Yesterday my local model helped me when looking for a doctor who is in-network for my insurance. I threw it a screenshot from the providers search results and it looked up reviews for all of them.
Zetaphor
·mese scorso·discuss
> Daniel Lemire’s blog is one of the top 50 most popular blogs on Hacker News, the standard tech news aggregation site.

Citation needed