Whoops. I've been using a fork of HN to manage my job interviews, and this time I accidentally submitted it to the real HN. I was very confused when I couldn't edit the story text.
Apparently you can no longer delete submissions, so I've deleted the Notion page.
Thanks for telling me about Inference Endpoints. That’s awesome.
So glad local models are getting good enough to be deployed. The uncensored model’s output was far better than expected in a domain that triggers guardrails with ChatGPT and Claude.
That is in fact what happened to me, except I think the final decision was made by a human since the ban came later. I didn’t issue any queries in between, so I know it was my convo about barbiturates.
It’s a PITA to offer a language model as a service. You’d need a beefy server, at minimum.
This particular use case might work, since no one can write fast enough to consume too many tokens — the whole session should fit in the context window. But you’ll need to handle all the people connecting to your service indefinitely, which will become expensive for a hobby project.
But sure, theoretically you could deploy it if you have resources. I’m not sure what you’d use to create instances of chat sessions, or if llama.cpp offers an API you can build the app on top of (probably) or whether that’s a workable solution.
As far as I can tell, Claude flagged me as high risk of suicide and then Anthropic issued a ban later on.
It wasn’t one prompt, it was a detailed conversation where I was trying to find out the exact dosage of barbiturates that assisted suicide programs use.
It would be ironic if you have to actually sign out from Twitter, not just use incognito mode, to bypass the signup nag. Or if they mark IPs as “signed up” or something. Thanks.
Every AI would refuse the prompt. I was banned for researching Nordic assisted death and asking which drug exactly they administered (and what quantity). Claude refused, alerted Anthropic, and I was banned a couple days later. Thankfully the appeal form worked, but by then I was using a different Claude account praying they didn’t ban me again.
There’s an uncensored model floating around that you can run locally with llama.cpp: https://www.reddit.com/r/LocalLLaMA/comments/1rq7jtm/qwen353... it’s annoying to use since you run out of context window quickly, and it’s certainly not able to be deployed in production (i.e. Tom Riddle’s diary as a service).
For better or worse, fun is no longer allowed. It coincided with “AI psychosis” being coined as a term.
Because Twitter posts consisting of YouTube links die quickly. It’s very obvious once you have a few thousand followers.
Your option is basically either upload to twitter, or put the YouTube link at the end just before a screenshot. Or both a video and a YouTube link, I suppose.
If you trigger their YouTube embed, it seems like it gets penalized quite harshly. I’ve seen other people agree with the sentiment.
They’re saying if the average task you actually use the model for is far less difficult than the benchmarks, you might incorrectly conclude that the model is costly when in fact it’s the best performing model for your actual use case.
I want a model that generates commit messages fast. Currently I have to wait up to a minute or two. That model doesn’t need to score very highly on SWE benchmarks, just highly enough that it can write out a good enough message in a few seconds. If you tested it on ${current top tier benchmark} you’d think it’s way too costly when in fact it’s the best tradeoff.
I was confused too. What they’re saying is, the average task you’re likely to do if you buy the model is the main predictor of costs. So if the average task benchmark is far higher than what you’re normally doing with it, you get a skewed perspective.
I’ve wanted a fast model to generate commit messages. No idea what that would be, but it doesn’t have to pass the SWE benchmarks very well. Just well enough that it understands the codebase.
Concrete example: I’ve been trying to use Claude to generate all my commit messages, but it takes 5-10x longer than if I just write them myself. Mine are less detailed, but one line changes are sometimes inconsequential (especially white space reformatting). I wish there was a model that understood the codebase well enough to generate commit messages in half the time.
Greenboots is so iconic. Other people use him as a marker. Glad he got some attention. It’s always seemed a shame that it’s impossible to give him a proper burial.
email: [email protected], but twitter DM is a more reliable way to reach me. I’m happy to talk!
website: https://shawwn.net
twitter: https://x.com/theshawwn
github: https://github.com/shawwn
Previous HN usernames: sillysaurus3, sillysaurus2, sillysaurus, shawn.
ML discord: http://discordapp.com/invite/x52Xz3y