Yeah, I was a bit baffled by the author complaining about cache prefixes getting destroyed when more than one user hit the model, but then continuing to use llama.cpp instead of switching to vLLM.
A competent person does not summon every senior leader in his worldwide organization to be physically present for an hour in an auditorium while he blusters and attempts to deliver TV-ready one-liners. A competent person also does not take over a massive organization that relies on these senior executives’ decades of experience and immediately fire a non-trivial number of them because of their gender or skin color.
I just started going a few years back and I agree, it is really well done. One of my neighbors is the king’s main guard, so we get all sorts of behind the scenes anecdotes at the kids’ bus stop in the morning.
The website has some really good tutorials you can knock out in an afternoon that teach you the basics of using the interface, along with the more important stuff about GIS terminology and jargon.
We have some friends (husband and wife) that we met because our kids play together and they are the first people I’ve met that are licensed ham operators! They each have their operator number on their license plates for their respective cars since our state has radio operator plates you can upgrade to!
> the product occupies little space and it's trivial to keep around
Haha, I have found this to absolutely NOT be the case! Each individual mini only takes up and inch or so, but they multiply and between them and the brushes/airbrush/paint racks and the ever-increasing grey Pile of Shame, it’s not a small amount of space taken up!
It is an extremely rewarding hobby with a low bar to entry, though, and I agree that I will probably never stop.
This definitely works. I have two profiles on IG: one for musical instrument related things and one for painting miniatures. I’ve been able to keep both profiles strictly on topic by aggressively using the “not interested” button whenever something not related pops up.
Yeah, I’m specifically responding to the parent’s comment about the 48GB tier. When you’re looking in that range, it’s usually because you want to pack in as much vram as possible into your rack space, so consumer level cards are off the table. I definitely agree multiple 3090 is the way to go if you aren’t trying to host models for smaller scale enterprise use, which is where 48GB cards shine.
At the 48GB level, L40S are great cards and very cost effective. If you aren’t aiming for constant uptime on several >70B models at once, they’re for sure the way to go!
This is bad advice. Ollama may be “just a wrapper”, but it’s a wrapper that makes running local LLMs accessible to normal people outside the typical HN crowd that don’t have the first clue what a Makefile is or what cuBlas compiler settings they need.
Johns Hopkins University Applied Physics Laboratory | AI Software Engineer, AI Researcher | ONSITE - Laurel, MD
We are tackling some really tough problems ranging from judging ML model competency to teaching a system to make decisions based on fused data from multiple sensor modalities, as well as figuring out how to deploy the solutions to those problems on platforms in tons of different environmental conditions. Come join us to see your cutting edge research used in the real world!