This is so easy to do because we are unable to contemplate every edge case at the time of inception right? At least that is what I am telling myself as I brick myself on a "manifest driven system" that surely will detect drift as I bolt more and more on, right? no... bricked as fah.
At least I am learning to build modular so I can reuse parts like image gen, audio gen (STT/TTS), knowledge management. I have probably built 4 of these systems in the last year, each one gets better and lasts longer until I brick the crap out of it. Super fun.
It has been this way since the beginning, unfortunately. There is certainly no harm in trying on local models on local workloads with modest guardrails.
Like most of these models (Qwen, Gemma, Llama, gpt-oss), finding all the little gotchas like, special tokens and prompt structure, model preference are a PITA right now. The reward are really nice models that run exceptionally well in agentic harnesses tuned with the prompts and parameters you fought so hard to learn.
My thinking is totally aligned with yours, perhaps its because I am trying to do a second act at almost 50 from blue-collar to white collar office work. I have no formal degree, but I have been hobby programming for 20 years. I have made a habit of "letting myself be available to all lessons"... the localllama group has made this journey really fun if nothing else. I have learned an ABSOLUTE ton from this era!
Not really an article, but yeah, I was hoping they went into the underlying mechanism a bit deeper. This paper could be confirmation of what localllamaians have been saying for months; Keep your harness surface small, allow the model to use the harness to build _your workflow_.
I have been doing a LOT of work around this with Qwen3.6 and its been super fun. There are some neat benchmarks that help guide, but nothing beats reading the output... and there is a lot of output to read when trying different quants, etc. Which leads me too...
The other thing I have learned is the "harness" is only as good as the model tuning that goes into it. If your prompt(s) are buggered from the beginning, you are going to have a bad time. The prompt structure and special tokens can be a PITA or really help depending on how much you know.
I don't know how agentic harnesses can work without being optimized for the models running within them. This is the biggest insight into working with agents for me. First thing I have always looked at were the prompts and parameters... everything else is orchestration to me.
Disclaimer: I am not a professional dev, but I have dabbled as a hobby dev for years. My day job is wayyyy different.
I receive at least 10 python job ads a day from indeed and linkedin. Are any of them legit? I have no reason to apply, so I am curious if they are bullshit or not. Any insight for this poorly constructed question?
I don't understand why so many people focus on Trump and Left and Right and all the theater of politics in general. Subtract the politics from TFA and it is really, really good.
The author and I agree on some "pop-stoicism" critiques and disagree on others. Well reasoned and articulate arguments to support or dismiss "teachings" from the neo-pop-stoic culture.
The we get to passages like this; "It is, I suppose, strictly speaking accurate that if the approximately 8.6 million people who die each year due to a lack of access to quality healthcare were to wish their fate, their desires would not be frustrated, but tautological truth does not make for philosophical profundity."
Yes it does. How does this person know the 8.6 million people that died did not live meaningful lives while they were able bodied and healthy? How does the author know what is "right" for them? Whether I agree/disagree with the aspects of the authors POV; any sober and/or objective reading of this just reeks of ego and "holier than thou" attitude.
I suspect this type of attitude is a large reason why people that devote large amounts of time to thinking about politics end up categorizing "justice" into politically ideological boxes.
Edit to add;
I am reading the comments and not many are talking about this point either which I think is profound.
I hit ctrl-f inside TFA and did not find a word used in Stoic literature that I have read; Virtue.
You would think a critique on Stoicism would at least cover the basics, no?
Here is a reminder for everyone that cares;
Stoic virtue is the highest good and the only true path to a flourishing life, encompassing four cardinal virtues: wisdom, courage, temperance (or self-control), and justice.
I hope you get traction with your letter, it is vitally important and one of the main reasons I stopped using Obsidian.
I had maybe 10 plugins and they managed to step all over each other. One would break sync, another would break some theme and so on. There is nothing to ensure they play nice with the system itself, let alone anything malicious as you pointed out. From what I understand, there are people that have upwards of 50 plugins installed! They must never shut it down because it probably takes 20 minutes for it to start with that many plugins.
I will say that I am experimenting with various PKMS software. I am running SiYuan and it is has been very pleasant. I have only installed one plugin to make cool indexes on folders as folder notes. Everything else can be configured and used as is out of the box.
There are several others that I will try in the coming months as well. Anytype, Flusterapp, Joplin and Craft are going into my evaluation cue.
Nailed it. To make matters worse, Ollama obfuscate the models so their users don't really know what they are running until they dig into the model file. Only then can they see that what they thought was Deepseek-r1 is actually an 8B qwen distillation of Deepseek-r1, for example.
Luckily, we have Jan.ai and LM Studio which are happy to run GGUF models at full-tilt on various hardware configs. Added bonus; both include very nice API server as well.
I can not answer for GG, but the early days of llama.cpp were crazy and everything was so very hacky. Remember, Textgen-webui was 'the way' to run models at first because it supported so many different quant types and file extensions. At the time, most people were using multiple different quantization methods and it was really hard to figure out which were performing better or worse objectively.
GGUF/GGML was like the 4th iteration of file type quantization from llama.cpp and I remember that I had to consciously begin watching the bandwidth usage from my ISP. Up to that point, I had never received an email warning me about reaching limits of my 2TB connection. All for the same models just in different forms. TheBloke was pumping out models like he had unlimited time/effort.
I say all that to say, llama.cpp was still trying, dare I say 'inventing', all the things throughout these transitions. Ollama comes in to make the running part easier and less CLI flag dependent building off of llama.cpp. Awesome.
GG and company are down in the trenches of the models architecture with CUDA, Vulkan, CPU, ROCm, etc. They are working on perplexity, token processing/generation and just look at the 'bin' folder when you compile the project. There are so many different aspects to make the whole thing work as well at it does. It's amazing that we have llama-server at all with the amount of work that has gone into making llama.cpp.
All that to say, Ollama shit the bed on attribution. They were called out on r/localllama very early on for not really giving credit to llama.cpp. They have a soiled reputation with the people that participate in that sub-reddit at least. They were called out for not contributing back if I remember correctly as well, which further stained their reputation among the folks who hang in that sub-reddit.
So it's not a matter of "ease" to build what Ollama built... At least from the perspective of someone who has been paying close attention from r/localllama; the problem was/is simply the perception (right or wrong) of the meme; Person 2 to person 1: "You built this?" -> Person 2: takes item/thing -> person 2: Holds up item/thing -> "I built this". A simple act that really pissed off the community in general.
How do you know you have the right application to open the file? I have tried at least a dozen apps to open *.stl files... some work, most don't. Some have features I need and don't work opening files. Some are sensitive to stl's that require repair and stop working... and I could go on and on for different file types. The point is; sharing isn't "guaranteed" to work across apps and is generally far, far from "it just works". You have almost zero control over the actual files because there might not be a way to save different formats from the app you are using, or the files app doesn't recognize the extension, or you can't find the folder for the other app, or it doesn't have a user accessible folder, or it will not load files from the "files app"... or <insert 50000 more "what if's here>. The whole things just sucks so, so bad.
Do you use "files"? Its garbargio on steroids. You can't just open a <insert about 100 file extensions here> file from the files app. Moving files around is cumbersome at best and downright infuriating most of the time. The "sandbox" nature of iOS is simply not intuitive enough to know how things will react when you move and try to open a file in another app. It just sucks so bad.
Furthermore, "sharing" is broken. Does it copy the file? Does it move the file? Am I duplicating this 200mb pdf when I move it to books? How the ____ do I know? There is a dearth of information and I imagine most people, like myself, give up and use it to read before bed or watch a few videos on the couch. I am never going to by another iPad until the OS is useful beyond drawing, creating music or reading.
My parents use their iPad(s) for 100% of their compute needs. At 70+ years old, they will tell you those needs are minimal.
If you use one program at a time, do not need an actual file system, have no need to install software from a variety of places (Github, Vendor sites, etc), have no problem installing multiple "apps" that only work behind paywalls or not at all and you don't care about replacing a functional device whenever Apple obsoletes it... iOS is the best thing since sliced bread.
If you need anything outside of iOS's limited list of abilities, its a trash operating system that has crippled amazing hardware.
That is just one very simple part, connecting the mouse. Literally everything else sucks on iOS. File management, hidden menus, running multiple apps, system management... and the list goes on. Need to convert a STEP file to something else on the iPad? Download 15 apps to see which one works, then try to find the converted file in the abomination of a file system? iOS is hot garbage.
I have been programming as a hobby for almost 20 years. At least for me, there is huge value using LLM's for code. I don't need anyone else's permission, nor anyone else to participate for the LLM's to work for me. You absolutely can not say that about blockchain, nft, or crypto in general.
http://caffeineindustries.com - I am a full time firefighter/EMT that moonlights with Python and the web.