My system is quite similar to your, my GPU is a 6950 XT and CPU a Ryzen 5 2600x, same amount of RAM, and I feel your pain. It sounds very similar to my experience from a few months ago. When it comes to tool calling, there are multiple possible issues; some models have borked templates bundled with the model file, some models are not trained on tool calling, some agent harnesses doesn't support the tool call output from the model very well, some quantizations ruin the models' abilities to call tools.
My suggestions if you want to further experiment with local models are to use llama.cpp instead of ollama [1], learn a little about the parameters that tune how much VRAM is used [2], look online for jinja template fixes for the model you're testing [3], and choose a model that was designed to do the task you want to achieve, with as high quantization as you can fit. The maximum model size you can run is VRAM + RAM, although you want as little of the model to be in system RAM as possible.
I'm running North Mini Code IQ3_XXS with some tuned parameters to fit my current tasks, and while it is not perfect for everything, it has not failed any tool calls I've asked it to make, or that it figured it should make on its own.
I'm in the same boat as you, and every time I touch a Windows PC I am again and again surprised at all the junk they put in there by default. Not only Microsoft, but also the OEMs. So much junkware, so many accounts they want you to create, and all without any benefit to the user.
Thanks for the link and tl;dr, even though it's a quite short video (3:16). I found your explanation very interesting because I have intuitively felt like this is accurate, but never knew what the underlying process is. I have been following this for years already, first absorbing information about anything where I need to make a decision, and then just leave it to stew in the back of my mind. And after a while the answer just appears in my mind, without me really understanding where it comes from.
There are probably several libraries that offer this functionality, the question is just what your other requirements are. It is also built into most modern web browsers, just check out the MDN for Intl.DateTimeFormat with the dayPeriod option [1].
I think that that is supposed to prepare web apps for verification in the browser, in order to not allow them to run or connect to a remote server unless they are confirmed to be not tampered with, and that the main goal of this is to disallow usage of websites and related services unless ads are served. An adblock-blocker in the browser, sold as a security feature that protects against no real threats.
I refuse to believe that rouge browser extensions and userscripts are such a big problem that Meta decides to invest in security against those attack vectors.
My suggestions if you want to further experiment with local models are to use llama.cpp instead of ollama [1], learn a little about the parameters that tune how much VRAM is used [2], look online for jinja template fixes for the model you're testing [3], and choose a model that was designed to do the task you want to achieve, with as high quantization as you can fit. The maximum model size you can run is VRAM + RAM, although you want as little of the model to be in system RAM as possible.
I'm running North Mini Code IQ3_XXS with some tuned parameters to fit my current tasks, and while it is not perfect for everything, it has not failed any tool calls I've asked it to make, or that it figured it should make on its own.
[1]: https://sleepingrobots.com/dreams/stop-using-ollama/
[2]: https://github.com/ggml-org/llama.cpp/blob/master/tools/serv...
[3]: https://gist.github.com/jscott3201/e4b155885cc68c038d6ac8909...