Thanks!! I had disabled that previously while debugging, I can confirm this is helping accuracy from what I can tell so far. (And speed since the cache is preserved more often!)
I'm running Qwen 3.6 27B Q5 K M GGUF on a Tesla P40 and koboldcpp using pi.dev as the harness, I gotta say I am impressed. Took some setup and configuring but I already have some code it has made commited and pushed. It can be slow on my hardware at >50k tokens, but the fact I bought this one P40 for like $150 back when the LLM trend started I can't complain. (I have a second one too but I couldn't physically fit the card in my server unfortunately.)
The setup I had to do was important and I had to compile koboldcpp with a few special params for my hardware, I mostly just had Claude figure it out. I don't remember everything I did now but it was very slow and would often stop mid task, it seems it was mostly a parsing issue. It made the model seem broken/dumb, but once I had all that settled I actually am able to use this how I use Claude Code. Disclaimer, I am pretty explicit with requirements, I imagine this fails more when you leave it to figure out things on its own but for my flow its pretty rad.
Currently setting it up as an automated agent now to pull Trello cards, create PRs for them, and move the card to be reviewed.
Command I am using to run:
python koboldcpp.py \
--port 61514 --quiet --multiuser --gpulayers 999 --contextsize 262144 --quantkv 2 \
--usecublas normal --threads 4 --jinja --jinja_tools --jinja_kwargs '{"enable_thinking":true, "preserve_thinking":false}' \
--skiplauncher --model /data/models/Qwen3.6-27B-Q5_K_M.gguf --smartcache 5
I am the dev of Pocket Squadron (https://play.google.com/store/apps/details?id=com.bombsight....) and a few years ago I tried to make a build for iOS due to many player requests. I did not have a mac so I setup a mac VM and a dev account to start making builds and see how big of a lift it would be. My account was banned unfortunately. Still no iOS build to this day, I'm probably missing out on a good bit of money.
This is true, I think its good to find a balance between living life and making sure you at least take some good memories.
I have had a number of times I just don't remember a moment, only for a friend to show an old photo they happened to see again and a flood of good memories come back.
This is the difference between science and engineering. What they have done is engineering. If the result is 90% of the way there with barely any effort, its best to move on to something else that may be low hanging fruit than to spend time chasing that 10%.