> I may try moving it all to Pi, but I wonder if I should learn how to better configure the things there.
Honestly if the difference was marginal, I would move to Pi. I just tried layzpi and I had the agent write/compile a hello world in c and it required 20k in tokens. My minimal setup required 5.3k in tokens.
People will say the cost is minimal since this is cached but 15k is a lot since that needs to be reasoned by the LLM. I haven't looked at Claude Code but I read somewhere the system prompt is like 20k so I can see how Claude Code might have seem smarter as what you are working was probably addressed in the system prompt.
For me the bloat is not worth it since I am more interested in the LLM being able to reason better.
which is designed around the Pi philosophy of less is better by focusing on ondemand context/guidance. I won't bloat the context unless the LLM needs to do something I know it will need better guidance with. I have a demo repo for this at https://github.com/gitsense/gsc-rules-demos
One of the examples is, if I know the agent is reading a specific file, I will inject additional context. So if the agent never needs to do something in a certain file or directory, I don't need to pollute the context with "what it may need to know".
> One of the most annoying things I've seen AI do is accidentally pull in old requirements and start building off of them.
This is actually the reason for me wanting to create what I call "Brain checkpoints". After chatting with the agent and letting it do its thing for discovery, I needed an easy way to understand what it knows at any given point. What files it has read, what tool calls were made and so forth.
Checkpoints work extremely well in Pi since you can create a branch, and have the agent generate a checkpoint based on what has changed since the last checkpoint and this information will not pollute the main conversation but you benefit from the token caching.
I think it makes no sense to share the conversation but I do think developers will be required to provide lessons, notes and checkpoints in the conversation.
What I think will happen is you will be required to run the conversation through a company required prompt that will include things like, does the conversation contain "think harder" without providing any guidance for how to correct things.
Basically, did the developer treat the conversation like a literal slot machine.
> Nobody wonders out loud how a senior developer arrives at the result he does without an AI; they care that his deliverable is high quality and meets whatever standard/requirement exists.
I think the issue that many may not want to think about is, how do we promote/reward/fire developers in the AI assisted programming age. As part of code reviews, I think it will be second nature to ask "how did you arrive at the code".
The basic idea is, you always ensure you inject the instructions for how to use jj or any other scm. LLMs are pattern matchers and they can fully map behaviour, but due to heavy training on git, they can forget.
With hooks, you can easy detect when 'git' is used and basically tell the agent "As stated earlier you must use jj". So it will try again using `jj` and you block the agent when ever it tries to use git.
It is a wasted turn but the reality is, it won't cost must since you can leverage caching and you only waste tokens on the invalid command and not the output generated from git.
I will make another update by the end of this week that contains what I call "brain checkpoints" that will make it easier for developers to debug and understand AI reasoning.
The idea is after a task has been finished, you would commit lessons, notes, and "brain checkpoints" that are designed to live with the code.
Would usuability (for what herdr is optimized for) not be the bigger use case for users?
I never heard of this project but in 5 minutes I got the features that I wanted without having to introduce muscle memory. I just wanted a dead simple way to point and click to select different terminals. The only thing I neeed to remember now is "ctrl-b + v" to create a new pane.
> got essentially threatened by leadership for not using AI.
This sounds made up or your workplace is rather odd to say the least. Maybe english isn't your first language and "threatened" is not the correct word?
I makes navigating a 4000+ file repo extremely context efficient. It is important to note that the goal is too keep the context as clean as possible and not necessary speed.
Honestly this is where I would have fable generate a checklist and you just monitor opus to ensure it is going through the checklist. I think ignore is often the result of a context that is not focused enough.
I've found after the first hook injection they get it, but there are occasions it can forget, but since everything is driven by hooks, you can inject as often as needed.
The issue with skills is, they are a one time thing, so you really can't use skills to correct haviorial issues.
This is a hard problem, but one worth solving, I think, since it means less tokens and better AI reasoning. I believe LLMs are good enough that, if given the right context, it can very much solve almost all tasks.
If this works, it means we can probably get by with smaller models (since it doesn't need to know everything). LLMs are pattern matchers, and if you can provide them with the right shape (context), they should produce the expected output.
For my solution to work, you need business buy-in, which I don't think will be a problem. Enterprise wants to know how tokens are being spent, so I can see them wanting structured analysis during code reviews.
What may also not be obvious is that the information is ultimately designed to live with your code. Lessons and notes are designed to be mapped to files, so if you want to know why a piece of code is implemented in a certain way, you can have the LLM filter by files to help find the needle in the haystack.
It is a hard problem, but the only missing piece is discipline, which I believe business leaders will not have an issue with enforcing since we are ultimately talking about eliminating/significantly reducing the bus factor in our code.
> So I want to say there's still a lot of value in context engineering though it seems to diminish with each model release
I can't see how it would diminish unless you are literally working on public domain stuff. Unless stuffing context becomes cost effective and will not affect AI reasoning (this will be much harder), I don't see why context engineering is here to stay until we have close to AGI.