I have been using LLMs & Coding Agent since early 2024. A large problem with Coding Agents & LLMs in general is context compression.
To give you some numbers, when I analysed my own sessions across Claude Code, Codex & Sakana, I found that most of my agents spent >90% of time re reading context and upon further investigation into the markdowns it was reading, I have a hand-wavy estimate of at least ~20% of this being useless to the task at hand.
When digging a bit more into this problem, I realised that this is an active area of frontier research, wherein some have even proposed solutions like having the LLM reason in an abstract compressed language illegible to humans which is more token efficient than human languages & then using a decoder model on top of this for human readability & access.
Curious to know, what other approaches are being used out there ? What is your experience of working with these agents & are you concerned about this "token-rot" as I call it or not ?
To give you some numbers, when I analysed my own sessions across Claude Code, Codex & Sakana, I found that most of my agents spent >90% of time re reading context and upon further investigation into the markdowns it was reading, I have a hand-wavy estimate of at least ~20% of this being useless to the task at hand.
When digging a bit more into this problem, I realised that this is an active area of frontier research, wherein some have even proposed solutions like having the LLM reason in an abstract compressed language illegible to humans which is more token efficient than human languages & then using a decoder model on top of this for human readability & access.
Curious to know, what other approaches are being used out there ? What is your experience of working with these agents & are you concerned about this "token-rot" as I call it or not ?