IME LLMs are kind of like a projection of your current expertise - your prompting and guidance etc. biases LLM plans kind of 'in the direction' of your thinking. I think this is one reason why it seems like senior engineers get more lift vs. juniors.
What I am exploring is another step to the classic 'research / plan / implement' pattern: 'research / plan / LEARN / implement' where LEARN involves the human doing AI tutoring sessions to ensure a deep understanding the concepts etc. that the LLM is planning to implement so you can refine / iterate on plans and direct the LLM in ever more effective ways. My idea is that this then compounds your human capital and reduces the occurance of 'sounds smart, doesn't work' pattern.
Error: Error during compaction: API Error: Claude Code is unable to respond to this request, which appears to violate our Usage Policy (https://www.anthropic.com/legal/aup).
Commoditize your complement - I expect to see this most in consumer AI (after that starts actually working...)
It will be important for Apple to have good enough, cheap local LLM models that run on-device.
If the barrier to performance shifts from fundamental model capability to context collection and management I would expect to see folks focused on that problem continuing to drive open-weight LLM model development in some shape or form.
My take is that B2C AI applications are kind of structurally limited by how hard it is to build personalized context.
The idea of capable local models could be a huge unlock here if they are able to do the bottom-up context collection research / tagging / etc. at scale.
I think the research / plan / execute idea is good but feels like you would be outsourcing your thinking. Gotta review the plan and spend your own thinking tokens!
Yeah I vibe coded a simple app that takes an org-mode file, renders it as a kanban board, and lets me spin up agents for each task with the prompt in the body in a named tmux session. The frontend gets updated via Claude code hooks when an agent is idle.
I think the key is to combine human and agent task tracking in one pane of glass.
I have been doing the same but with happy. It works quite well for quick brainstorms etc. but for deeper work on a real research / plan / implement thing I think you need to actually engage with the output which is hard to do on mobile. Maybe if I had a better UI than terminus to read and check the remote files I would be able to get more done.
I am also hoping / trying to put Claude code on top of a personal zettlekasten to automate more of my “personal life” tasks and get more stuff done for me. Haven’t gotten it really singing yet but I think that could also be really cool.
This isn't study mode, it's a different AI tutor, but:
"The median learning gains for students, relative to the pre-test baseline (M = 2.75, N = 316), in the AI-tutored group were over double those for students in the in-class active learning group."
Looks really cool - I noticed Enterprise has smart consent management?
The thing I think some enterprise customers are worried about in this space is that in many jurisdictions you legally need to disclose recording - having a bot join the call can do that disclosure - but users hate the bot and it takes up too much visibility on many of these calls.
Would love to learn more about your approach there
For me this benchmark suggests that an LLM will try to “force the issue” which results in compounding errors. But I think the logical counterpoint is that you may be asking the LLM to come up an answer without all of the necessary details? Some of these are “baked into” historical transactions which is why it does well in months 1-2.
My takeaway is scaling in the enterprise is about making implicit information explicit.
My question on all of the “can’t work with big codebases” is how would a codebase that was designed for an LLM look like? Composed of many many small functions that can be composed together?
What I am exploring is another step to the classic 'research / plan / implement' pattern: 'research / plan / LEARN / implement' where LEARN involves the human doing AI tutoring sessions to ensure a deep understanding the concepts etc. that the LLM is planning to implement so you can refine / iterate on plans and direct the LLM in ever more effective ways. My idea is that this then compounds your human capital and reduces the occurance of 'sounds smart, doesn't work' pattern.