Coding agents have changed how I build. Constantly switching between the terminal and an IDE started to feel inefficient, so I wanted a better terminal-first setup where I could manage multiple agent sessions and make quick edits without the overhead of a full IDE. So I built Helm for myself: https://github.com/samirkhoja/helm
Coding agents should help us reduce dependencies overall. I agree Go is already best positioned as a language for this. Using random dependencies for some small feature seems archaic now.
Hot take - Local LLM computing will move to stationary, always on devices (Mac mini & studio). Developers and users will move to lighter, portable devices to interface with their long running agent workers (MacBook Airs & iPads).
Codex built my old startup in a weekend. What took me a few months to build 2+ years ago can now be done with coding agents in 2 days. The last couple months feel like a step change with Codex and Claude.
Autonomous agents could be the next evolution of AI, extending the capabilities of LLMs. Interested to see which agent implementations seem the most promising today.
This is really interesting. Could this lead to eventually being able to deconstruct these "black-boxes" to remove proprietary data or enforce legal issues?
This wasn't mean't to say that all training would stop. I think, to some extent, the model won't need additional recent data (that is already similar in structure to what it has) to better understand language and interpret the next set of characters. I could be completely wrong, but I still think techniques like transformers, RLHF and of course others will still exist and evolve to eventually get to some higher intelligence level.
OpenAI doesn't let you fine-tune GPT-4 or GPT-3.5 yet (https://platform.openai.com/docs/guides/fine-tuning), but fine-tuning models on a set of documents is still an option but not really scalable if you want to keep feeding it more relevant information over time. I guess it could depend on the base model you are using and its size.
The article is definitely still high level and mean't to provide enough understanding of what capabilities are today. Some of what you are mentioning goes deeper on how you take these learnings/tools and come up with the any number of solutions to fit the problem you are solving for.
> "Do you use the whole document as context directly? Do you summarize the documents first using the LLM (now the risk of hallucination in this step is added)?"
In my opinion the best approach is to take a large document and break it down into chunks before storing as embeddings and only querying back the relevant passages (chunks).
> "What about that trick where you shrink a whole document of context down to the embedding space of a single token (which is how ChatGPT is remembering the previous conversations)"
Not sure I follow here but seems interesting if possible, do you have any references?
> "What about simply asking the LLM to craft its own search prompt to the DB given the user input, rather than returning articles that semantically match the query the closest? This would also make hybird search (keyword or bm25 + embeddings) more viable in the context of combining it with an LLM"
This is definitely doable but just adds to the overall processing/latency (if that is a concern).
Agreed, GPT answering based on its own training data has been the best experience by far (aside from hallucinations) and comparing against that is difficult. Embeddings might not even be the long term solution. I think it's still early to really know for certain but models are already getting better at interpreting with less overall training data so there are bound to be some new ideas.