Am working on making this layer currently. It’s a more interesting problem even when you remove AI agents from the picture, I feel a context layer can be equally as useful for humans and deterministic programs. I view it as a data structure sitting on top of your entire domain and this data structure’s query interface plus some basic tools should be enough to bootstrap non trivial agents imo. I think the data structure that is best suited for this problem is a graph and the different types of data represented as graphs.
Stitching api calls is analogous to representing relationships between entities and that’s ultimately why I think graph databases have a chance in this space. As any domain grows, the relationships usually grow at a higher rate than the nodes so you want a query language that is optimal for traveling relationships between things. This is where a pattern matching approach provided by ISO GQL inspired by Cypher is more token efficient compared to SQL. The problem is that our foundation models have seen way way way more SQL so there is a training gap, but I would bet if the training data was equally abundant we’d see better performance on Cypher vs SQL.
I know there is GraphRAG and hybrid approaches involving vector embeddings and graph embeddings, but maybe we also need to reduce API calls down to semantic graph queries on their respective domains so we just have one giant graph we can scavenge for context.
I've had this same thought. Given humans were the primary authors of code, the tooling has optimized for writing (IDEs, Text Editors, LSPs, etc). Even though we all know we read more code than write, the tooling space for reading code is rather small, I can only think of stuff like SourceGraph and Github. I'm sure big companies have internal tooling for this, but even then I feel the best experience exploring a single repo today is your EDITor.
I'm also experimenting with building a platform that is optimal for reading code particularly distributed systems since they have many non-local dependencies and are harder to explore locally in code editors/viewers. I def fall in the camp that current AI tools can probably help us understand our systems better than they can contribute code to them. One win of using AI to help us analyze large codebases is that if they can extract useful things for us, they can also extract useful things for their own agentic loops.
Let me know if you wanna chat more about this, would love to bounce some ideas/contribute!
I’m actually interested in solving the documentation problem. Imo we as engineers are thinking too small and keeping docs as this side thing sounds like a recipe for irregular maintenance. Instead, what if docs were more like live blueprints of running systems? We don’t want obvious stuff documented like there is a function called foo, but foo’s relationship to other parts of the code and its runtime characteristics seem important. I think I’m imagining a different form of documentation that is tied with observability but that’s because I feel it’s information that’s far away from code currently and ideally I’d like all information derived from a piece of code to be available at the same place.
Probably slightly off topic but I’d be curious to hear what other people want out of automated systems in this space. I have so many half baked ideas and would love to hear what’s others think/want.
Stitching api calls is analogous to representing relationships between entities and that’s ultimately why I think graph databases have a chance in this space. As any domain grows, the relationships usually grow at a higher rate than the nodes so you want a query language that is optimal for traveling relationships between things. This is where a pattern matching approach provided by ISO GQL inspired by Cypher is more token efficient compared to SQL. The problem is that our foundation models have seen way way way more SQL so there is a training gap, but I would bet if the training data was equally abundant we’d see better performance on Cypher vs SQL.
I know there is GraphRAG and hybrid approaches involving vector embeddings and graph embeddings, but maybe we also need to reduce API calls down to semantic graph queries on their respective domains so we just have one giant graph we can scavenge for context.