> Having a CI job that identifies places where the docs have drifted from the implementation seems pretty valuable.
Testing with lat isn't about ensuring consistency of code with public API documentation. It is about:
* ensuring you can quickly analyze what tests were added / changed by looking at the English description
* ensuring you spot when an agent randomly drops or alters an important functional/regression tests
The problem with coding agents is that they produce enormous diffs, and while reading tests code is very important in practice your focus and attention drifts and you can't do thorough analysis.
This isn't a new problem though, the same thing applies to classic code reviews -- rarely coding is a bottle neck, it's getting all reviews from humans to vet the change.
Lat shifts the focus from reading test code to understanding the semantics of the test. And because instead of reviewing 2000 lines of code you can focus on reviewing only 100 lines change in lat.md you'll be able to control your tests and implementation more tightly.
For projects where code quality isn't paramount I now just glance over the code to spot anti-pattern and models failing to DRY and resorting to duplicating large swaths of code.
One anecdote: I added lat to quickjs codebase and asked questions like "how does quickjs gc circular object graphs" and was able to get an answer in 12 seconds vs 2 minutes without lat. But take this with a grain of salt, I need better benchmark harness. Benchmarks are hard to do in general, for LLMs it's its own hell. :)
I'm working on a blog post and on benchmarks. Here [1] Armin suggested I take something like quickjs, built lat base for it, and compare side by side how, say, claude code works with lat vs. without.
I'm very early into this and need to build proper harness, but I can see sometimes lat allowing for up to 2x faster coding sessions. But the main benefit to me isn't speed, it's the fact that I can now review diffs faster and stay more engaged with the agent.
> I suspect there's ways to shrink that context even more.
Yeah, I'm experimenting with some ideas on that, like adding `lat agent` command to act as a subagent to search through lat and summarize related knowledge without polluting the parent's agent context.
I think it's a great idea and I'm considering building this in lat too. Code embedding models can definitely speed up grepping further, but they still wouldn't help much when you have a business logic detail encoded across multiple complex files. With lat you'd have it documented in a paragraph of text + a few [[..]] links into your code.
Because lat gives agents more tools and enforces the workflow.
Unlike obsidian, lat allows markdown files link into functions/structs/classes/etc too.
This saves agents time on grepping but also allows you to build better workflows with tests.
Test cases can be described as sections in `lat.md/` and marked with `require-code-mention: true`. Each spec then must be referenced by a `// @lat:` comment in test code. `lat check` flags any spec without a backlink, so you can review and maintain test coverage from the knowledge graph.
> My concern is scale though. Once you have thousands of nodes the Markdown files themselves become a mess to navigate
The agent will update the graph.
If you have thousands of nodes in md it means you have a highly non-trivial large code base and this is where lat will start saving you time - agents will navigate code much faster and you'll be reviewing semantic changes in lat in every diff, potentially suggesting the agents to alter the code or add more context to lat.
You still have to be engaged in maintaining your codebase, just at a higher level.
The problem is that for any non-trivial question agents have to grep A LOT to understand the high-level logic expressed in code. In big projects grepping can take minutes some times. Lat short-circuits grepping significantly with `lat search` that agents happily use.