"For AI agents". I understand why everything needs to be marketed in this way, but it's just ... an easy-to-generate language for expressing charts. That's impressive! That's useful.
Speculating here, but perhaps your coworker was too ambitious? In my opinion, you should start with AI-generated PRs that do small, linting refactors and then work up from there. In particular, if this is done in parts, one of the strategies you can employ is to:
- add tests
- break files up into smaller parts
- test the smaller parts
- then actually improve behavior
(Which is no different than what you would do as a human)
I'll add another use case for letting an AI go ham: many small, atomic refactors where the name of the game is never breaking anything.
My personal OSS projects don't have the scale to necessarily make this worth it, but at work I run three pipelines using Barnum (https://barnum-circus.github.io/). First, one that ingests files, identifies refactors (from a pre-approved list), and places a precise description of the refactor to be done in a queue; second, one that reads from said queue, implements and creates PRs (there is a lot of "check that the PR is correct" here as well); and a third that babysits PRs until they land. I've landed hundreds of PRs in this way, with very little effort on my part.
Folks that are interested in a way of doing work locally that doesn't suck, but which integrates LLMs, may be interested in [Barnum](https://barnum-circus.github.io/). The TLDR is that it's a programming language whose frontend is a DSL in TypeScript that is well suited for managing async and parallel work, focused on control flow, from which it is easy to invoke LLMs, and which is easy for LLMs to write. I use it to autonomously ship a very large number of PRs.
That's exactly the tradeoff I made with Barnum (https://barnum-circus.github.io/). It's just not important to optimize the performance of the rust side for the reason you stated. So instead, all focus goes into making it easy for an LLM to build a reliable pipeline (from which LLMs are invoked).
Hey folks! This talk is about GraphQL in a world of fullstack, rich clients and about Isograph. The question it asks is: does GraphQL need to exist? Can we get its benefits without a GraphQL schema, without a GraphQL server, and without sending GraphQL over the wire?
It's my opinion that Isograph gives us the benefit of GraphQL, without many of its limitations. Which is to say, if you start from scratch, you can avoid mistakes.
This describes a future iteration of Isograph. Currently, much of what's described here is on the roadmap. But it's coming!
A simple task ("convert this file from JS to TS, here are the types of all imported things") is much more likely to continue to work with a nerfed model compared to a complicated task ("convert this repo to TS, make sure to run tsc afterward and fix all errors"). The former is a subtask of the latter!
Taking a moment to create a workflow where these steps are separated (or rather, having an LLM build this workflow) and the LLMs are asked to just do minor leaf tasks increases your resilience to nerfed models.
You may want to check out Barnum, which is a programming language/agent orchestration tool that makes it easy to build things like /loop, or Claude code routines. And you won't end up dependent on the specifics of how Claude code routines work!
You can lessen your dependence on the specific details of how /loop, code routines, etc. work by asking the LLM to do simpler tasks, and instead, having a proper workflow engine be in charge of the workflow aspects.
For example, this demo (https://github.com/barnum-circus/barnum/tree/master/demos/co...) converts a folder of files from JS to TS. It's something an LLM could (probably) do a decent job of, but 1. not necessarily reliably, and 2. you can write a much more complicated workflow (e.g. retry logic, timeout logic, adding additional checks like "don't use as casts", etc), 3. you can be much more token efficient, and 4. you can be LLM agnostic.
So, IMO, in the presence of tools like that, you shouldn't bother using /loop, code routines, etc.
I'm as nerdy as they come (my current project is the fourth compiler I've worked on), and I absolutely love this new way of working. There's a lot more time spent in discussion with the agent (an extremely frustrating discussion, to be fair). All of a sudden, there's an extremely high payoff to investing in good fundamentals (namely, clarity of requirements, good tools, etc.), which are the things I want to invest in anyway! If you get these fundamentals right, you can let the agent rip and produce hundreds of PRs that are correct, or create workflows that are actually not slop or ship code that is, while not yet as high quality as if you wrote it manually, quite close, at easily five times the speed.
And throughout this, if I'm ever curious about how the ideas relate to some other topic, I can just ask the agent, "Are we designing XYZ right now? Categorically, is it this?" Lots of really cool discussions to be had.
I might be less enthusiastic if I was just shipping CSS changes and the like.
I agree. I think we simply don't have the tools yet to hold agents to that high architectural standard. It simply takes a lot of focused effort and berating and close comprehension of the code at the moment to ship anything good, but there are lots of people (myself included) working on that problem. I'm pretty sure in a matter of months it will be solved.
I think what you're implying is that the agent ships unmaintainable slop. Certainly, if I don't pay attention and review the code line by line, it will ship slop. And even sometimes, when I'm certain that it is implemented one way, I'll come back to the code many days later and discover that it went a completely different route than I expected. Very frustrating.
But it doesn't have to be that way. You just have to put an effort into shipping fewer, better features as opposed to more features. The projects I'm working on (e.g. agent orchestration, because who isn't nowadays) have a small surface area and high payoff and thus are uniquely well positioned for this.
If I couldn't use an LLM, I would still work on this, and it would have roughly the same architecture. But because I'm able to go probably 100x as fast, I'm able to be much more ambitious. Or rather, I'm able to discover that my initial ideas were not on point and pivot, and not have any sense of sunk cost
The skill atrophy point strikes me as tenuous at best. Obviously, the plural of anecdote is not data, but I find myself able to work on projects of greater complexity than I would have been able to otherwise. 90% of my time is spent going back and forth on Markdown files, discussing the architecture, trade-offs, etc. I don't think it's necessarily impossible to use all this newfound power to ship more sloppier code. It's clearly possible to use all this newfound power to ship better code too.
- you don't have a normalized cache. You may not want one! But if you find yourself annoyed that modifying one entity in one location doesn't automatically cause another view into that same entity to update, it's due to a lack of a normalized cache. And this is a more frequent problem than folks admit. You might go from a detail view to an edit view, modify a few things, then press the back button. You can't reuse cached data without a normalized cache, or without custom logic to keep these items in sync. At scale, it doesn't work.
- Since you don't have a normalized cache, you presumably just refetch instead of updating items in the cache. So you will presumably re-render an entire page in response to changes. Relay will just re-render components whose data has actually changed. In https://quoraengineering.quora.com/Choosing-Quora-s-GraphQL-..., the engineer at Quora points out that as one paginates, one can get hundreds of components on the screen. And each pagination slows the performance of the page, if you're re-rendering the entire page from root.
- Fragments are great. You really want data masking, and not just at the type level. If you stop selecting some data in some component, it may affect the behavior of other components, if they do something like Object.stringify or JSON.keys. But admittedly, type-level data masking + colocation is substantially better than nothing.
- Relay will also generate queries for you. For example, pagination queries, or refetch queries (where you refetch part of a tree with different variables.)
There are lots of great reasons to adopt Relay!
And if you don't like the complexity of Relay, check out isograph (https://isograph.dev), which (hopefully) has better DevEx and a much lower barrier to entry.
You may be interested in checking out https://www.youtube.com/watch?v=lhVGdErZuN4, where I talk about the benefits of Relay. This isn't (currently) possible without GraphQL, so it's a pretty compelling case for GraphQL.
But yeah, IMO, GraphQL doesn't justify itself unless you're using a client like Relay, with data masking and fragment colocation.
Yeah, you can get a lot of features out of the same primitive. The primitive (called loadable fields, but you can think of it as a tool to specify a section of a query as loaded later) allows you to support:
- live queries (call the loadable field in a setInterval)
- pagination (pass different variables and concatenate the result)
- defer
- loading data in response to a click
And if you also combine this with the fact that JS and fragments are statically associated in Relay, you can get:
- entrypoints
- 3D (if you just defer components within a type refinement, e.g. here we load ad items only when we encounter an item with typename AdItem https://github.com/isographlabs/isograph/blob/627be45972fc47.... asAdItem is a field that compiles to ... on AdItem in the actual query text)
And all of it is doable with the same set of primitives, and requiring no server support (other than a node field).
Do let me know if you check it out! Or if you get stuck, happy to unblock you/clarify things (it's hard for me to know what is confusing to folks new to the project.)
I would encourage you to write an educated person's critique of GraphQL, because OP's article + https://bessey.dev/blog/2024/05/24/why-im-over-graphql/ etc. suck up all of the oxygen, and no one hears about the genuine issues like that.
(And don't forget lack of generics, no support for interfaces with no fields, lack of closed unions/interfaces, the absolutely silly distinction between unions and interfaces, the fact that the SDL and operation language are two completely different things...)
This is a genuinely accurate critique of GraphQL. We're missing some extremely table-stakes things, like generics, discriminated unions in inputs (and in particular, discriminated unions you can discriminate and use later in the query as one of the variants), closed unions, etc.