This has been the thesis behind our product since the beginning (~3 years), before a lot of the current hype took hold. I'm excited to see it gain more recognition.
Chat is single threaded and ephemeral. Documents are versioned, multi-threaded, and a source of truth. Although chat is not appropriate as the source of truth, it's very effective for single-threaded discussions about documents. This is how people use requirements documents today. Each comment on a doc is a localized chat. It's an excellent interface when targeted.
Karpathy recently tweeted about the importance of being close to the data when training LLMs, and I think a similar level of rigor and transparency is essential when judging evaluations. Thanks for bringing us this insight from the data.
Building hierarchical abstractions on top of code is IMO the only way to truly enable AI to write code beyond better autocomplete. Kevin's work with DreamCoder shows that hierarchical abstractions can be built automatically in the code domain.
Once these abstractions exist side-by-side with natural language (essentially a code-natural language world model), it'll enable arbitrarily complex code generation from descriptions of the outcomes/results.
Is that really that much of a barrier? Off the top of my head: you could start with a prompt like "write a summary of book XYZ, followed by a summary of each chapter". Then dive deeper into each one from there using the same prompt recursively, etc.
I appreciate the example, but here's where I think it differs as an analogy of what LLMs do:
A summary doesn't have infinite or variable depth. If you read the summary of a non-fiction (I'll limit my argument to that, as another poster pointed out) book, and either aren't convinced, or want to learn more about the matter, you'd have to purchase the book.
An LLM that has ben trained on the book, if somehow designed not to hallucinate, would be able to answer any question you have about the book at any depth, seamlessly blending in material from other books to answer a question or explain a concept. That seems like an entirely better experience than reading the book from start-to-finish. I don't see how the original can compete.
You're right. I think it's fair to carve out fiction from my argument. For that, I would surely go to the source material until the point where the LLM was coming up with better long-form fiction de-novo. But for non-fiction, which I would guess is the economically and intellectually more important category to protect, the effects may be devastating.
I also agree that attribution can't be solved easily in the current paradigm. Perhaps, during training, one could deduce how much of the net gradient on a particular weight was derived from the batches covering some book, and then during inference, assign attribution based on the effect of that weight on the output. All of this is very expensive to do, and I don't have strong intuitions for whether the resulting attributions would be in any way meaningful.
To your point about hallucinations, if there's not a solution to that, then perhaps the whole point is moot when, after a while, the hype dies down. But if somehow hallucinations are solved (I don't see a technical way this can happen now, but who knows?), then I think we'll need to address attribution for non-technical material.
"The difference, when it comes to AI, is one of scale. ChatGPT can “read” more published words in a few seconds than I could in several lifetimes and, unlike me, that data isn’t immediately replaced in my human-limited short-term memory by whatever I’m thinking of next."
I think this misses the point. The issue of scale isn't on the ingest side, it's on the output side. Once you train an LLM on a book (however long that takes), then the LLM can be the interface to that book for an unlimited number of users. That scales very differently to, say, a person reading a book and writing something influenced by it.
In the case of the LLM, it's a complete interface to the contents of the book. It lets you "talk to the book". If that exists, why would anyone buy the book? If I could ask ChatGPT to "summarize the new book by XYZ", then spend an hour or two asking the questions _I_ have about the book from it, then buying the book would be a net negative.
If we don't solve attribution (like BMI solved for music), then the financial upside of publishing might be majority-captured by whoever trains LLMs on the copyrighted material.
The addition of many remote positions since COVID has been a great hiring filter for us. For the type of work we do (AI, R&D), and the culture that we find most productive and enjoyable (enthusiasm because we love the work, a sense of working in a team), remote was a real downgrade when we tried it.
We advertise the job as on-site only, and because of that the applications self-select for those that want in-office work. It's made our interviews more focused on technical ability.
I think this is a better equilibrium overall. Those on either side of the remote/on-site preference can find the right respective jobs and work cultures.