I just want to ask the frontier builders and developers who are working on the flagship models a few questions. Is it still cost-efficient and worth it to keep making huge language models, when smaller, specialized models should be enough?
Meaning that, when a user is working in a codebase with a certain framework, should the agent/model also know the complete chemical composition of an element, world history, and other random facts? Or should it only know the related and needed things? For example, an agent working in a MERN stack should really only know:
- Language Documentation
- Framework and Library Documentation
- English Interpretation
- Composition and Combination of the above
The writing style and other details are already customized by developers who have been building for a long time; tools like Prettier and ESLint can do this. And in engineering, aren't the steps usually:
- What is needed?
- What are we working with?
- What is the end goal?
- What should be the best combination of libraries, and for what?
The schematics, blueprints, and high-level design should come first, and then we build on top of that. This seems like it would be very easy if we specifically made specialized models for development. Because most of the best system models, architectures, conventions, and structures for the needed code already exist and are well-defined in the community by developers. Just like ESLint and Prettier custom rules, shouldn't our AI models be structured like that too?
Or do the agents/LLMs/models really need to know all of these unnecessary things like chemical compositions and history?
Because if we only included what was necessary for a MERN stack-specific model, all of the needed structured data could fit into an ultra-lightweight model (under 200K parameters), assuming a separate interpreter handles the English. If we make specialized models for each framework and stack, then a swarm of small agents is more than capable of taking a project all the way to completion, not just to an MVP.
Furthermore, massive models suffer from stale training data. If a library updates, you can't easily retrain a 1-trillion parameter behemoth. But in a decoupled system (where a small llm model handles the English reasoning, and sub-100K parameter structured data handles the framework rules), you can update the framework data instantly on release day. We should be building efficient Compound AI Systems that separate reasoning from knowledge, rather than burning massive GPU compute to calculate world history just to output a React component.
Meaning that, when a user is working in a codebase with a certain framework, should the agent/model also know the complete chemical composition of an element, world history, and other random facts? Or should it only know the related and needed things? For example, an agent working in a MERN stack should really only know:
- Language Documentation - Framework and Library Documentation - English Interpretation - Composition and Combination of the above
The writing style and other details are already customized by developers who have been building for a long time; tools like Prettier and ESLint can do this. And in engineering, aren't the steps usually:
- What is needed? - What are we working with? - What is the end goal? - What should be the best combination of libraries, and for what?
The schematics, blueprints, and high-level design should come first, and then we build on top of that. This seems like it would be very easy if we specifically made specialized models for development. Because most of the best system models, architectures, conventions, and structures for the needed code already exist and are well-defined in the community by developers. Just like ESLint and Prettier custom rules, shouldn't our AI models be structured like that too?
Or do the agents/LLMs/models really need to know all of these unnecessary things like chemical compositions and history?
Because if we only included what was necessary for a MERN stack-specific model, all of the needed structured data could fit into an ultra-lightweight model (under 200K parameters), assuming a separate interpreter handles the English. If we make specialized models for each framework and stack, then a swarm of small agents is more than capable of taking a project all the way to completion, not just to an MVP.
Furthermore, massive models suffer from stale training data. If a library updates, you can't easily retrain a 1-trillion parameter behemoth. But in a decoupled system (where a small llm model handles the English reasoning, and sub-100K parameter structured data handles the framework rules), you can update the framework data instantly on release day. We should be building efficient Compound AI Systems that separate reasoning from knowledge, rather than burning massive GPU compute to calculate world history just to output a React component.
Is this the real current issue?