Hah yea. But, we use RAG to ensure that the choices made are really good. LLMs on their own can't be trusted, they are just great communicators. LLMs combined with reputable sources (like documentation and code examples) provide really great results! And these data sources actually can be queried, as they are shipped with each module.
Great question! So our AI agents actually use a combination codemods plus generated AI results. We use static analysis and codemods as much as possible, but there are lots of situations where AI is the best tool. We have found that AI is particularly good at transforming EXISTING logic from one state to another, in a fairly predictable and consistent way, so long as the code transformations are individual functions or code files.
In combination with our RAG approach, you will find that if you run the same module multiple times, the generated results are incredibly similar with very little variation. Give it a go for yourself! You can try it for free on codebases up to 2MB, or use any of our example repos.
In general, as our AI agents produce code files, we collect dependencies, and then at the end, use npm to determine the correct version numbers without actually installing on disk
We use LangChain to crawl documentation and code examples for relevant frameworks and libraries, to make sure that the AI generated PR is up-to-date. We also have a custom dependency resolver that detects which new dependencies need to be added, as well as updating the package.json files with the correct version numbers
Yes today the bots assume you are connecting to a Next.js 13 app. Bad assumption I know! I am working to add repo a scanner (ChatGPT prompts!) to understand the general tech stack of a connected repo. This will be coming soon.
I believe that now is the perfect time. Generative code generation is legit with GPT-3. I am constantly surprised at the results. And it will only get better.
Yea great question! So yes, I used Second to build the foundation, auth, and DB stuff (starter plan stuff). But other parts of my app have not been ported back into modules yet (which will become Pro modules).
At some point I want to do a full rebuild of Second using Second to test out all of the modules together. But I'm not quite there yet!
Correct! Second generates new features from configurable modules in the Second UI. I cannot use a prompt because this would require massive context windows (4k tokens is the limit). I'm using configurations so that I can use a mix of compilers and GPT-3. My compilers construct individual prompts for different tasks.
Yea, this is one of the capabilities I want to support. Auto upgrades. I am very confident it is possible, but it will be later on in my roadmap. Right now, most users and customers are interested in speeding up feature development vs. one time upgrades. But I'm with you!
So the Second UI is based on feature modules. You, the human, choose a feature to add. You configure it and then press "Commit Code" or "Raise PR". You can customize in two ways:
1) I can add more configuration options to the modules
2) you can tweak the results with code! After all, the generated code is yours to keep!
Agreed, we shall see! I empathize with what you are saying, because personally I am very slow to understand other people's code.
But, because Second builds commodity features, everything is so vanilla that it's actually pretty easy to understand. No weird anti-patterns or anything.