Great point. I regret not being more systematic about this. I have tried on most popular models since gpt 3.5 launched, but it’s all been very ad hoc with the same general approach of building up a language feature by feature.
It took about 10 hours. Arguably would have taken me 100 to do it manually but likely would have fewer bugs, there’s a few I’m aware of, but I bet there are many.
I started with a basic syntax for expressions and specified a lot up front such as it being a bytecode interpreter and using a recursive descent parser.
I found building it up feature by feature to be much more effective than one shotting an entire feature rich language. Still there was a lot of back and forth.
Testing the limits of vibe coding. Created a programming language 100% via prompting a o4-mini-high, but did carefully review the code. https://github.com/ryanmcdermott/spress