I did something very similar last year, but with programming languages that were REALLY out of distribution; they were generated specifically for the benchmark. I call it TiānshūBench (天书Bench): https://jeepytea.github.io/general/introduction/2025/05/29/t...
Some models were OK at solving very simple problems, but nearly all of them would, for example, hallucinate control structures that did not exist in the target language.
Historically, the cycle has been requirements -> code -> test, but with coding becoming much faster, the bottlenecks have changed. That's one of the reasons I've been working on Spark Runner to help automate testing for web apps: https://https://github.com/simonarthur/spark-runner
I've recently found that my ability to add new features and squash bugs has outpaced my ability to do full end-to-end tests. To help with this, I created Spark Runner for automated website testing. It will create and execute a plan for tasks you give it in plain text like "add an item to the shopping cart" or you just point it at your front end code and have Spark Runner create the tests for you. It also makes nice reports telling you what's working and what's not.
Lisp has been around for 65 years (not 50 as in the author believes), and is one of the very first high-level programming languages. If it was as great as its advocates say, surely it would have taken over the world by now. But it hasn't, and advocates like PG and this article author don't understand why or take any lessons from that.
Depends. If your employer doesn't offer health insurance, you can shop for plans on the government health insurance marketplace. These plans can have substantial subsidies.
One of my favorite address stories: the Russian Post Office successfully delivered a package with an address hand-written with the wrong character encoding.
As a regular shipper, the addresses I have the biggest problem with are at colleges and universities. No address verification/normalization software recognizes things like "3948-B Engineering Wing, Richguy Hall, Anystate University, Cityville, ST, 99999". But if you force the address through, the local UPS guy will know where that location is every time and deliver the package without a problem.
