How come we have all these benchmarks for models, but none whatsoever for harnesses / whatever you'd call this? While I understand assigning "scores" is more nuanced, I'd love to see some website that has a catalog of prompts and outputs as produced with a different configuration of model+harness in a single attepmt
I mean, can you expect a vibecoding company to do stuff with 0 downtime? They brought the models down and are now panicking at HQ since there's no one to bring them back up
I know it's one me for thinking this -- since the domain is fly.io -- but I was really hoping this is some local solution.
Not self-hosted, but just local. A thin command line wrapper to something (docker? bubblewrap?) that gave me sort of a containerized "VM" experience for my local machine using CoW.
I strongly agree with this.
At $WORK I usually work on projects comprising many small bits in various languages - some PowerShell here, some JS there, along with a "build process" that helps minify each and combine them into the final product.
After switching from shell scripts to Just and having to deal with a ton of issues on the way (how does quoting work in each system? How does argument passing? Environment variables?) I simply wrote a simple script with Python, UV shebang and PEP723 dependencies. Typer takes care of the command line parsing, and each "build target" is a simple, composable, readable python function that takes arguments and can call other ones if it needs to. Can't be simpler than that and the LLMs love it too.