I just shipped 3pio, a drop-in test runner that context-optimizes your test output. It uses your existing test runner and tests so zero changes to your codebase or tooling to use it.
IME it results in much less context clutter from your test output.
Mark Brown of Game Maker's Toolkit fame runs a yearly game jam centered around a specific theme [1], which would line up well with this initiative. The quality of submissions is really, really impressive.
You see a similar loosening of the term in other fields e.g. open source journalism. Although that seems to be more about crowdsourcing than transparency or usage rights.
Hey all, I found the post about Don Knuth and ChatGPT [1] very interesting so I hacked together this project over the weekend. Optimystic periodically re-runs Knuth's questions against the latest GPT model (additionally I've asked GPT to score the updated answers as PASS/FAIL).
Initially I thought it would just be a fun thing, but I've realized there could be some value to the larger LLM community. Also I think it would be interesting to apply this format to experts in other fields.
The flux docs indicate that you're using a custom-trained LLM and in another comment it was mentioned you're using LangChain (to integrate non-deterministic tooling perhaps). Just curious, but are you using some combination of your own model, GPT3/4, and calculators / datasheet readers / etc?
IME it results in much less context clutter from your test output.
https://github.com/zk/3pio