kernelsanderz·5 maanden geleden·discussTheo’s snitch bench is a good data driven benchmark on this type of behavior. But in fairness the models are prompted to be bold to take actions. And doesn’t necessarily represent out of the box or models deployed in a user facing platform.https://snitchbench.t3.gg/
kernelsanderz·4 jaar geleden·discussIf you look at the dockerfile you can see that it’s copied from alice.py
https://snitchbench.t3.gg/