kernelsanderz·5 mesi fa·discussTheo’s snitch bench is a good data driven benchmark on this type of behavior. But in fairness the models are prompted to be bold to take actions. And doesn’t necessarily represent out of the box or models deployed in a user facing platform.https://snitchbench.t3.gg/
kernelsanderz·4 anni fa·discussIf you look at the dockerfile you can see that it’s copied from alice.py
https://snitchbench.t3.gg/