VibeBench: Measuring 1k Engineers' Opinions of New Models(vibebench.standardagents.ai)

12 pointsby jpschroederil y a 2 mois4 comments

https://vibebench.standardagents.ai/

4 comments

ramon156il y a 2 mois

Love the idea!

Page is incredibly slow on mobile, probably the avatars

800 commits in a year...

mhi3il y a 2 mois

"Published benchmarks are gamed, optimized, and overfit, and no longer yield a useful signal."

Is this true?

But I love this concept!

jpschroederil y a 2 mois

Oh very true. Benchmaxxing itself is basically gaming them.