Show HN: Arenas suck, here's why we just added one to Windsurf(windsurf.com)
windsurf.com
Show HN: Arenas suck, here's why we just added one to Windsurf
https://windsurf.com/blog/windsurf-wave-14
Benchmarks don't reflect real-world coding ability. So we made real-world coding the benchmark.
2 comments
(team member) my comparison matrix of why Product Arenas differ from Global Arenas here: https://x.com/swyx/status/2017342647963431363
the trick is to get it to be usable within context. what started out as a simple evals concept quickly became a lot of debating over how to properly present worktrees in an IDE. hope to hear your feedback.
the trick is to get it to be usable within context. what started out as a simple evals concept quickly became a lot of debating over how to properly present worktrees in an IDE. hope to hear your feedback.
I work at Windsurf and would love to discuss product-agnostically any ideas/thoughts people have around how we as a community can evaluate models better. I feel like benchmarks like SWEbench are all saturated and gamed/trained on. I also feel like online arenas are mostly used by vibecoders. And our arena mode def isn't the final form factor either!