SWE-bench will hit 90% this year(fabraix.com)

https://fabraix.com/blog/swe-bench-90-percent

5 comments

upmind3개월 전

Maybe unpopular opinion but I think at this point SWE-Bench has done its part and we need a new benchmark because Gemini being on/near the same level as Claude is obviously wrong

[deleted]3개월 전

amazingamazing3개월 전

I use both and think they’re comparable. AMA.

zachdotai3개월 전

Not sure which version of Gemini are you using but Claude is so much better for me. Gemini is generally overeager to make a code change even when I am just asking conceptual questions, among other issues.

lern_too_spel3개월 전

Gemini at the same level as Claude is believable. Gemini CLI is not at the same level as Claude Code.