Top AI models fail at >96% of tasks(zdnet.com)
zdnet.com
Top AI models fail at >96% of tasks
https://www.zdnet.com/article/ai-failed-test-on-remote-freelance-jobs/
12 comments
Kinda sus that least known model did best and none of the more recent models were tested. Capabilities grow very fast. So things that now routinely succeed rarely ever succeeded even half a year ago.
I mean performance is so bad across the board that this is likely essentially random. Monkeys accidentally doing a bit of Shakespeare.
That's wildly overestimating what monkeys can do on a typewriter.
It takes a lot to just be mediocre. Which, don't get me wrong, I'll agree current ML is, it's just that "mediocre" is an incomprehensibly huge step up from "random".
It takes a lot to just be mediocre. Which, don't get me wrong, I'll agree current ML is, it's just that "mediocre" is an incomprehensibly huge step up from "random".
This post really should be edited to say 96% of tasks posted on Upwork. Since we would all expect that to happen.
You think they don't? You think AI can replace programmers, today?
Then go ahead and use AI to fix this: https://gitlab.gnome.org/GNOME/mutter/-/issues/4051
Then go ahead and use AI to fix this: https://gitlab.gnome.org/GNOME/mutter/-/issues/4051
Rewrite it in react it will.
[deleted]
ChatGPT: when you want spellcheck to argue with you.
[deleted]
Models released a few days ago, Opus 4.6 and GPT 5.3, haven't been tested yet, but given the performance on other micro-benchmarks, they will probably not be much different on this benchmark.