So, I was checking out this research paper that compares Nemotron-3-Super-120B, GPT-OSS-120B, and Qwen3.5-122B. They looked at how these models performed on different benchmarks like IFBench, SWE-Bench, Tau Bench, and RULER.
One thing that stood out was the trade-off between accuracy and inference throughput, especially with formats like NVFP4 vs BF16.
I'm really interested to know which benchmarks folks here actually rely on when they're checking out models for real-life tasks. What seems to work best for you?
Do you rely more on reasoning benchmarks, coding benchmarks, or long-context tests?
0 comments
—
Ask HN: What benchmarks do you trust most when comparing large LLMs? · HackerTrans
One thing that stood out was the trade-off between accuracy and inference throughput, especially with formats like NVFP4 vs BF16.
I'm really interested to know which benchmarks folks here actually rely on when they're checking out models for real-life tasks. What seems to work best for you?
Do you rely more on reasoning benchmarks, coding benchmarks, or long-context tests?