Why most AI coding benchmarks are misleading (COMPASS paper)arxiv.org13 points·by jmeaden·10 เดือนที่ผ่านมา·2 comments
jmeaden·10 เดือนที่ผ่านมา·discussHi, I’m one of the authors. Happy to answer questions about the dataset (LLM coding performance compared to 390k+ human submissions), the scoring approach, or the methodology behind COMPASS. Feedback and critique are welcome.
jmeaden·10 เดือนที่ผ่านมา·discussThis "taste" concept has overlap with conscientiousness, a trait that companies look for when hiring. But, it is hard to develop.