Benchmark is not designed for the red team testing. I don't even think it make sense to "fix" the issue the article is suggesting. Yes, you can break the running contest by driving a car. Does this mean we need to make running contest car-proof?
100% agree. it not only applies to the training time but also applies to the inference time. Domain specific context and skills will be the only moat for companies