Every chip company is misleading to some extent when using benchmarks (lying with statistics and all that).
I’m especially seeing this in AI chip comparisons with startups trying to boast about their incredible performance multipliers over the competition with toy networks or playing around with impractical accuracy metrics and batch sizes.
I’ve never understood this either. I guess it’s technically under a “beta” label so it gets a pass for now. But I can’t imagine this continuing without consequence forever, unless Tesla actually manages to solve self-driving with a vision-only approach.