Stabilizer: Statistically sound performance evaluation [pdf](people.cs.umass.edu)
people.cs.umass.edu
Stabilizer: Statistically sound performance evaluation [pdf]
https://people.cs.umass.edu/~emery/pubs/stabilizer-asplos13.pdf
6 コメント
> why do those things have a (statistically significant?) impact in the first place?
In a word, caches. Not just the instruction / data cache, but also page faults and micro-architectural features like micro-op caches, instruction TLB entries, loop stream buffers, cache-line alignment, and aliasing in the branch predictor tables (which can also be thought of as caches).
In a word, caches. Not just the instruction / data cache, but also page faults and micro-architectural features like micro-op caches, instruction TLB entries, loop stream buffers, cache-line alignment, and aliasing in the branch predictor tables (which can also be thought of as caches).
I suppose I was musing more along the lines of "why isn't this a solved problem". Clearly, it isn't an easy one or compilers would already take this into account and then the statistical variance would be reduced.
Yes, it's important to measure and control the effects of things like topology/affinity, which can have factors of several effects on HPC program efficiency. You shouldn't suffer much variance on single-node compute, but it's different when you're subject to effects on/of the network fabric and filesystem.
> We find that, while -O2 has a significant impact relative to -O1, the performance impact of -O3 over -O2 optimizations is indistinguishable from random noise.
That's definitely not true generally for GCC on floating point computation. O3 gets you vectorization, especially with -ffast-math (and extra loop optimizations recently). That's not to say it's always faster, of course; measure, as ever.
It could be that the problem arises when trying to measure very small speed increases (small relative differences => noise matters more). But in that case the fact that such a small speed increase is wiped out by random layout effects surely means that time would be better invested in finding a more performant layout?