Actually, thinking about it a bit more I think there is a natural explanation. If measured ability follows a normal distribution then the farther out you go the more spread out (on average) individuals become. For example, draw a billion samples from the right half of a normal distribution with mean 0 and stddev 1. There will be more drawn between 4 and 5 than between 5 and 6. The extreme outliers will be in some sense more lonely.
Edit: here are the most extreme values from a run of a billion draw with the distance to the next smallest value.
I've always wondered about what it is that kicks someone up from great to "how is that even possible?". For example Mozart being asked to remove a ring because his playing was so good an audience thought the ring was a talisman. It's one thing to be so good people think you cheat. It's another level when people think you are magic.
Clearly opportunity, talent, and work play their parts, but there have been many people that have all three and don't turn out a Gauss, or a Newton, or a Pele.
Edit: here are the most extreme values from a run of a billion draw with the distance to the next smallest value.
5.967211966 0.056293894
5.910918072 0.043174783
5.867743289 0.081026984
5.786716305 0.043416671
5.743299634 0.094533593
5.648766041 0.000088773
5.648737164 0.005932669
5.642804495 0.011669773
5.631134722 0.003206244
5.627928478 0.004643796
5.623284683 0.013723654
5.609561028 0.014653298
5.59490773 0.018714748
5.576192982 0.001690759
5.574502223 0.031026791
5.543475432 0.002331827
5.541143605 0.022237834
5.51890577 0.004242771
5.514662999 0.009347961
5.505315038 0.004298492
5.501016547 0.02271157