Thursday, February 11, 2016

Theorem: You are exceptional

“Everyone excels at something.” We hear it all the time, usually said to console someone who is miserable after underperforming at something. Parents, in particular, often fall back on it with their children. What few people realize, though, is that the statement can be mathematically verified. You need only consider a collection of 200 essentially independent human performance characteristics for 98% of people to measure as exceptional in at least one of them, where exceptional is defined as being in the top or bottom 1%. (The mathematics gives extremal values; if you want to effectively guarantee being in the top 1%, you need more characteristics. The phenomenon is asymptotic.)

This result is a consequence of a rather surprising, but little known, observation about high-order hypercubes: as the dimension increases, the proportion of points in the interior (i.e., not on the bounding shell) decreases without limit.

Here is how you can prove to your child, spouse, student, best friend, or whoever, that they—or you, as the circumstances may require—can or will excel at something.

Fig 1. The bell curve (normal distribution)

Everyone is familiar with the bell curve (normal distribution) showing the typical distribution of performance measures of a single characteristic across a sufficiently large population. This graph captures the fact that the scores for the majority of the population cluster around an “average”, middling value, with only a few individuals at either end (exceptionally poor or exceptionally good).

For the purposes of the multi-dimensional computation, we can start with a geometrically simpler model, namely the closed interval [0,100], as in Figure 2. We define the exceptional points to be those in the unit intervals at each end. In this model, for a single characteristic, only 2% of the population are exceptional. The remaining 98% are “normal."

Fig 2. A simple model of exceptionality in one characteristic

Now consider two characteristics, X and Y (assumed to be independent). The distribution then is represented by a 100x100 square with an inner 98x98 cube, as in Figure 2.


Fig 3. A simple model of exceptionality in two characteristics

An individual’s X measure is shown by the x-coordinate, their Y measure by the y-coordinate. The ordinary individuals are represented by points in the inner square; the exceptional individuals by points in the outer perimeter region.

The total number of points is 100x100. The number of normal points is 98x98. So the number of exceptional points is 10,000 – 9,604 = 396.

The proportion of exceptional points is thus 396/10,000 = 0.0396, i.e., 3.96%. Thus, more individuals are classified as exceptional when you consider two characteristics (3.96% as opposed to 2%).

Going to three characteristics, X, Y, and Z, the model will be a 100x100x100 cube with an inner 98x98x98 cube, as in Figure 4.

Fig 4. A simple model of exceptionality in three characteristics

The volume of the outer cube (representing the total population) is 1,000,000. The volume of the inner cube (representing the normal individuals) is 941,192. Thus the volume of the perimeter-region (representing the exceptional individuals) = 1,000,000 – 941,192 = 58,808. Hence, the proportion of exceptional individuals = 58,808/1,000,000 = 0.0588, i.e. 5.88%.

So far, everything seems fairly straightforward and reasonable. Going beyond three characteristics, the model is a hypercube of four or more dimensions, and we can no longer provide meaningful illustrations. But by now we have grown familiar with the idea that the model represents exceptional individuals by points in the outer 1% shell. To see what this entails, let’s jump to 10 characteristics, X1,…,X10. In that case, our model will represent the situation as a 10010 hypercube with an inscribed 9810 hypercube.

The volume of the outer hypercube (~ total population) = 10010. The volume of the inner hypercube (~ normal individuals) = 9810. Thus, the volume of the perimeter-region (~ exceptional individuals) = 10010 – 9810, and the proportion of exceptional individuals = (10010 – 9810)/10010. At this point, it’s time to bring in Wolfram Alpha to do the calculation. This gives the result that, with 10 characteristics, 18.29% of the population is exceptional.

With 100 characteristics, X1,…,X100, our model gives: Volume of hypercube (~ total population) = 100100. Volume of inner hypercube (~ normal individuals) = 98100. Volume of perimeter- region (~ exceptional individuals) = 100100 – 98100. Proportion of exceptional individuals = (100100 – 98100)/100100. Calling on Wolfram Alpha again, we compute that with 100 characteristics, 86.74% of the population is exceptional.

With 200 characteristics, X1,…,X200, our model gives: Volume of hypercube (~ total population) = 100200. Volume of inner hypercube (~ normal individuals) = 98100. Volume of perimeter-region (~ exceptional individuals) = 100200 – 98200. Proportion of exceptional individuals = (100200 – 98200)/100200. So with 200 characteristics, 98.24% of the population is exceptional. (Once again calling on the services of the normally unflappable Wolfram Alpha.)

And there’s our result.

Of course, we have been working with a model. As always, that entails making various assumptions and simplifications. If the final result surprises you, you have two choices. Either go back and modify your initial assumptions and generate another model. Or accept the result and modify the prejudices that led to your surprise.

In this case, we have to accept that in higher dimensions, almost all the material in an equal-sided, rectangular, solid (!) box is on the outer shell. The (solid) inside is almost empty.

When we consider more dimensions to a situation, the math can sometimes lead us to a counter-intuitive—but correct—conclusion we did not expect. Not everyone can accept that.

Yes, in this US Election Season, this is a story with a moral.