TRIUMPH OF THE MEDIAN
In many walks of life it pays to have a neutral appearance, not stick out. However, sometimes one has to to voice an opinion. What’s the optimal strategy when voicing an opinion that will be judged against a reference, Truth, at some point?
Say life is a guessing contest. Contestants need to voice an opinion, an estimate, between 1 and 100. At a later point, a mechanism will reveal Truth, which can be any integer number between 1 and 100. The contest is about making an estimate which is as close to Truth as possible.
For every run of the contest, our contestant will be ranked among his many peers of other guessers. If our contestant has the best guess, i.e. his guess is the closest to Truth, his rank will be 1st percentile. If he’s worst, i.e. his guess is furthest away from Truth, his rank is 100th percentile, etc.
As in real-life our contestant has the ability to use his head. He’s quite lazy too and he hates to stick out. So he decides to observe what all the other hard-working guessers do and he makes a guess which is the median guess of all the other contestants.
So the question is: Over time, after several runs of the guessing contest, which average percentile will our lazy guesser, who hates to stick out, attain? Which percentile rank will the other contestants attain over time, after several runs?
For simplicity, you can assume no guessing skill among the other hard-working guessers and a uniform distribution of guesses (however, you could also assume skill, if you insist).
The correct answer is:
Percentile rank 37.5
Mathemetically: 50%*50 + 50%*25 = 37.5
Given no or little skill among other contestants, the experiment will lift the median guess over time to 1st percentile (the competitors’ average percentile rank willl be 50).
WHAT WISDOM LIES IN THE MEDIAN?
By doing nothing, simply free-riding on the work of others, the median strategy will never work very poorly (never below 50th percentile), but the strategy will in half of the time lift you to a rank that is higher than 50th percentile.
By applying the median strategy, one will never stick out, but over time one will look pretty smart. Have you ever met a person who performed the median strategy? He was always in the middle, never sticking out, never doing big idiosyncratic errors or mistakes, and over time he became successful.
Please note that the median strategy works well independent on the revealed number in the contest, if the number is Truth or Nonsense. The median strategy will work in either case, and from a scientific point of view, this is the food for thought: How can you know that the optimal strategy in this vox populi game was Truth seeking or Nonsense seeking?
If this vox populi based approach was an optimal one for our lazy contestant, can it work outside of a model? I already raised the question if you had ever noticed such a person, a smooth person who was was celebrated for his ability to define and talk for the consensus, a characteristic that led to his success? Would this strategy work for companies too? Smooth talking companies that never challenged the consensus, spending their money on marketing instead of science, R&D and capital expenditure? Only the philosopher would ask: The success of the median strategy is apparent, but is it a successful strategy for society? The cynical observer, inspired by homo economicus, would respond: The strategy makes money. What other need for evidence do you need?
Every man for himself. The median strategy is my advice to people who aspire for success. Consensus is a comfortable place too. Probably good for one’s well being and long-term health. The strategy’s mathematics is unbeatable, and it doesn’t matter if the median is based on Truth or Nonsense.
A THOUGHT ON THE MEDIAN SPEAKER
For this exercise one can illustrate the point for oneself by the RANDBETWEEN function in Excel.
Say a speaker has 7 tone controls, from A to G.
Here’s one empirical observation on all over the place EQ settings:
“However, many of us have seen evidence of such listener preferences in the “as found” tone control settings in numerous rental and loaner cars”.
Source: Floyd Toole, Sound Reproduction (latest ed.), chapter 12.3
For the ease of seeing the point, make a column in Excel with seven rows from A to G. Apply the formula RANDBETWEEN in every cell, and use minus 10 and plus 10 [RANDBETWEEN(-10,10)] to generate random numbers for every tone control, A to G. Say minus and plus 10 is the equivalent of plus/minus 10 dB deviation from “Flat”. Your first row of seven random generated numbers will look...random...much like the tone control you saw in the rental and loaner cars? If this column represented the only person in the world, the numbers associated with every tone control from A to G would represent Truth as per the vox populi way of thinking - even if this column of numbers doesn’t look like “Flat” (i.e. zeros from A to G) at all.
To expand the experiment, make 100 or even 1000 columns to represent 100 or 1000 individuals’ preferences. I will bet you a free beer that none of the columns shows straight zeros. However, if you calculate the median of the 100 or 1000 tone preferences from A to G, you’ll start to see that the median is zero for each of the tone controls. In other words, none of the individual users preferred straight zeros, but the way we made our calculation made it look like zero (i.e. “Flat”) is the preference of the population.
Food for thought: Was it real science or the design of the research model that guided the “scientist” to conclude that flat is Truth? Is the vox populi method better at finding a compromise, an average, a consensus independent of Truth?
In audio reality, I believe an average (median) of a large number of speakers would look a bit like the experimental model above. The thing is, will the power of the median strategy lead us to “Flat” in terms of frequency response because it is Truth or because it is the result of averaging a large number of (random) preferences where the median strategy rewards the consensus speaker?
Idea based science told us that Flat (in terms of frequency response) was correct long before vox populi-processes were used to arrive at the same conclusion by accident or real insight. The thing about Flat in terms of frequency response, is that its representation is a one-dimensional, linear one. What if there are multidimensional, non-linear aspects of sound that are not as easily picked up by my Excel sheet? What if an important dimension were “colour”, or a set of coordinates from a to z?
In a previous post, AES Fellow John Watkinson was ridiculed for presenting the idea of perfect waveform reproduction, not only through dacs and amplifiers, but through speakers as well. Vox populi polls has shown us that Watkinson’s “perfect waveform” is not what listeners wanted, so Truth is discarded in favor of a vox populi based result. Could one argue that Truth is Nonsense because vox populi said so?
To finish off this short essay, let me add that I am in favor of vox populi based institutions and methods. But I am also aware of the vox populi method’s limits. In economics, there has been vox populi related discussions for decades. Even though very few economists reject the idea of a market, most economists find it enjoyable to discuss the limits of vox populi. In psychology, old Truth may be about to be replaced by new Truth. There is a growing suspicion that Truth may be Nonsense in some cases for which for example Daniel Kahneman is a well-known figure. The new insight is that the results of legacy research on preferences and behaviour may be the results of framing, the way questions were asked in the original research on which later work was based. In other words, there is a growing awareness in psychology that the design of research methods influences the results. So old Truth is new Nonsense.
Some people have criticized economics - which arguably is a failed research program - for physics envy, and for applying principles from physics onto economics related problems, principles that were later abandoned by younger generations in the physics discipline. Does audio have a certain economics envy, applying a narrow focus on vox populi as more and more economists raise questions about sins of emission in their discipline? Do even younger audio researchers have a psychology envy too, putting too much weight on preference related research at the same time as psychologists too are waking up to the fact that old Truth is new Nonsense?
I am not saying, either, that vox populi based research in audio is Nonsense. But a certan awareness, a sound discussion of sins of omission, is in place in audio science as in other research areas. Wouldn’t you agree?
