MarkS
Major Contributor
- Joined
- Apr 3, 2021
- Messages
- 1,070
- Likes
- 1,510
This is too vague to be a prior.
In my old AES paper, I assumed that, for each guess, the listener hears a difference in a fraction h of the tests (0<=h<=1) and responds correctly. In a fraction 1-h of the tests, the listener does not hear a difference (even if thinks he does), and guesses. The probability of a right answer (for a single trial) is then h + (1/2)(1-h) = (1/2)(1+h), and the probability of a wrong answer is (1/2)(1-h). In a sequence of trials, the probability of getting R right answers and W wrong answers is N (1+h)^R (1-h)^W, where N is a constant that depends on R and W but not h (and whose value will not matter). This is called the likelihood function.
Now we need a prior. Before doing the experiment, what do we think about h? Skeptics of the audibility of differences would want to say that it is very likely that h is at or very near zero. Someone with no skepticism (or knowledge of prior audio test results) could apply Laplace's Principle of Indifference, and say that any value of h between zero and one is equally likely.
So if we call the prior Prior(h), the "indifference" prior is Prior(h)=1.
To get the posterior we mulitply the likelihood by the prior,
Post(h) = N' (1+h)^R (1-h)^W Prior(h)
where N' is a different constant, and is fixed by the requirement the integral over h from 0 to 1 of Post(h) is 1. (In words, the posterior probabilites sum to 1.)
For R=9 and W=1, and the "indifference" prior Prior(h)=1, here is a graph of Post(h):
So if we had no idea of what h should be going in, we now think it's most likely to be near 0.8.
Note that, because there was one wrong answer, we are now certain that h=1 is not possible, and so Post(h) drops to zero as h approaches 1.
What happens if we think, going in, hearing a difference is very unlikely?
I won't go through the math (and this case was not treated in the paper), but suppose you assign a prior probability of 1-d to h=0 exactly, and probability d to h being equally likely to be anywhere between 0 and 1. If d is very small, it means that you're prelty darn sure that no difference can be heard, but you'll allow a little wiggle room. Then, after the listener gets 9 right and 1 wrong, the posterior vaule of d goes up to 18.5 times the original value. So if d=0.01 going in (that is, you allow a 1% probability that a difference can be heard before the test), then afterward, that probability rises to 0.185. That's a noticeable effect. But if your starting value of d is much smaller, say 0.0001, then after it will be 0.00185: still really small. Then you'd need more convincing.
Anecdote: I wrote that AES paper shortly after discovering Bayesian analysis and being thoroughly besotted with it as just the bees knees. I was explaining it, rather excitedly, to an experimental physicist. He yawned and said, "If how you do the statistical analysis matters, then you need to get more data."
And that, ladies and gentlemen, is correct! 10 tests just isn't very many. But get 90 right out of 100, and the factor increase in d goes from 18.5 to 10^15. Now THAT would be data that could only be ignored by the most confirmed skeptic going in.
Last edited:
), perfectly valid comment.
