- Joined
- Feb 23, 2016
- Messages
- 20,690
- Likes
- 37,415
In a couple of recent threads it has been discussed how you can be sure blind test results are not chance at very low percentages with large numbers of trials. (Read Amir's reference on the topic if you have not yet).
http://audiosciencereview.com/forum/index.php?threads/statistics-of-abx-testing.170/
So 15 of 20 correct is enough you have less than 5% probability the result is chance. So instead of guessing 2 of 4 you are correctly picking 3 of 4 or a 75% rate correct.
With 5100 of 10,000 correct you also have less than a 5% probability the result is random. You are correct in 51% of your choices rather than just 50% you get guessing. This strikes some people as odd or at least non-intuitive. (also lets not get picky about precise numbers mine are more or less in the real ballpark)
In blind testing one should avoid getting too caught up in p-values (the odds the results are random). You also need to keep in mind the size of the effect. Is it really large enough to matter in listening to music?
So would like to hear thoughts on this from others. I will also break up this post into another few to give my thinking on how this matters.
http://audiosciencereview.com/forum/index.php?threads/statistics-of-abx-testing.170/
So 15 of 20 correct is enough you have less than 5% probability the result is chance. So instead of guessing 2 of 4 you are correctly picking 3 of 4 or a 75% rate correct.
With 5100 of 10,000 correct you also have less than a 5% probability the result is random. You are correct in 51% of your choices rather than just 50% you get guessing. This strikes some people as odd or at least non-intuitive. (also lets not get picky about precise numbers mine are more or less in the real ballpark)
In blind testing one should avoid getting too caught up in p-values (the odds the results are random). You also need to keep in mind the size of the effect. Is it really large enough to matter in listening to music?
So would like to hear thoughts on this from others. I will also break up this post into another few to give my thinking on how this matters.