solderdude
Grand Contributor
Yes, doing a controlled test, level matched and blind is very hard to do. The 'huge' and 'obvious' differences disappear.
That does not mean one cannot differentiate in an AB test.
The choice of music is important as well as the used headphone and training.
What isn't clear but appears to be so is that he listened for 10 mins and then made 1 attempt to give the answers.
Now there are 3 amps. One of which has -50dB distortion at 1V output.
This is a level that will be reached and is considered just audible by me.
The Magni3+ and Heresy are both below any considered audible levels (using music) so IME would be indistinguishable.
These will give you a 50% chance of being guessed right when one only makes one attempt.
Maybe there were more guesses but what I read was he made one guess and may have picked the Vali2 correct.
That leaves a 50% chance one guesses right.
His guess was lucky (based on earlier attempts he did not pass) and he stopped right there 'while he was ahead'.
I am quite convinced that had he continued and made 10 attempts he would have gotten about 5 out of 10 most likely failing to detect the 3+ and Heresy.
Maybe someday some of the better hearing crowd will take the test with 3 more similar measuring amps and he keeps notes of correct and incorrect guesses just for the sake of stats.
IMO Jason did not convince me he can tell the Heresy and 3+ apart with statistically relevance but makes it appear as though he did. He could have been lucky.
The thing is suppose that more audiophiles take the same test and get it right (by accident) on try one they will say with confidence: I passed a blind test so yes I can detect differences.
Those that got unlucky and guessed wrong may accidentally guess right the second time (50% chance) and will say: The first time I failed but the second time I learned the differences and had it correct so I can hear but it was not easy.
This is why statistically valid tests are important when comparing 2 (as good as identical) amps. I consider the Vali2 distinguishable in most cases.
This is extremely hard to pass and very fatiguing. It should be spread over several days or so. That's where the shoe doesn't quite fit for these experiments. Time, concentration and attempts. This is essential for truth finding but most already have their truth and only need validation (like what I wrote above).
The test would be harder to pass when the Vali2 were to be replaced by say a Jotunheim and a forth is added say Ragnarok.
4 amps that should be guessed correctly is much more difficult to guess right. Certainly when one has 3 attempts or so.
Switch position (when the guesser is operating it) should change after each attempt as well.
Looking forward for folks passing that test. And those that guessed right must be obligated to do the test 3 times at least.
That does not mean one cannot differentiate in an AB test.
The choice of music is important as well as the used headphone and training.
What isn't clear but appears to be so is that he listened for 10 mins and then made 1 attempt to give the answers.
Now there are 3 amps. One of which has -50dB distortion at 1V output.
This is a level that will be reached and is considered just audible by me.
The Magni3+ and Heresy are both below any considered audible levels (using music) so IME would be indistinguishable.
These will give you a 50% chance of being guessed right when one only makes one attempt.
Maybe there were more guesses but what I read was he made one guess and may have picked the Vali2 correct.
That leaves a 50% chance one guesses right.
His guess was lucky (based on earlier attempts he did not pass) and he stopped right there 'while he was ahead'.
I am quite convinced that had he continued and made 10 attempts he would have gotten about 5 out of 10 most likely failing to detect the 3+ and Heresy.
Maybe someday some of the better hearing crowd will take the test with 3 more similar measuring amps and he keeps notes of correct and incorrect guesses just for the sake of stats.
IMO Jason did not convince me he can tell the Heresy and 3+ apart with statistically relevance but makes it appear as though he did. He could have been lucky.
The thing is suppose that more audiophiles take the same test and get it right (by accident) on try one they will say with confidence: I passed a blind test so yes I can detect differences.
Those that got unlucky and guessed wrong may accidentally guess right the second time (50% chance) and will say: The first time I failed but the second time I learned the differences and had it correct so I can hear but it was not easy.
This is why statistically valid tests are important when comparing 2 (as good as identical) amps. I consider the Vali2 distinguishable in most cases.
This is extremely hard to pass and very fatiguing. It should be spread over several days or so. That's where the shoe doesn't quite fit for these experiments. Time, concentration and attempts. This is essential for truth finding but most already have their truth and only need validation (like what I wrote above).
The test would be harder to pass when the Vali2 were to be replaced by say a Jotunheim and a forth is added say Ragnarok.
4 amps that should be guessed correctly is much more difficult to guess right. Certainly when one has 3 attempts or so.
Switch position (when the guesser is operating it) should change after each attempt as well.
Looking forward for folks passing that test. And those that guessed right must be obligated to do the test 3 times at least.