Catalogue of blind tests

[ahofer]

Not sure if this digital filter test by Archimago was ever linked, but I mentioned it in another thread today

INTERNET BLIND TEST: Linear vs. Minimum Phase Upsampling Filters

A blog for audiophiles about more objective topics. Measurements of audio gear. Reasonable, realistic, no snakeoil assessment of sound, and equipment.

archimago.blogspot.com

 

[ahofer]

Here’s an article on high-res recordings causing IM distortion in some equipment. Along the way it cites the Waldrep and BAS tests supporting the null hypothesis of the inaudibility of digital resolution above redbook.

https://www.****************/2025/0...a5vIDiQ7tEvP4FWTDY_aem_fi47-_mVrin4rDdkAldEbQ

 

[ahofer]

This is a nice summary. It goes back to the OP, but adds some others.

https://www.****************/2026/03/blind-tests-reveal-only-one-upgrade-matters/

The domain that is blocked from linking here is a portmanteau of “Headphone” and “Honesty”.

 

[MarkS]

Apparently ASR does not allow links to your source. You can find the second article by googling "blind-tests-reveal-only-one-upgrade-matters". It is a good article, entirely in accord with ASR principles. The first link is still broken because it got partially redacted (the ... after the final /).

 

[Palladium]

Found this old 2010 thread about a 2009 AES speaker blind test, from a Behringer repair thread here: https://www.diyaudio.com/community/threads/linkwitz-orions-beaten-by-behringer-what.177403/

In short: Even the late Linkwitz's own then-$9000 of Orions, under his own recommendations on room placement, was still unable to outperform a $360 pair of B2031As in any of his preferred positions under the test environment and criteria.

 

[SimpleTheater]

Apparently ASR does not allow links to your source. You can find the second article by googling "blind-tests-reveal-only-one-upgrade-matters". It is a good article, entirely in accord with ASR principles. The first link is still broken because it got partially redacted (the ... after the final /).

What confuses me a bit with the article, is just after they talk about the $100,000 challenge to distinguish two amplifiers, which no one could, they write "In 1985, Stereophile challenged Bob Carver to make a $400 amplifier indistinguishable from a Conrad-Johnson Premier Five worth more than $6,000. Stereophile gave him 48 hours. Using null-difference testing, Carver modified his amp until the output matched the Conrad-Johnson’s"

So they COULD? hear the difference between these two amps prior to Carver making the changes?

There's a lot of amps that make a lot of distortion, so I could imagine you could hear the difference between at least two. If not, why do we bother Amir to test amps?

 

[ahofer]

And don’t forget the one where they couldn’t tell a Futterman OTL from a Pioneer receiver.

Master Thread: Are Measurements Everything or Nothing?

No, because it's the critical issue that you are overlooking. You guys are just making stuff up and stating it as axiomatic. Can we discuss the actual research instead? In a nutshell, 100% of listeners prefer the Harman Curve to within 1 dB above 300 Hz, but below 300 Hz listeners split into...

www.audiosciencereview.com

 

[RandomEar]

What confuses me a bit with the article, is just after they talk about the $100,000 challenge to distinguish two amplifiers, which no one could, they write "In 1985, Stereophile challenged Bob Carver to make a $400 amplifier indistinguishable from a Conrad-Johnson Premier Five worth more than $6,000. Stereophile gave him 48 hours. Using null-difference testing, Carver modified his amp until the output matched the Conrad-Johnson’s"

This seems to be related to this test. The Sterephile employees going into this challenge sound overconfident and are simply wrong about certain things they confidently talk about. For example, they state

We made no effort to do A/B testing, since we feel it does not replicate normal listening conditions, and there is still insubstantial evidence that A/B testing reveals small differences as well as does prolonged listening to each unit under test.

which is just bollocks.

So they COULD? hear the difference between these two amps prior to Carver making the changes?

They did not do blind testing before the challenge started, only sighted listening. And as far as I can see not even level-matched and with the amps having different gains. So there likely were (level) differences, but their testing was insufficient to actually conclude what they wrote. I'm pretty sure most of the efforts invested by Mr. Carver for this challenge were just to satisfy the ego of the editors. Had they done correctly controlled blind tests from the beginning, they might have lost the challenge in step 1 before modifying anything about the Carver amp.

There's a lot of amps that make a lot of distortion, so I could imagine you could hear the difference between at least two. If not, why do we bother Amir to test amps?

We can imagine a lot of things, but in the end, you need to test correctly and with controls to determine if your impressions are indeed correct. Outside of tube amps, there's really really few amps with relevant levels of distortion. We test to be sure that everything is in order, to judge if the amps are well engineered and that they deliver on the specs they promise.

 

[MAB]

This seems to be related to this test. The Sterephile employees going into this challenge sound overconfident and are simply wrong about certain things they confidently talk about. For example, they state

which is just bollocks.


They did not do blind testing before the challenge started, only sighted listening. And as far as I can see not even level-matched and with the amps having different gains. So there likely were (level) differences, but their testing was insufficient to actually conclude what they wrote. I'm pretty sure most of the efforts invested by Mr. Carver for this challenge were just to satisfy the ego of the editors. Had they done correctly controlled blind tests from the beginning, they might have lost the challenge in step 1 before modifying anything about the Carver amp.


We can imagine a lot of things, but in the end, you need to test correctly and with controls to determine if your impressions are indeed correct. Outside of tube amps, there's really really few amps with relevant levels of distortion. We test to be sure that everything is in order, to judge if the amps are well engineered and that they deliver on the specs they promise.

Yes, the Carver Challenge was a bit of theater, not a blind test, and actually relies on people's inability to hear the differences in amps.

Even tube amps are hard to distinguish, and the tell is more often the frequency response, not the distortion (even or odd).

 

[mikewxyz]

If I remember, Carver electrically matched the distorted output of the tube amps. He took away any real reason that there would be a audible difference. That the Stereophile crew conceded that they sounded the same was the surprising part.

 

[ahofer]

If you go over to ************ (ASR does not allow links) you can see they’ve done something similar to this thread in clickbait form. I don’t see any new tests, but I can only stand the ads for so long.