• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Blind test listening results by professional audio engineers in the Los Angeles area

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,786
Likes
37,685
This test was actually done in 2019.

 

olieb

Senior Member
Joined
Jul 1, 2023
Messages
356
Likes
584
I don't really know why people think the story is interesting.
Yes, people do invent and imagine things, even out of the blue (or the red). Happens all the time.
 
OP
E

Erici

Active Member
Forum Donor
Joined
Jun 17, 2020
Messages
181
Likes
293
Location
Seattle Area
Yes, I gathered. Not meaning to be super-critical of your OP language, just that I would have called it an article/report/storey rather than test results myself.
Point taken. I guess results implied data....
 

kemmler3D

Major Contributor
Forum Donor
Joined
Aug 25, 2022
Messages
3,358
Likes
6,885
Location
San Francisco
People were trying to hear what wasn't there.
And, they did hear something that wasn't there - as expected, or maybe unexpected as the case may be.
What a waste of time. He lied to them and I'll bet nobody will be interested in his 'tests' ever again.
Maybe a waste of their time, maybe not. If you have never experienced a false positive / biased listening result, it's eye-opening and a valuable experience.

If you HAVE experienced it before and you don't care about the culture of audio marketing (snake oil) then it's a waste of time.

It's not a waste of time for the general public - examples like this can possibly help clear the smoke from this snake-oil-fueled industry.
 

Ken Tajalli

Major Contributor
Forum Donor
Joined
Sep 8, 2021
Messages
2,081
Likes
1,891
Location
London UK
Hopefully they call an extraordinary general meeting of their 'exclusive' club and throw out that member for wasting their collective time. All those blokes could have been shooting the sh#t at the bar and drinking, instead of participating in that clown-show of a test. ;)
I wonder, how could that be a blind test, when clearly it was put to them (lied to them) that there were two circuits.
If the question was, can you hear a difference, or simply, describe A vs B, with no other info. it would have been a blind test.
By giving any info (even a false one), it no longer qualifies as blind.
my 2 pence.
 

kemmler3D

Major Contributor
Forum Donor
Joined
Aug 25, 2022
Messages
3,358
Likes
6,885
Location
San Francisco
I wonder, how could that be a blind test, when clearly it was put to them (lied to them) that there were two circuits.
If the question was, can you hear a difference, or simply, describe A vs B, with no other info. it would have been a blind test.
By giving any info (even a false one), it longer qualifies as blind.
my 2 pence.
"Blind" AFAIK technically just means the test subject doesn't know which is which. "Double blind" means the subject and the administrator of the test don't know which is which. In either case, it doesn't mean they have no idea what is being tested.

For example in a double blind test of a medicine, it's no mystery to anyone what disease is being treated. They just don't know which pills are real and which are placebo.
 

Ken Tajalli

Major Contributor
Forum Donor
Joined
Sep 8, 2021
Messages
2,081
Likes
1,891
Location
London UK
"Blind" AFAIK technically just means the test subject doesn't know which is which. "Double blind" means the subject and the administrator of the test don't know which is which. In either case, it doesn't mean they have no idea what is being tested.

For example in a double blind test of a medicine, it's no mystery to anyone what disease is being treated. They just don't know which pills are real and which are placebo.
This is no different to what snake-oil merchants do.
First they implant the notion that there is an improvement, and then sit back and allow their mark's imagination do the rest.
These engineers, trusted their colleague that it was a genuine test.
a lot of presumptions went into that, to produce the desired false results.
 

restorer-john

Grand Contributor
Joined
Mar 1, 2018
Messages
12,734
Likes
38,971
Location
Gold Coast, Queensland, Australia
The entire premise of the 'test' is just dumb, and in reality proves absolutely nothing other than the guy with the LED boxes is a bit of a dick.

Imagine all that collective time wasted. Some of those guys look pretty old too...
 

MAB

Major Contributor
Joined
Nov 15, 2021
Messages
2,154
Likes
4,851
Location
Portland, OR, USA
And that's why blind testing gets the bad rep it has. Some clown tricking his friends attempting to be smarter by not introducing a circuit element he said he had introduced. People were trying to hear what wasn't there. What a waste of time. He lied to them and I'll bet nobody will be interested in his 'tests' ever again.

I'd have given the guy a piece of my mind and walked out- got better things to do in life.
Except it does illustrate one unfortunate feature of our hearing, fabrication.

Perhaps not the most polite way to go about making a point, but we do try to hear differences that aren't really there. And since the inverse of this parlor-trick is used to sell snake-oil, I am OK with these icons of the recording industry getting a serving of humble with their dinner pie.

It is a normally accepted practice to provide a placebo in tests of population responses. While it seems ingenuine to have one of the sub-populations in a medical study get sugar pills, it does improve the test statistics, and the people actually do get better.;) My position is: most of this is a figment of our imaginations, otherwise I would agree with you!:cool:
 

olieb

Senior Member
Joined
Jul 1, 2023
Messages
356
Likes
584
This is no different to what snake-oil merchants do.
I cannot follow.
One tells you to trust him about a "big improvement"no matter what and manipulates your imagination.
The other shows you the impact of your imagination and tells you to not trust yourself and be sceptic.
 

Ken Tajalli

Major Contributor
Forum Donor
Joined
Sep 8, 2021
Messages
2,081
Likes
1,891
Location
London UK
Wasn’t that exactly the point…
No.
Imagine if I had arranged the test! those engineers, would not trust me, would not assume anything, and most probably told me to get lost.
The test worked, because the engineers trusted their friend, and his integrity, accepted it was a real test.
Imagine if Amir, had arranged a similar test and his marks were ASR devotees, they probably fall into the same trap. Equally if the guy from PS Audio does the same thing, most ASR members would catch him out.
The trust of the marks, was necessary for this test to produce its false results.
 

kemmler3D

Major Contributor
Forum Donor
Joined
Aug 25, 2022
Messages
3,358
Likes
6,885
Location
San Francisco
This is no different to what snake-oil merchants do.
First they implant the notion that there is an improvement, and then sit back and allow their mark's imagination do the rest.
These engineers, trusted their colleague that it was a genuine test.
a lot of presumptions went into that, to produce the desired false results....

The trust of the marks, was necessary for this test to produce its false results.
Indeed, that's why I think it was a worthy "experiment". It's a clear demonstration of how cognitive bias makes us think we heard things that we didn't hear. Snake oil salesmen are counting on two things:

1. This cognitive bias functions as expected, as it did in this "test"
2. The marks don't realize that cognitive bias - and nothing else - is at work.


A simple, clear demonstration of the snake oil process like this is a useful weapon in the war against snake oil. If nothing else, there's a clear cut example of experienced critical listeners hearing stuff that isn't there. We need a lot of these examples if we're going to convince the closed-minded out there.
The entire premise of the 'test' is just dumb, and in reality proves absolutely nothing
Disagree, there are plenty of people who resist/deny the entire notion of cognitive bias and false positives in listening tests. And then they turn around and spend $10K on a cable. Showing that even elite professionals are subject to these effects can't hurt. It might open some minds and save some wallets.

So I agree it's not an experiment so much as a public service. Naturally, this "experiment" (or demonstration, or prank, as you prefer) can't be done with the knowledge or consent of the subjects. Hopefully they had a sense of humor about it anyway.

There's a big elephant in the room when it comes to audiophile stuff, and if this "test" forces people to acknowledge it, I support it. No matter how good a listener you are, you will most likely hear differences between two devices, even if there isn't one. Yes, you.
 
Last edited:

tw99

Senior Member
Joined
Mar 13, 2019
Messages
469
Likes
1,074
Location
West Berkshire, UK
The entire premise of the 'test' is just dumb, and in reality proves absolutely nothing other than the guy with the LED boxes is a bit of a dick.

Imagine all that collective time wasted. Some of those guys look pretty old too...

How much of an actual circuit change would have made it a valid test in your opinion ?
 

kemmler3D

Major Contributor
Forum Donor
Joined
Aug 25, 2022
Messages
3,358
Likes
6,885
Location
San Francisco
How much of an actual circuit change would have made it a valid test in your opinion ?
A valid test would have two devices from the same category (e.g. DAC, Amp, pre) that are at least known to have measurable differences and are known or claimed to have audible differences. Otherwise there isn't a real test going on, I guess.

I agree with restorer-john in that this wasn't a test per se, I just think it's a useful anecdote / "case study" when it comes to the task of convincing people that snake oil is indeed snake oil. Just because someone heard something proves nothing, and ... here's the proof of that. :)

As an aside, I think this shows that the greatest golden ear skill is not hearing small differences in noise, distortion, or FR... true mastery is being able to confidently and accurately determine when there is NO difference.
 

Chrispy

Master Contributor
Forum Donor
Joined
Feb 7, 2020
Messages
7,938
Likes
6,097
Location
PNW
The entire premise of the 'test' is just dumb, and in reality proves absolutely nothing other than the guy with the LED boxes is a bit of a dick.

Imagine all that collective time wasted. Some of those guys look pretty old too...
Like most subjective audio opinions are dumb? Sometimes you need a bit of dickishness in audiophilia, or at least counter-dickishness.
 

Philbo King

Addicted to Fun and Learning
Joined
May 30, 2022
Messages
669
Likes
877
I remember another listening test done where nobody noticed one option was mono. I think it may have been at a NAM show.
 

MAB

Major Contributor
Joined
Nov 15, 2021
Messages
2,154
Likes
4,851
Location
Portland, OR, USA
The entire premise of the 'test' is just dumb, and in reality proves absolutely nothing other than the guy with the LED boxes is a bit of a dick.

Imagine all that collective time wasted. Some of those guys look pretty old too...
Wait, the entire premise of Topping is to sell us new but functionally identical boxes with nothing but a new LED on the front.;)
 
Top Bottom