• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Blind test listening results by professional audio engineers in the Los Angeles area

restorer-john

Grand Contributor
Joined
Mar 1, 2018
Messages
12,995
Likes
39,838
Location
Gold Coast, Queensland, Australia
Clearly generally the same as everyone... it helps if people are trained listeners, not "golden ears".
... but clearly in this case, that wouldn't have helped either! :D


JSmith

That Harman thing is a 113MB download and that was 2011! What is it, an operating system? LOL.
 

restorer-john

Grand Contributor
Joined
Mar 1, 2018
Messages
12,995
Likes
39,838
Location
Gold Coast, Queensland, Australia
It's probably full of WAV files...
And viruses.

Connects to Harman and downloads the Revel speaker catalogue and a JBL screensaver you didn't ask for, all while brainwashing you into believing the boom-tizz curve is 'HiFi'... ;)
 

Mr. Widget

Major Contributor
Forum Donor
Joined
Oct 11, 2022
Messages
1,249
Likes
1,929
Location
SF Bay Area
Everyone is subject to expectation bias. Everyone!
The most interesting thing to me is that we must rely on our ears if we want to actually hear the music instead of simply admiring a set of beautiful plots and yet we absolutely can not trust our ears.

And we have been unable to trust our ears for a very long time. We can imagine that anyone alive today would not mistake an "Edison Disk" with an actual live event, but in 1910 the expectation bias was tilted a bit differently than ours is today.


In 1910, he [Thomas Edison] invented the Edison Diamond Disk Phonograph, which he claimed had “no tone” of its own. To prove it, a series of road shows involving 4,000 live-versus-recorded demonstrations of his phonograph were conducted in auditoriums across the United States At some point during the live music performance there would be a switch over to the recorded performance, and apparently audience members could not tell the difference between the live and recorded performances
After a 1916 live-versus-recorded demonstration in Carnegie Hall, the New York Evening Mail stated “the ear could not tell when it was listening to the phonograph alone, and when to actual voice and reproduction together. Only the eye could discover the truth by noting when the singer’s mouth was open or closed”

 

radix

Major Contributor
Joined
Aug 1, 2021
Messages
1,440
Likes
1,410
Or they are both part of a larger category of phenomena where non-aural stimuli are mistaken for actual aural differences.

yes, this was the spirit in which I thought the McGurk effect is analogous. The brain lies to you. If you have some expectation of A, you can hear A even if it's not there. The fake AB test was, like many psychology experiments, purported to do one thing to measure something else. It wasn't really about AB tests.
 

dweeeeb2

Active Member
Joined
Feb 10, 2023
Messages
248
Likes
243
Location
Melbourne
This test has confused me. Am I actually better spending money on the science or the story? Sounds like the story is more important.
 

kemmler3D

Major Contributor
Forum Donor
Joined
Aug 25, 2022
Messages
3,971
Likes
8,042
Location
San Francisco
The most interesting thing to me is that we must rely on our ears if we want to actually hear the music instead of simply admiring a set of beautiful plots and yet we absolutely can not trust our ears.

And we have been unable to trust our ears for a very long time. We can imagine that anyone alive today would not mistake an "Edison Disk" with an actual live event, but in 1910 the expectation bias was tilted a bit differently than ours is today.


In 1910, he [Thomas Edison] invented the Edison Diamond Disk Phonograph, which he claimed had “no tone” of its own. To prove it, a series of road shows involving 4,000 live-versus-recorded demonstrations of his phonograph were conducted in auditoriums across the United States At some point during the live music performance there would be a switch over to the recorded performance, and apparently audience members could not tell the difference between the live and recorded performances
After a 1916 live-versus-recorded demonstration in Carnegie Hall, the New York Evening Mail stated “the ear could not tell when it was listening to the phonograph alone, and when to actual voice and reproduction together. Only the eye could discover the truth by noting when the singer’s mouth was open or closed”

Really interesting. But I could imagine (even in 1916) that if you put the best mono speaker you could find, up against a singer on stage, it might not sound so bad, especially if you got the singer to try and sound a bit like a record.
 

LTig

Master Contributor
Forum Donor
Joined
Feb 27, 2019
Messages
5,912
Likes
9,753
Location
Europe
Indeed, that's why I think it was a worthy "experiment". It's a clear demonstration of how cognitive bias makes us think we heard things that we didn't hear. [..]
Disagree, there are plenty of people who resist/deny the entire notion of cognitive bias and false positives in listening tests. And then they turn around and spend $10K on a cable. Showing that even elite professionals are subject to these effects can't hurt. It might open some minds and save some wallets.
Yep. In 2003 I took part in a non blind AB test between SPDIF coax and toslink cables, to find out whether there are differences in sound at all, and if yes which would sound better. Other than most participants my golden ears told me clearly that coax sounded fuller and wider than toslink. However they still told me this when the person handling the switch did not switch from toslink back to coax but continued with toslink. It was embarassing but a real eye opener for me and forever changed my trust into what I can hear.

You really have to experience this. Reading about it is not sufficient, hence I'm all in favor for doing this kind of test as often as possible.
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
21,254
Likes
38,658
Yep. In 2003 I took part in a non blind AB test between SPDIF coax and toslink cables, to find out whether there are differences in sound at all, and if yes which would sound better. Other than most participants my golden ears told me clearly that coax sounded fuller and wider than toslink. However they still told me this when the person handling the switch did not switch from toslink back to coax but continued with toslink. It was embarassing but a real eye opener for me and forever changed my trust into what I can hear.

You really have to experience this. Reading about it is not sufficient, hence I'm all in favor for doing this kind of test as often as possible.
I concur with this.

I have a couple times been part of inadvertant blind tests. Where switching was done (or not) in error. So we all thought we were doing comparing and in fact nothing had changed. Several people heard and discussed the differences. Everyone was sheepish once it was realized we had mistakenly not listened to anything different in one case. In another case a mixup meant we thought what we were comparing was backwards from what was being compared. Basically same results same feelings.

Most people glossed over it and forgot it after that day. Couple of people had it stick in their mind and eventually helped them come to a realization about how easily sight/knowledge biases what they heard.
 

Pareto Pragmatic

Active Member
Joined
Sep 5, 2023
Messages
244
Likes
268
Location
Upper Mid-West, USA
Interesting that people key in on the light colors, but unless I missed it, no one has mentioned this part:

"After about 15 minutes of listening, there was much discussion among the membership, describing the audible differences between A and B."

See the Asch conformity experiments for just how powerful other people are when talking someone into seeing (or hearing) things that are not true.

A relatively small percentage of people resist such influence on their sense data. IIRC, about 25% in Asch's experiments. But that was for BIG differences in line length that were obvious. If the differences are small or non-existent, then the number of non-influenced people in the situation will be much smaller. Also, more people, more influence, and this seems to have been a pretty big group

I will note that there were a "few" who did not follow the herd after the discussion. Kudos to them.
 

dasdoing

Major Contributor
Joined
May 20, 2020
Messages
4,387
Likes
2,885
Location
Salvador-Bahia-Brasil
It would be interesting to study which colors and or brightness have which effect on the perception of sound. The findings can then enter somehow into tweaking considerations of a system.
 

Ifrit

Active Member
Joined
Apr 10, 2020
Messages
156
Likes
93
I wonder how Ron even got into that group, if it exists. Weird cat he is, and his former AES presidency doesnt’t help. :)
 

Pareto Pragmatic

Active Member
Joined
Sep 5, 2023
Messages
244
Likes
268
Location
Upper Mid-West, USA
It would be interesting to study which colors and or brightness have which effect on the perception of sound. The findings can then enter somehow into tweaking considerations of a system.
Color affects mood, mood affects perception. That's the link.

I did find a study that used VR headset to change the color of a concert hall, and headphones. N=30 so big grain of salt. Red rooms sound best. Just like red cars are faster.

In general, red/orange/yellow give people energy and make them happier, blue/green/purple produce calm and mellow states of mind. So a color that helps for high energy music might detract from more mellow music, and vice versa.

Not my area of expertise, but makes sense to me.
 

IAtaman

Major Contributor
Forum Donor
Joined
Mar 29, 2021
Messages
2,473
Likes
4,360
And that's why blind testing gets the bad rep it has. Some clown tricking his friends attempting to be smarter by not introducing a circuit element he said he had introduced. People were trying to hear what wasn't there. What a waste of time. He lied to them and I'll bet nobody will be interested in his 'tests' ever again.

I'd have given the guy a piece of my mind and walked out- got better things to do in life.
Ha! You seem to be enjoying trolling.
 

AaronJ

Senior Member
Joined
Nov 9, 2021
Messages
331
Likes
589
No.
Imagine if I had arranged the test! those engineers, would not trust me, would not assume anything, and most probably told me to get lost.
The test worked, because the engineers trusted their friend, and his integrity, accepted it was a real test.
Imagine if Amir, had arranged a similar test and his marks were ASR devotees, they probably fall into the same trap. Equally if the guy from PS Audio does the same thing, most ASR members would catch him out.
The trust of the marks, was necessary for this test to produce its false results.
Purveyors of snake oil literally pour all their efforts into building trust in their audience. It’s the sole purpose of a con. The only differences are between the marks.
 

SIY

Grand Contributor
Technical Expert
Joined
Apr 6, 2018
Messages
10,730
Likes
25,831
Location
Alfred, NY
Ha! You seem to be enjoying trolling.
John isn't trolling, but I think he IS missing the concept of "first define the question to be answered by an experiment when designing it." The question to be answered (which was not disclosed to the participants because it would invalidate the experiment) was "Are experienced and professional listeners subject to the placebo effect?" And the experiment clearly showed that they are- totally unsurprising, but since we so often hear claims of immunity due to the claimant's "vast experience," this is a nice data point confirming the contrary.
 

fpitas

Master Contributor
Forum Donor
Joined
Jul 7, 2022
Messages
9,885
Likes
14,274
Location
Northern Virginia, USA
If the participants were free to choose "No discernible difference", in my opinion there was nothing wrong with the test.
 

Ken Tajalli

Major Contributor
Forum Donor
Joined
Sep 8, 2021
Messages
2,238
Likes
2,032
Location
London UK
John isn't trolling, but I think he IS missing the concept of "first define the question to be answered by an experiment when designing it." The question to be answered (which was not disclosed to the participants because it would invalidate the experiment) was "Are experienced and professional listeners subject to the placebo effect?" And the experiment clearly showed that they are- totally unsurprising, but since we so often hear claims of immunity due to the claimant's "vast experience," this is a nice data point confirming the contrary.
Amen
 
Top Bottom