Blind test listening results by professional audio engineers in the Los Angeles area

[restorer-john]

Clearly generally the same as everyone... it helps if people are trained listeners, not "golden ears".

Welcome to How to Listen!

Overview How to Listen is a desktop software application developed by the Harman Internationa l R&D group for the purpose of training an...

harmanhowtolisten.blogspot.com

... but clearly in this case, that wouldn't have helped either!


JSmith

That Harman thing is a 113MB download and that was 2011! What is it, an operating system? LOL.

 

[kemmler3D]

That Harman thing is a 113MB download! What is it, an operating system?

It's probably full of WAV files...

 

[restorer-john]

It's probably full of WAV files...

And viruses.

Connects to Harman and downloads the Revel speaker catalogue and a JBL screensaver you didn't ask for, all while brainwashing you into believing the boom-tizz curve is 'HiFi'...

 

[JSmith]

113MB download

Are you still on 56K dial up there in QLD mate?


JSmith

 

[Mr. Widget]

Everyone is subject to expectation bias. Everyone!

The most interesting thing to me is that we must rely on our ears if we want to actually hear the music instead of simply admiring a set of beautiful plots and yet we absolutely can not trust our ears.

And we have been unable to trust our ears for a very long time. We can imagine that anyone alive today would not mistake an "Edison Disk" with an actual live event, but in 1910 the expectation bias was tilted a bit differently than ours is today.


In 1910, he [Thomas Edison] invented the Edison Diamond Disk Phonograph, which he claimed had “no tone” of its own. To prove it, a series of road shows involving 4,000 live-versus-recorded demonstrations of his phonograph were conducted in auditoriums across the United States At some point during the live music performance there would be a switch over to the recorded performance, and apparently audience members could not tell the difference between the live and recorded performances
After a 1916 live-versus-recorded demonstration in Carnegie Hall, the New York Evening Mail stated “the ear could not tell when it was listening to the phonograph alone, and when to actual voice and reproduction together. Only the eye could discover the truth by noting when the singer’s mouth was open or closed”

Why Live-versus-Recorded Listening Tests Don't Work

Figure 1: Singer Frieda Hempel conducting a Tone Test at Edison Studios, NYC in 1918. Note that many of the listeners' ears are covered by t...

seanolive.blogspot.com

 

[radix]

Or they are both part of a larger category of phenomena where non-aural stimuli are mistaken for actual aural differences.

yes, this was the spirit in which I thought the McGurk effect is analogous. The brain lies to you. If you have some expectation of A, you can hear A even if it's not there. The fake AB test was, like many psychology experiments, purported to do one thing to measure something else. It wasn't really about AB tests.

 

[dweeeeb2]

This test has confused me. Am I actually better spending money on the science or the story? Sounds like the story is more important.

 

[kemmler3D]

The most interesting thing to me is that we must rely on our ears if we want to actually hear the music instead of simply admiring a set of beautiful plots and yet we absolutely can not trust our ears.

And we have been unable to trust our ears for a very long time. We can imagine that anyone alive today would not mistake an "Edison Disk" with an actual live event, but in 1910 the expectation bias was tilted a bit differently than ours is today.


In 1910, he [Thomas Edison] invented the Edison Diamond Disk Phonograph, which he claimed had “no tone” of its own. To prove it, a series of road shows involving 4,000 live-versus-recorded demonstrations of his phonograph were conducted in auditoriums across the United States At some point during the live music performance there would be a switch over to the recorded performance, and apparently audience members could not tell the difference between the live and recorded performances
After a 1916 live-versus-recorded demonstration in Carnegie Hall, the New York Evening Mail stated “the ear could not tell when it was listening to the phonograph alone, and when to actual voice and reproduction together. Only the eye could discover the truth by noting when the singer’s mouth was open or closed”

Why Live-versus-Recorded Listening Tests Don't Work

Figure 1: Singer Frieda Hempel conducting a Tone Test at Edison Studios, NYC in 1918. Note that many of the listeners' ears are covered by t...

seanolive.blogspot.com

Really interesting. But I could imagine (even in 1916) that if you put the best mono speaker you could find, up against a singer on stage, it might not sound so bad, especially if you got the singer to try and sound a bit like a record.

 

[LTig]

Indeed, that's why I think it was a worthy "experiment". It's a clear demonstration of how cognitive bias makes us think we heard things that we didn't hear. [..]

Disagree, there are plenty of people who resist/deny the entire notion of cognitive bias and false positives in listening tests. And then they turn around and spend $10K on a cable. Showing that even elite professionals are subject to these effects can't hurt. It might open some minds and save some wallets.

Yep. In 2003 I took part in a non blind AB test between SPDIF coax and toslink cables, to find out whether there are differences in sound at all, and if yes which would sound better. Other than most participants my golden ears told me clearly that coax sounded fuller and wider than toslink. However they still told me this when the person handling the switch did not switch from toslink back to coax but continued with toslink. It was embarassing but a real eye opener for me and forever changed my trust into what I can hear.

You really have to experience this. Reading about it is not sufficient, hence I'm all in favor for doing this kind of test as often as possible.

 

[Blumlein 88]

Yep. In 2003 I took part in a non blind AB test between SPDIF coax and toslink cables, to find out whether there are differences in sound at all, and if yes which would sound better. Other than most participants my golden ears told me clearly that coax sounded fuller and wider than toslink. However they still told me this when the person handling the switch did not switch from toslink back to coax but continued with toslink. It was embarassing but a real eye opener for me and forever changed my trust into what I can hear.

You really have to experience this. Reading about it is not sufficient, hence I'm all in favor for doing this kind of test as often as possible.

I concur with this.

I have a couple times been part of inadvertant blind tests. Where switching was done (or not) in error. So we all thought we were doing comparing and in fact nothing had changed. Several people heard and discussed the differences. Everyone was sheepish once it was realized we had mistakenly not listened to anything different in one case. In another case a mixup meant we thought what we were comparing was backwards from what was being compared. Basically same results same feelings.

Most people glossed over it and forgot it after that day. Couple of people had it stick in their mind and eventually helped them come to a realization about how easily sight/knowledge biases what they heard.

 

[Pareto Pragmatic]

Interesting that people key in on the light colors, but unless I missed it, no one has mentioned this part:

"After about 15 minutes of listening, there was much discussion among the membership, describing the audible differences between A and B."

See the Asch conformity experiments for just how powerful other people are when talking someone into seeing (or hearing) things that are not true.

A relatively small percentage of people resist such influence on their sense data. IIRC, about 25% in Asch's experiments. But that was for BIG differences in line length that were obvious. If the differences are small or non-existent, then the number of non-influenced people in the situation will be much smaller. Also, more people, more influence, and this seems to have been a pretty big group

I will note that there were a "few" who did not follow the herd after the discussion. Kudos to them.

 

[dasdoing]

It would be interesting to study which colors and or brightness have which effect on the perception of sound. The findings can then enter somehow into tweaking considerations of a system.

 

[Ifrit]

I wonder how Ron even got into that group, if it exists. Weird cat he is, and his former AES presidency doesnt’t help.

 

[Pareto Pragmatic]

It would be interesting to study which colors and or brightness have which effect on the perception of sound. The findings can then enter somehow into tweaking considerations of a system.

Color affects mood, mood affects perception. That's the link.

I did find a study that used VR headset to change the color of a concert hall, and headphones. N=30 so big grain of salt. Red rooms sound best. Just like red cars are faster.

In general, red/orange/yellow give people energy and make them happier, blue/green/purple produce calm and mellow states of mind. So a color that helps for high energy music might detract from more mellow music, and vice versa.

Not my area of expertise, but makes sense to me.

 

[IAtaman]

And that's why blind testing gets the bad rep it has. Some clown tricking his friends attempting to be smarter by not introducing a circuit element he said he had introduced. People were trying to hear what wasn't there. What a waste of time. He lied to them and I'll bet nobody will be interested in his 'tests' ever again.

I'd have given the guy a piece of my mind and walked out- got better things to do in life.

Ha! You seem to be enjoying trolling.

 

[IAtaman]

This test has confused me. Am I actually better spending money on the science or the story? Sounds like the story is more important.

Lean in on the placebo. It is much cheaper, and as effective.

 

[AaronJ]

No.
Imagine if I had arranged the test! those engineers, would not trust me, would not assume anything, and most probably told me to get lost.
The test worked, because the engineers trusted their friend, and his integrity, accepted it was a real test.
Imagine if Amir, had arranged a similar test and his marks were ASR devotees, they probably fall into the same trap. Equally if the guy from PS Audio does the same thing, most ASR members would catch him out.
The trust of the marks, was necessary for this test to produce its false results.

Purveyors of snake oil literally pour all their efforts into building trust in their audience. It’s the sole purpose of a con. The only differences are between the marks.

 

[SIY]

Ha! You seem to be enjoying trolling.

John isn't trolling, but I think he IS missing the concept of "first define the question to be answered by an experiment when designing it." The question to be answered (which was not disclosed to the participants because it would invalidate the experiment) was "Are experienced and professional listeners subject to the placebo effect?" And the experiment clearly showed that they are- totally unsurprising, but since we so often hear claims of immunity due to the claimant's "vast experience," this is a nice data point confirming the contrary.

 

[fpitas]

If the participants were free to choose "No discernible difference", in my opinion there was nothing wrong with the test.

 

[Ken Tajalli]

John isn't trolling, but I think he IS missing the concept of "first define the question to be answered by an experiment when designing it." The question to be answered (which was not disclosed to the participants because it would invalidate the experiment) was "Are experienced and professional listeners subject to the placebo effect?" And the experiment clearly showed that they are- totally unsurprising, but since we so often hear claims of immunity due to the claimant's "vast experience," this is a nice data point confirming the contrary.

Amen