Do Your Eyes Trick Your Ears - Do We Need Blind Testing?

[Deleted member 14468]

I'm wondering what the ASR consensus is on blind testing. I have no doubt that our ears can be easily tricked -- and I've been thinking about this topic since we did this short interview with David Chesky. What do the ASR readers/posters think?

Doug Schneider
SoundStage!

 

[SIY]

Most basic control.

edit: This said, the Chesky recordings I’ve heard have been excellent.

 

[pozz]

I'm wondering what the ASR consensus is on blind testing. I have no doubt that our ears can be easily tricked -- and I've been thinking about this topic since we did this short interview with David Chesky.

In controlled circumstances you can measure the variance in responses. When the error rate is high (inconsistent responses) you can conclude that the listener was guessing and unable to distinguish between the various sources/speakers under test.

So the argument that you can be tricked or affected by bias is not really a reason to disregard the results, since that element is already an important consideration in serious blind testing.

In short: the effects of trickery can be measured.

 

[Eetu]

The absence of Chesky Records on Tidal is a bummer.

I think it's safe to say the ASR membership is pro (double) blind testing contra sighted listening when a/b testing.

 

[Xulonn]

Listening to music with your eyes open or closed is subjective and many variables can have an effect on your "perception". The same music can "sound" different in the same environment at different times depending even on your mood or energy level. Who cares? That's all fun and part of the enjoyment of listening to recorded music. I don't expect a perfect recreation of the physical aspects of the original soundfield.

I was an usher for the Chicago Symphony Orchestra in my high school days in the late 1950's, and I have most of the classic RCA stereo recordings from that era. I enjoy those recordings without feeling any angst over whether my listening sessions perfectly recreate the original experience.

But claiming to hear differences in "sound quality" - or indeed any sonic difference - between electromechanical devices where no differences in frequency response, distortion or noise exist is what will cause science-literate ASR members to protest vigorously. It is purveyors of such false claims that invite ridicule and derision after they rail to acknowledge reality.

 

[Wes]

eyes, taste, smell can all trick the ears (or better put: cross sensory effects modulate perception)

there are entire laboratories dedicated to such cross sensory effects - some at universities, but much of the research is top secret and done at large food and perfume companies

 

[Blumlein 88]

I find watching on a large projector some good video of concerts, the eye candy makes the sound quality less important. Not the sound less important, just that you are more forgiving when you have video. At least with good concert videos that don't jump cut all over the place.

I'd say ASR is very pro double blind testing. No reason not to let video improve our experience of music. However, if we are trying to find the reality of reproduced sound then blind listening is one of the biggest tools we have that won't fool us.

 

[Speedskater]

Ears only testing and eyes closed listening are two different things, entirely.

 

[solderdude]

Blind and level matched testing is an important tool to combat cognitive bias.
It is not easy to do but usually is enlightening when done well.
Often this isn't the case which can skew results.

The term blind testing is confusing as it does not have to be with eyes closed or blindfolded.
A better description would be 'not knowing what one listens to'.
Double blind isn't always needed but can be relevant when the test results are used for truthfinding (court of law style) or for important papers.

Then there is the 'getting used to' issue. The brain is very good at this and can make a new 'reference' which may not be a reference.
I think it is important to have a reference available to reset one's brain. Of course this is not needed when trying to find one's personal preference.
The latter is another obstacle as is room and HRTF.

'Blind' comparison of electronics and digital music files is relatively easy. Speakers and above all headphones is another story (so is vinyl cartridge+RIAA amp)

 

[Blujackaal]

Anything can fool the brain with biases. Like audiophiles assuming that 192 - 320kbps is a must when lossy codecs can be tranparent at 80 - 170kbps.

 

[Eetu]

Anything can fool the brain with biases. Like audiophiles assuming that 192 - 320kbps is a must when lossy codecs can be tranparent at 80 - 170kbps.

I believe there's been way too much fussing over high res/mqa/dsd etc. but there's also been studies showing that people prefer high res over cd quality. Which codec is transparent at 80kbps and why would you use one if 320 (or lossless) is readily available?

 

[Blujackaal]

I believe there's been way too much fussing over high res/mqa/dsd etc. but there's also been studies showing that people prefer high res over cd quality. Which codec is transparent at 80kbps and why would you use one if 320 (or lossless) is readily available?

Opus 1.3, Why would i use a setting that waste of space when 160kbps VBR can shoot up to 320+ on challenging samples?. Many of my Ambient albums avg 420kbps in lossless, That would mean some tracks are larger with no sound gains. People use android phones with <128gb cards since there dirt cheap & not everyone can tell lossless from lossy(even at <192kbps).

There a reason why CBR was phased out in favour for VBR modes.

 

[mhardy6647]

eyes, taste, smell can all trick the ears (or better put: cross sensory effects modulate perception)

there are entire laboratories dedicated to such cross sensory effects - some at universities, but much of the research is top secret and done at large food and perfume companies

ain't it the truth?!
Even grocery stores figured this out -- modern food stores arrange their traffic patterns carefully, right down to what one smells as one walks in the door.
As a kid, I can remember the unmistakable smell of fish in a grocery store -- but "they" manage that much better now
https://flavourjournal.biomedcentral.com/articles/10.1186/s13411-015-0041-1

 

[Eetu]

Opus 1.3, Why would i use a setting that waste of space when 160kbps VBR can shoot up to 320+ on challenging samples?. Many of my Ambient albums avg 420kbps in lossless, That would mean some tracks are larger with no sound gains. People use android phones with <128gb cards since there dirt cheap & not everyone can tell lossless from lossy(even at <192kbps).

So you like the file management hassle of local lossless/hi-res but with the added benefit of lower sound quality than that of streaming services

Jk, I can understand there's reasons going your route. For me the pros of a service like Tidal (w/ Roon) outweigh the cons but good to hear that the Opus codec's working for you.

 

[Blujackaal]

So you like the file management hassle of local lossless/hi-res but with the added benefit of lower sound quality than that of streaming services

Jk, I can understand there's reasons going your route. For me the pros of a service like Tidal (w/ Roon) outweigh the cons but good to hear that the Opus codec's working for you.

I use Musepack --standard on my phone, As my main music hub since i get ground noise on my PC. Tranparent to me even with challenging samples insane it overlooked since it a gem for the 160kbps area. My lossless files are just treated as a archive, Which is why I'm thinking of getting a back up HDD for the important files.

Size means nothing since i could do 384kbps wavpack hybrid with a correction file for the back up.

 

[GD Fan]

ain't it the truth?!
Even grocery stores figured this out -- modern food stores arrange their traffic patterns carefully, right down to what one smells as one walks in the door.
As a kid, I can remember the unmistakable smell of fish in a grocery store -- but "they" manage that much better now
https://flavourjournal.biomedcentral.com/articles/10.1186/s13411-015-0041-1

Glass of wine #2 vs #3 is also not to be discounted when listening, but I digress.

 

[Eetu]

I use Musepack --standard on my phone, As my main music hub since i get ground noise on my PC. Tranparent to me even with challenging samples insane it overlooked since it a gem for the 160kbps area. My lossless files are just treated as a archive, Which is why I'm thinking of getting a back up HDD for the important files.

Have you thought about getting a NAS drive? You could then play all your files with your phone using Plex or similar and not have to worry about phone storage.

 

[Milesian]

Since the general belief here is that anything that measures the same sounds the same, there’s really no point in blind testing. It’s just an academic exercise without value. Best to spend any virus free time you have left in more creative or philanthropic pursuits.

 

[LTig]

I find watching on a large projector some good video of concerts, the eye candy makes the sound quality less important. Not the sound less important, just that you are more forgiving when you have video. At least with good concert videos that don't jump cut all over the place.

This is my experience as well. Just never close your eyes while viewing, it may spoil the fun when you realize how bad the sound actually is.

I'd say ASR is very pro double blind testing. No reason not to let video improve our experience of music. However, if we are trying to find the reality of reproduced sound then blind listening is one of the biggest tools we have that won't fool us.

+1

 

[Berwhale]

Opus 1.3, Why would i use a setting that waste of space when 160kbps VBR can shoot up to 320+ on challenging samples?. Many of my Ambient albums avg 420kbps in lossless, That would mean some tracks are larger with no sound gains. People use android phones with <128gb cards since there dirt cheap & not everyone can tell lossless from lossy(even at <192kbps).

There a reason why CBR was phased out in favour for VBR modes.

Storage is cheap; I have 1TB of storage on my phone, I have no need for lossy compression and store everything in FLAC. I am not claiming that I could hear the difference between 320VBR and FLAC, but I simply have no need to find out because the potential monetary saving in storage cost is not worth my time.