Audio Blind Tests and Listener Training (video)

[amirm]

I see so many people claiming to have done blind tests in youtube videos recently without knowing what it means to perform one. Also, the topic of training comes up with many abusing the term. So I decided on a whim to create a video on it. It is rather long at 44 minutes but you can speed it up if you like. I like to use it as a reference in the future so I don't have to keep typing the same answers over and over again.

There is also a reference to an article I wrote a few years ago based on research published at AES on usability of long term listening in blind tests: https://www.audiosciencereview.com/...ity-and-reliability-of-abx-blind-testing.186/

Please let me know what you think of these lecture heavy videos.

 

[Chrispy]

Been watching separately for a bit now (from new stuff on my youtube feed)...my favorite video so far!

 

[wpwoodjr]

Fascinating video, confirmed a lot of my thoughts as well. Years ago I used a track called fatboy to stress test compression bit rate and algorithms. It was very educational, you could hear a sort of tearing sound if the codec wasn't really good. It was apparent to me that WMA was less good than MP3 at the same bit rate. At 256 kbs MP3, fatboy sounded pretty good, at 320 I really couldn't tell. At the same bit rate, AAC was better than WMA and MP3.

 

[xarkkon]

cool vid! i particularly liked the part where you went through the various file tests. I've gone through so many of those and could never know what to look out for. will try them again in the future with the tips you've provided.

love @Archimago's blog as well. found out about raspberry pi streamers objective performance through his blog and am thinking of building a pass aca because of that too.

 

[Docevil]

I'm actually really enjoying them to be honest, very informative!

 

[diddley]

Great video, love Ethan discoverd him through ASR, great anekdotes, 45minutes passed quickly.Thanks Amir

 

[Pdxwayne]

I like it.

I have been told no one can hear 0.2db channel imbalance with music.

@amirm, have you done such blind tests?

 

[B4ICU]

The human ear is not a measuring instrument. It's ability fades with age and exposing it to loud sounds, makes it worse.
Advising you all, to get a hearing test every while, if you consider yourself an audiophile. Just to get a proof of what you can hear and what is lost...
All listening tests (A-B) or other, are from memory. No test can be conducted by listening to both at the same time and compare, like looking on the "Delta" output of an Op. Amp. with a scope.
A-B is a short one. Playing a track with A and than do the swap and listen with B is a longer one.
When A-B is done, there are some significant issues with the test equipment and the switching device. I speak about Amp's and speakers, and even speaker cables.
Many audiophiles claim to have golden ears. Regardless of the test. They say...so what. It's going to play at your home, not his. More than that,
Some says might be biassed. Salesman kind of bias, look of the equipment bias, ownership bias and more. This is what the blind test comes to proof:
No bias.
For yourself: to remove the bias, on some occasions, it may help. The trick for a good blind test is to come with a very familiar track. One, you know well and you have it as a ref.

 

[posvibes]

Thanks Amir, that was really incredibly interesting. Well done! More power to you.

 

[zyghom]

I cannot even express myself how much I agree with your video today. Especially the "few seconds" memory.
It is like with the untrained "lungs": give 5 different cigarettes to someone who never smoked - he will NEVER tell the difference.
Or the beer to someone who never tried.
Examples can be multiplied here - it is always about experience/training in the subject.
Another story is: if we cannot distinguish the subtle differences: why do we go for "nirvana"? Especially if in the real application we stay with 1 system at home? (ok, ok, some have 10 systems - so what?
Because... we don't want to "fail". Our ears, our brains, our...selves

 

[Killy]

Fantastic video probably the most informative Hi-Fi video I’ve watched and your balance adds so much to your credibility.

Confirmed that I wasn’t going mad. So many times I’ve listened to different sources such as Apple Music vs hi-res Qobuz and really struggled to hear anything different. This was so hard as I was listening to a 10 to 20 second clip and normally it takes about 5 secs to change source. I just couldn’t make out any difference. Now I know why.

Really disappointed in the bull that the mainstream reviewers come out with now. I gave them the benefit of doubt as I don’t know how to critically listen. The descriptions they give of veil lifting, the singer is now in the room etc etc, I think they have a random phrase generator for the reviews. Also helps you to gauge the credibility of the audiophile who constantly states they can pick out the differences instantly trashing the ‘perceived’ lower quality stream or equipment. I also remember reading one post where an owner of massively expensive power cables >$3000 commented that they didn’t work well when first plugged in and needed to stabilise and when he came back to them in the afternoon they were fantastic.

I’ve seen the same at Dealers they’ve changed cables, support stands etc and they go into some form of rapture as their feet and legs bounce to the ‘difference’. I wanted to hear the difference but couldn’t. At the time it saved me a fortune. I always suspected PRaT was an in joke in the industry, now I’m sure it is.

This is a fantastic site thank you.

 

[AlexHRider]

Just one thought about the overlaid title on the top, i'd reduce the dimensions cause it's covering a good part of the screen and keeps distracting me xD

 

[johnnyx]

We have a music collection precisely because our memory for sound is so poor. Otherwise, we would only need to hear a track once, and that would be it.

 

[B4ICU]

Off Topic: EQ.
On some items that gets reviewed here, as headphones and speakers, The final comment goes like: I can not recommend it, without EQ!
EQ in high end is an absolute no go. I better invest a few more $ in a better sounding device, than put the money and effort in EQ.
EQ has some very unpleasant features, that make me doubt such a recommandation.
1. When a band (FR) is amplified compared to an other band, let's say high frequencies boost of +6dB, over the rest, it is actually drops the output power (FS) by 75%. If that was driven by a 100W Amp' now for the rest only 25W left!
2. As so, the SNR of the system just dropped by 6dB, regardless how good it was before.
EQ always come with some THD. When that is put into that article with -135dB noise level, I'll go to converter of noise (dB) and THD (%):
http://www.sengpielaudio.com/calculator-thd.htm
3. A typical EQ THD figure would routine SNR by 30-40 dB and get THD above 0.1%. It is like pay an arm and a leg for the sound system and than get it all lost for a "recommended" EQ. What a shame...

 

[CrankyRat]

Excellent Amir. Your explanatory videos are some of the most enjoyable to view. They are very helpful and I encourage you to continue. Thank you very much.

 

[BDWoody]

I was hoping you'd do some just like this.
Some fundamentals will go a long way.

Think back over the questions/challenges that come up here pretty much every day...

Sampling theory would be a great one, covering many birds with one stone.

Monty is great, but it might be nice to post your link instead of his!

 

[magicscreen]

will try them again in the future with the tips you've provided.

But you do not have to.
Amir has proved it, it is enough for us.
Thank you very much, Amir!

 

[Plcamp]

I particularly like the deliberately inserted distortion investigations.

It implies a study that deliberately inserted various defects, each of which could be continuously controllable in intensity, would reveal which defects matter to exactly what degree.

We used to do something similar when testing rf susceptibility of phone equipment, where we would test until failure (ie past the acceptance test limits) to gauge the robustness of competing solutions. One interesting anecdote on that is the discovery that two sources for a telephone 4 layer PCB had very slightly different inter layer spacing...it made a 20db difference to conducted rf susceptibility. Another interesting one was a similar phone with two sources for its control processor...one had 40 dB worse emissions performance, traced eventually to a simultaneous conduction problem in the driver stage on one source’s buss outputs.

Are we missing something by not testing audio equipment for things like the level of rf noise it can withstand prior to detectable performance degradation? Should a test deliberately insert increasing power supply noise into DACs to gauge as a figure of merit which ones are most to least immune?

I don’t know answers to this but I do think it’s a reasonable question.

 

[Plcamp]

I can thing of other ways a square wave could be fuzzy that don’t relate to intended amp bandwidth. An unintended resonance could cause under damped ringing, and a feedback loop instability could do the same.

I believe that if such other contributors were present, they would manifest in the FFT view as a difference in the envelope of the harmonics seen vs what you would expect if it were solely due to bandwidth limitations?

 

[Deleted member 27694]

Off Topic: EQ.
On some items that gets reviewed here, as headphones and speakers, The final comment goes like: I can not recommend it, without EQ!
EQ in high end is an absolute no go. I better invest a few more $ in a better sounding device, than put the money and effort in EQ.
EQ has some very unpleasant features, that make me doubt such a recommandation.
1. When a band (FR) is amplified compared to an other band, let's say high frequencies boost of +6dB, over the rest, it is actually drops the output power (FS) by 75%. If that was driven by a 100W Amp' now for the rest only 25W left!
2. As so, the SNR of the system just dropped by 6dB, regardless how good it was before.
EQ always come with some THD. When that is put into that article with -135dB noise level, I'll go to converter of noise (dB) and THD (%):
http://www.sengpielaudio.com/calculator-thd.htm
3. A typical EQ THD figure would routine SNR by 30-40 dB and get THD above 0.1%. It is like pay an arm and a leg for the sound system and than get it all lost for a "recommended" EQ. What a shame...

Interesting. Since I have about zero technical knowledge I would like to know if this is also true for headphones?
Thank you!

PS I never use EQ because i am not bothered by e.g. something like the treble spike on my DT 1990. I notice that it is different than my akg 371 or er2se but I like it nevertheless.