• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Audio Blind Testing - You Are Doing It Wrong! (Video)

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
45,918
Likes
256,370
Location
Seattle Area
Just uploaded a video on the basics of performing controlled tests in audio. It was motivated by saving myself text in having to write all of this down when telling someone who to do these tests right. And as a counter to a few online personalities to keep saying they do "blind" tests yet when in reality they are not doing anything useful.

 
Haven't had time to watch it yet but liked it anyway just because it is sooo needed...
 
mia.JPG


but maybe it's a browser issue, mine is FF, I'll try Edge and report back.
 
Yes it's a browser issue - or one with my uBlock Origin, in some cases it's not really off even if it's turned off for a site (which is the case for ASR).

P.S. Yes, it's uBlock Origin - 3rd party frames must be "extra" enabled, though it's set to off :oops:, anyway, it's working now.
 
Working fine in Chrome and is embedded... good video, comments at the beginning are gold. :D

Anyone using IE (unsupported) still, this browser will no longer work on this site after version update and maybe some older phones.



JSmith
 
Good video, clear and simple explanation. Nice to see that we’d pretty much followed these processes when we compared the Aune S8 and the chord Cutest. The point that I don’t think that I mentioned in my comparison was that the results that we ended up with collaborated with Amir’s measurements of both DAC’s. It’s a shame that there’s currently no method of defining the audio presentation of devices because they were both markedly different, and that’s an aspect that never gets defined. I wonder if it could be possible to set up a mic array across the listening field to accurately measure the soundstage presentation of both different speakers & equipment. I’m pretty sure the BBC used to do similar, I did have a link to an old paper on it, I’ll have a dig through and see if I can find it. :)
 
It’s a shame that there’s currently no method of defining the audio presentation of devices...
Of course there is. But the catch is... no peeking.
 
Yes it's a browser issue - or one with my uBlock Origin, in some cases it's not really off even if it's turned off for a site (which is the case for ASR).

P.S. Yes, it's uBlock Origin - 3rd party frames must be "extra" enabled, though it's set to off :oops:, anyway, it's working now.
I remember the days of using uMatrix, that's even more a puzzle to figure out what to disable/enable.
 
Another great much needed master course on controlled audio testing Amir.

I totally agree with the fast audio A-B switching, something I started doing many years ago after I realized how short auditory memory is. I laugh when I read that a reviewer mentions that he doesn't remember ever hearing a particular piece of music sounding as good as through that special usually very expensive gear at some audio venue.

Another great point on your video is listening with the same reference audio tracks one has been using for many years. I've been using the same audio tracks for over 15 years. It always amuses me when I see professional audio reviewers that list their source music du jour that changes from one month to the next; they don't use a valid reference point for observation which becomes a totally subjective exercice.
 
Wonderful review. Learning how hard it is to keep people from deceiving themselves ought to provoke humility. It certainly has for me. I'm finding it more and more difficult to make perception statements without qualifiers such as "I think" or "it appears that". : )

A note on Matching Levels when the testing involves class D amplifiers. I recently was setting up such a test using my fancy, wide-bandwidth multimeter and could not get a reliable reading on the 1kHz* sine wave. This won't happen with all class D amps, but the one in my test had enough ultrasonic noise that I had to enable a low-pass filter on the multimeter to get the reading. Use a cheap meter and you'll be fine! : )

God bless you and your precious family - Langston

* Don't use 1kHz, Amir recommended 200Hz in the video and that's much better. I didn't think of it until he mentioned it, but 1kHz is painful and potentially expensive if you do something wrong. : )
 
I laugh when I read that a reviewer mentions that he doesn't remember ever hearing a particular piece of music sounding as good
Yes, you are right.
Yesterday
Today

And you cannot say that the today version is better sounding. Because of the short memory.
 
Excellent video. I've been despairing at the number of people claiming to have "easily heard a difference in a blind test" without a whit of documentation of how the test was controlled and the number of trials done. I can hear the difference between sources 2dB apart too, blind or not, and I can get three coin tosses in a row right about once every eight tries :p
 
Excellent video. Not only for audio!
Much needed in times where an increasing number of people think that their subjective personal experiences are more trustworthy than scientific methods.
 
Back
Top Bottom