Reviewing Speakers - Measurements and Listening Tests (Video)

[amirm]

Members here are no stranger to the battle between trusting measurements versus listening tests as performed by reviewers at large. It is a difficult topic to try to address in text so a while ago I decided to create a presentation and video for it. It was a harder job than I thought but finally managed to create a cohesive presentation based on research. I go through the formal research on how listening tests are performed and correlated with measurements.

It is a long, 1+ hour presentation but hopefully you find it worthwhile to set aside that much time to watch it (or speed it up).

Research papers:
https://www.aes.org/e-lib/browse.cfm?elib=9822 "
A Survey Study of In-Situ Stereo and Multi-Channel Monitoring Conditions

https://www.aes.org/e-lib/browse.cfm?elib=12206
Differences in Performance and Preference of Trained versus Untrained Listeners in Loudspeaker Tests: A Case Study

 

[beeface]

This is great - very informative

 

[Katji]

Great. Well done.
... I need to put shortcut on desktop and...deal with it when I've slept enough. ...(Doesn't increasing the speed make it like Chipmunks or something? ...Just a bit?)

 

[amirm]

(Doesn't increasing the speed makeit like Chipmunks or something

That was the old method. New method will perform pitch correction so will just play faster. I watched at 1.5X and it is pretty understandable. Gets harder above that for most listeners.

 

[Johannes AU]

Thank you Amir

 

[diablo]

Excellent presentation. Thank you.

My faith in the average reviewer has fallen vastly over the last twenty years. Though I have still let it influence me at times - mainly when I want to have a reason to like something and some of them say nice things about it. Thanks to your info about trained listeners, audio dealers and reviewers relative reliability, I will move their views even lower down my list of things to check.

 

[Snarfie]

Quite informative thanx.

 

[Thomas_A]

Just a question; that speakers must be compared in the same location. That disqualifies comparisons between on-wall, in-wall speakers and stand-mount speakers (?)

 

[AMPaul]

@amirm , you are killing the fun in all the review business with this scientific method. In any case, what do you reckon then, a good scoring system for lazy customers might be? thanks

 

[Bachemar]

That was the old method. New method will perform pitch correction so will just play faster. I watched at 1.5X and it is pretty understandable. Gets harder above that for most listeners.

On iOS, the embedded player on a web page does not have speed controls. You have to share/copy the link and open the YouTube page for the video, to get speed controls

 

[ck42]

Very informative! (listened at 1.5X and completely able to follow along) Definitely appreciate the in-depth presentation, Amir!

Something that I never seem to see mentioned though are the variations that exist with each listener's own physiology (outer ear) and their own biological frequency 'response'. I find myself constantly thinking things like....someone could have a big dip in biological frequency response from 3-5KHz....and so a speaker with a bad response that is measured as being too HOT from 3-5KHz would sound good to this person. And therefore, a speaker with a 'proper' freq response (more flat or whatever house curve) would then sound too weak in that same 3-5KHz range. This would certainly be a good justification for trusting microphone measurements more than relying on subjective reviewers (what if the reviewer's own ears have a crappy freq response? All their reviews, at best, are going to be completely skewed). At the same time though, it could also mean that a OBJECTIVELY good speaker could potentially sound NOT so good to someone.
Forget where I saw it, but I seem to recall seeing something several years ago where a laser could be fired at a person's ear drum and, just like a speaker plot, that person's ear drum's freq response could be measured...and then used to create a corrective eq. Seems like the ultimate 'tweak'

 

[HooStat]

Forget where I saw it, but I seem to recall seeing something several years ago where a laser could be fired at a person's ear drum and, just like a speaker plot, that person's ear drum's freq response could be measured...and then used to create a corrective eq.

My children and my dog would probably be chased out of the room by my corrective tweak.

 

[DualTriode]

@amirm,

Good job.

While we are in analytic mode; in the standard listening room what is the modeled or standard +- 6dB angle of coverage for a CD plus waveguide mid/tweeter for frequencies to fit the standard Directivity Index on a standard spinorama.

The JBL flagship M2 monitor has a waveguide that covers +- 6dB at 120 degrees, vertical and horizontal.

Thanks DT

 

[spacevector]

That was the old method. New method will perform pitch correction so will just play faster. I watched at 1.5X and it is pretty understandable. Gets harder above that for most listeners.

I recently discovered this feature on Youtube and have been watching most my videos at 1.5X.
Saves me a bit of time! Only downside is - now real life seems too slow.

 

[Madlop26]

Thank you Amir, what a great presentation, much informative.
And just yesterday GR-Research put another video about audiophile power cords, of course; he can feel the difference and the people who have tried them can also the difference and blah, blah blah....still he has like 370 thumbs up.
I work in medical field so I am familiar with the scientific method, I can only assume that people not familiar with it may have more difficulty figure out who is who when talking about reality.

 

[AdamG]

Professionally done. Valuable educational videos like this are incredibly helpful. Thanks for your hard work trying to enlighten us!

 

[Michel Forbes]

I suggest this very useful and free audio test material from the EBU:
https://tech.ebu.ch/publications/sqamcd

It's very interesting to listen to each individual acoustic instrument, you may create spectrogram with each file and learn how the sound and timbre are composed.
The voice-over are very useful for a quick testing.

 

[spartaman64]

idk if I 100% agree that you should only test with one speaker and think you should test them in the way they are meant to be used. and if that means a "bad speaker" is going to sound the same as a "good speaker" then doesnt that just mean in actual use they are going to sound the same?

 

[Steve Dallas]

idk if I 100% agree that you should only test with one speaker and think you should test them in the way they are meant to be used. and if that means a "bad speaker" is going to sound the same as a "good speaker" then doesnt that just mean in actual use they are going to sound the same?

Let's consider the next logical step in your argument. If 2 is good and 5 is better, and everything sounds the same by the 5th, why not always listen to 5 speakers? Or why not abandon all this tedium and have everyone buy Bose cubes?

If you are trying to uncover the last 10% of performance, you need to create the conditions to hear minor differences, which means listening to a single speaker.

 

[amirm]

idk if I 100% agree that you should only test with one speaker and think you should test them in the way they are meant to be used. and if that means a "bad speaker" is going to sound the same as a "good speaker" then doesnt that just mean in actual use they are going to sound the same?

Only if spatial effects are there to cause you take your eye off the ball of tonality.