[YouTube] The Big Measurement & Listening Mistake Some Hi-Fi Reviewers Make - SoundStage! Real Hi-Fi

[diddley]

Yep Erin is first listening and then measuring.

 

[abdo123]

Once again, proper measurements are not an improper bias. They are the most reliable information we have to predict listener performance. Throwing them out is like telling your doctor to first diagnose you with just your verbal description without any diagnostic tests. You wouldn't, right? Why with speaker measurements?

Doctors don't just order tests blindly because of the risk of false positives. in the case of a false positive indeed the doctor would make a biased (and thus wrong) diagnosis, perhaps not the best analogy here.

using measurements to judge gear is only as reliable as the measurements themselves.

 

[amirm]

If you don't believe me, believe Dr. Toole telling Erin exactly the same thing (although there are nuances in there which I hope to cover in a video):

 

[amirm]

using measurements to judge gear is only as reliable as the measurements themselves.

Back by four decades of research, we know spinoroma is highly effectively in predicting listener preference. We have now measured some 150 speakers this way. As a whole, it has been an incredible validation of this measurement system. To throw it all out and say, "let's listen fresh" makes absolutely no sense. Why did we do all this research if all we needed to do is get a gray haired reviewer to listen to a speaker with whatever music, in whatever setting and opine about its sound???

 

[MarkS]

Sadly we don't know what they hear. We know what they are writing. And possibly what they think they are hearing. These are different than the actual sound that arrived at their ear drum.

Now if you put them through a test that showed correlation between what they think they hear and reality, then we would have something to go by. As it is, we have an essay written by them with no idea of its reliability.

Does this mean that you are going to stop writing about your subjective impressions of speakers, and just post the measurements?

Because sadly we don't know what you hear ... Now if you take a test that shows correlation between what you think you hear and reality, then we would have something to go by.

 

[abdo123]

Back by four decades of research, we know spinoroma is highly effectively in predicting listener preference. We have now measured some 150 speakers this way. As a whole, it has been an incredible validation of this measurement system. To throw it all out and say, "let's listen fresh" makes absolutely no sense. Why did we do all this research if all we needed to do is get a gray haired reviewer to listen to a speaker with whatever music, in whatever setting and opine about its sound???

i didn’t mean to say that, i just meant to say that sometimes just listening to the speaker is enough to make an assumption of whether it’s a good speaker or not. (Asking the patient about their symptoms .etc instead of instantly ordering a suite of tests)

then with the spinorama you can get to the nitty gritty and compare fine details between two different speakers ( blood work .etc).

it doesn’t matter really which one goes first as long as people keep realistic expectations of the listening experience.

 

[dfuller]

What I will say no matter what is that with speakers I cannot rely solely on measurements . Electronics, sure, they're electronics - we can quantify everything about them easily. Speakers, even well designed ones, are still seriously subject to preference. Speaker measurements at this point for me are enough to weed out the obviously broken stuff and narrow the field. It's when you get stuff like the Adam A5X and similar that measurements fall apart some. It measures overall pretty good, but I still find it to sound pretty awful.

 

[amirm]

Does this mean that you are going to stop writing about your subjective impressions of speakers, and just post the measurements?

No, no. There is more to this story than I am writing here. The topic is actually quite nuanced and complex. I use a tool that none of the other reviewers do to bind these two domains together.

That aside, there are simple things we can assess by ear reliably that are better than measurements. One is SPL limit. I can easily detect using listening tests and precise music selections the point where the woofer bottoms out and sound goes to hell. As yet, I don't have a measurement to precisely quantify this. But can easily hear it subjectively. Sadly typical subjective reviewer doesn't bother with this test where they could have a defensible case.

Even with electronics this type of listening test works. I use it for headphone amplifiers for example. When they run out of power, that is that. They distort and sound worse than any frequency response error then.

 

[Blaspheme]

Listening first doesn't mean we "throw out the measurements". Asserting that is merely pearl-clutching.

Stereophile's protocol makes sense, particularly with a seperate listener and measurement person. I don't mistake JA's commentary—which reflects deeper understanding/experience of the audible consequences of measurements than we often see here—with apologia.

For myself, last time I acquired new speakers, I listened first and made some notes, then measured. I wanted to see how my uninformed listening would stack up. Not exactly the same as a review scenario. But if I were a reviewer I would do the same. So Erin's method is good one for a one-person show, I think.

 

[amirm]

i didn’t mean to say that, i just meant to say that sometimes just listening to the speaker is enough to make an assumption of whether it’s a good speaker or not.

I will quote Dr. Toole in that video: "Listening to a single speaker tells you almost nothing." I do this all the time folks. Two to three times a week I test a speaker and have to decide what the listening experience means. Even though I routinely have the measurements, I don't start by staring at them and listening. I just turn on the system and start to listen. Then comes the big question: "what am I hearing?" Is it bright? How sure are you that it is bright? Is it too boomy? How sure are you that this is not a room mode? Not enough highs? What if your ears are plugged today because you were working in the garden and your ears are plugged? Is it unexciting? What if you are just in that mood? Is it great? How sure are you of that? Maybe you just like these types of speakers?

To answer these questions I then look at the measurements. I analyze them and perform controlled experiments. Only then I think I have a reliable conclusion. Actually sometimes I don't. A few times I have just given up not knowing if I am wrong, or the measurements.

 

[Blaspheme]

... I use a tool that none of the other reviewers do to bind these two domains together. ...

Klippel—have we really banished Erin then? Or something else?

 

[amirm]

Listening first doesn't mean we "throw out the measurements". Asserting that is merely pearl-clutching.

That is precisely the question at hand. Virgin listening or using measurements in conjunction with listening tests as I do.

 

[amirm]

Klippel—have we really banished Erin then? Or something else?

Nope. Klippel has nothing to do with it.

 

[Blaspheme]

Nope. Klippel has nothing to do with it.

Ok, what was the "tool that none of the other reviewers do" that you referred to?

 

[amirm]

Ok, what was the "tool that none of the other reviewers do" that you referred to?

Go read my subjective reviews of headphones and speakers and see if you can figure it out. It is in plain sight....

 

[dominikz]

Go read my subjective reviews of headphones and speakers and see if you can figure it out. It is in plain sight....

Is it EQ?

 

[Blaspheme]

Go read my subjective reviews of headphones and speakers and see if you can figure it out. It is in plain sight....

And people call me passive aggressive.

Is it EQ?

That would make sense.

 

[Blaspheme]

That is precisely the question at hand. Virgin listening or using measurements in conjunction with listening tests as I do.

Listening, then measuring (Erin's method) and discussing audible consequences of measurements (also Stereophile's method) is not throwing out the measurements.

 

[amirm]

And people call me passive aggressive.

They have a very good case....

 

[amirm]

Listening, then measuring (Erin's method) and discussing audible consequences of measurements (also Stereophile's method) is not throwing out the measurements.

Once again, you are not paying attention to the topic. We are discussing whether listening tests should be done with knowledge of measurements or not. There is no half-way mark there. You are either listening with that information or not.