[YouTube] The Big Measurement & Listening Mistake Some Hi-Fi Reviewers Make - SoundStage! Real Hi-Fi

[Frgirard]

In case of two opposing reviews of the same product, which reviewer do you believe? and why?

Isn't that the magic of audiophile forums: from cables to speakers ...

do you think reviewers are people with extra power ?

 

[Sharur]

I'm really struggling to wrap my head around the complaints. Is it because ASR is often viewed as a go-to authority in the audiophile world that people are complaining about @amirm's methods? Why don't you ask yourself how this came to be? The solution to your problem is extremely simple: go read stereophile if you want subjective impressions first and measurements second.

 

[gsp1971]

do you think reviewers are people with extra power ?

No extraordinary powers required. Just formal training.

Imagine if there were the following disclaimer at the end of each review:

"The reviewer is not formally trained to correctly identify speaker performance through listening. All comments expressed are based solely on the reviewer's subjective impressions on the day of the review and will not necessarily apply in your case or anyone else's"

 

[goat76]

The very fact that different reviewers from different sites / publications give completely different opinions on the same product is a proof in itself that listening without measuring is an inconsistent process.

In case of two opposing reviews of the same product, which reviewer do you believe? and why?

But why would you focus on just those two reviews if they have completely different opinions?

Just like the thing Amir says about the listening tests with at least 3 or 4 speakers to know which speaker is the more balanced one, you should obviously read more reviews to find out the trend and some similarities.
If you see two reviews with their own measurements of the same speaker and the measurements don't match up, who do you trust?

When I'm interested in a product I scan the whole internet for listening impressions, I look for similar impressions from as many users and reviewers as I can find, both for the good and bad. In the end, I get a pretty good overall idea if the product is something that I think will match up to my liking. Most of us probably do this more or less no matter if we are good at interpreting measurements or gathering lots of written impressions and sort out the similarities.
In the end, we can't be completely sure we like the product before we hear it with our own ears (and hopefully in our own listening room).

 

[Sharur]

But why would you focus on just those two reviews if they have completely different opinions?

Just like the thing Amir says about the listening tests with at least 3 or 4 speakers to know which speaker is the more balanced one, you should obviously read more reviews to find out the trend and some similarities.
If you see two reviews with their own measurements of the same speaker and the measurements don't match up, who do you trust?

When I'm interested in a product I scan the whole internet for listening impressions, I look for similar impressions from as many users and reviewers as I can find, both for the good and bad. In the end, I get a pretty good overall idea if the product is something that I think will match up to my liking. Most of us probably do this more or less no matter if we are good at interpreting measurements or gathering lots of written impressions and sort out the similarities.
In the end, we can't be completely sure we like the product before we hear it with our own ears (and hopefully in our own listening room).

Can you give me an example of an objectively fantastic product that is not critically acclaimed?

 

[gsp1971]

But why would you focus on just those two reviews if they have completely different opinions?

How many reviews are enough then? 5? 10?

If you see two reviews with their own measurements of the same speaker and the measurements don't match up, who do you trust?

Provided they know what they are doing then:

• If they are using the same equipment (e.g. both use Klippel), the measurements should match.
• If they are using different equipment (e.g. one is using Klippel, one is using Clio), the measurements should not be far off.

It is very unlikely that one set of measurements will show treble elevated by 5dB, while the other set of measurements will show treble reduced by 5dB. While resolution and graph scaling might differ, the general idea should be the same in both measurements.

When I'm interested in a product I scan the whole internet for listening impressions, I look for similar impressions from as many users and reviewers as I can find, both for the good and bad.

So, apart from reviewers (who may not be formally trained but at least have some experience because they do this for a living), you also rely on other users' subjective evaluations? What brings you to ASR then?

In the end, we can't be completely sure we like the product before we hear it with our own ears (and hopefully in our own listening room).

Bingo! I could not agree more!
So, all it takes is a set of measurements in order to make a shortlist of well-engineered products, and then audition them yourself. It doesn't matter what 100 other users in other forums have said.

 

[witwald]

But why would you focus on just those two reviews if they have completely different opinions?

Why would you expect any subjective review to be useful in and of itself if its opinions cannot be independently corroborated?

Just like the thing Amir says about the listening tests with at least 3 or 4 speakers to know which speaker is the more balanced one, you should obviously read more reviews to find out the trend and some similarities.

And there you have it in a nutshell: the need to read more and more subjective opinions, each encumbered by the expectation that what is opined is actually useful in the greater context of loudspeaker performance assessment. It all points to the "golden eared" being the ones whose opining gains the greatest credence with the readers/listeners/viewers.

When I'm interested in a product I scan the whole internet for listening impressions, I look for similar impressions from as many users and reviewers as I can find, both for the good and bad. In the end, I get a pretty good overall idea if the product is something that I think will match up to my liking.

And the "golden eared" influencers are more than happy to oblige, providing lots of opinions for general consumption, uncorrelated with each other and flavoured with prose to target their audience. It is not unusual to see a lack of measurements, either before or after the "listening sessions" on which those opinions are based.

 

[goat76]

Can you give me an example of an objectively fantastic product that is not critically acclaimed?

I can't, but I still don't know if I personally will like the product even if it shows good measurements.

I simply want more information and for that, I like to read other users' opinions, both from current and previous owners who have used the product for a long period of time. A single reviewer's opinion is not much to go by, I need to see more of a trend and if a lot of users (and reviewers) are reporting similar things that I personally like about a speaker, I put it on the list.

 

[egellings]

Listening rooms will likely influence the sound of a speaker, and if, say, a room has a suckout at some midbass frequency and the speaker just so happens to have a little bit of a resonance there, the two could cancel out, yielding good SQ. Such a scenario is highly unlikely, though. If the room sounds awful, active EQ may be the only solution.

 

[MaxBuck]

No extraordinary powers required. Just formal training.

View attachment 144620

Imagine if there were the following disclaimer at the end of each review:

"The reviewer is not formally trained to correctly identify speaker performance through listening. All comments expressed are based solely on the reviewer's subjective impressions on the day of the review and will not necessarily apply in your case or anyone else's"

Yeah, well, no one ever confused Stereophile with Consumer Reports.

 

[goat76]

How many reviews are enough then? 5? 10?

I read every bit of information I can find on products I'm interested in, including measurements from Amir or someone else who do that stuff.

Provided they know what they are doing then:

• If they are using the same equipment (e.g. both use Klippel), the measurements should match.
• If they are using different equipment (e.g. one is using Klippel, one is using Clio), the measurements should not be far off.

It is very unlikely that one set of measurements will show treble elevated by 5dB, while the other set of measurements will show treble reduced by 5dB. While resolution and graph scaling might differ, the general idea should be the same in both measurements.

Clearly, one of them doesn't know how to make proper measurements.

So, apart from reviewers (who may not be formally trained but at least have some experience because they do this for a living), you also rely on other users' subjective evaluations? What brings you to ASR then?

As I said, I collect all the information I can on a product I'm interested in. I read measurements, I read reviews, and I read people's opinions on different forums and gather all the information I can. I think most of us do this as one of the steps before we go and buy the speaker.

What brings me to ASR? Probably the same reasons you are here, the measurements, reviews and the forum.

Bingo! I could not agree more!
So, all it takes is a set of measurements in order to make a shortlist of well-engineered products, and then audition them yourself. It doesn't matter what 100 other users in other forums have said.

I think it's fun to read other people's opinions and collect all this information I can about the product I'm interested in.

A set of measurements doesn't necessarily tell us the full story. Let's say the measurements of the speakers show they are lacking bass, well maybe they are designed to be placed close to the wall? Maybe they seem too hot in the treble on-axis, maybe they are not designed to have any toe-in? How do they perform in stereo?
Maybe most of the users know these things and can give us a more accurate report of how these speakers perform when set up properly?

 

[gsp1971]

A set of measurements doesn't necessarily tell us the full story.

Floyd Toole and Sean Olive seem to think otherwise.

Let's say the measurements of the speakers show they are lacking bass, well maybe they are designed to be placed close to the wall? Maybe they seem too hot in the treble on-axis, maybe they are not designed to have any toe-in?

The Spinorama measurements provide answers to all these questions. Perhaps you need to familiarize yourself a little more with the measurements.

How do they perform in stereo?

I don't understand this question ...

Maybe most of the users know these things and can give us a more accurate report of how these speakers perform when set up properly?

No comment ...

 

[tuga]

Measuring a device before subjective evaluation helps the reviewer provide an informed opinion on the product.

In my opinion, measuring before listening induces bias.

 

[tuga]

I personally find the pre-measurement listening to be good information. It's a decent way to correlate (or not) measurements to what someone heard - after all, we are listening to speakers at the end of the day, not reading the measurements.

What's there to stop you from correlating your listening assessment with measurements after you have listened?

It doesn't make sense fo measure before as it will induce bias. It's as "unsicentific" as sighted listening.

 

[amirm]

In my opinion, measuring before listening induces bias.

I think you are biased to dislike measurements.

 

[tuga]

I think you are biased to dislike measurements.

You are so wrong. I find them an invaluable tool for assessing performance/accuracy.
You might remember that I was that pesky guy who kept asking you to perform measurements of individual drivers and spectral decays because the spinoramas were insuficient to characterise performance.
I am also perfectly conscious that listening for pleasure / tasting is a completely different task to critical listening.

Are you affraid to listen before measuring?

 

[amirm]

You are so wrong. I find them an invaluable tool for assessing performance/accuracy

If so why would it impact accuracy of listening tests as you claimed?

 

[Katji]

I think you are biased to dislike measurements.

I'm sure that is a very common reason underlying much of the criticism, arises from a natural lack of technical understanding, then tries to minimise the usefulness of measurements.

 

[tuga]

If so why would it impact accuracy of listening tests as you claimed?

It will condition as much as reading a review, looking at the box or hearing a dealer's drivel.
I am surprised that you even ask?

Don't you endorse blind listening test?

 

[tuga]

I'm sure that is a very common reason underlying much of the criticism, arises from a natural lack of technical understanding, then tries to minimise the usefulness of measurements.

Have you given it a thought? Don't you find that looking at measurements before listening will affect the outcome of the assessment?
Please, cut the patronising cråp...