DO SPECS REALLY MATTER in Audio? - Understanding Speaker Measurements!

[BillG]

My current speakers I bought strictly by how they measured, never heard them before.

The vast majority of my audio equipment has been purchased blind (perhaps I should say, deaf, since we're dealing with audio) based solely on measurements as well, and I've not been disappointed by any of it. As a matter of fact, I've saved myself a significant amount of money by shopping this way.

One item in particular that I purchased recently, an IEM, had a rather poor frequency response, and when I first auditioned it I didn't like it all - I knew that I wouldn't, though, based upon its measurements. However, I purchased it regardless as I had hoped its transducers - a multiple balanced armature configuration - would respond well to EQ without audibly distorting. Its base THD characteristics were quite good and I guessed that would continue to be the case with equalization.

My hunch was correct and it's now my favorite IEM out of my collection.

So even a poorly measuring device can be transformed into a great sounding one if one knows what to look for in the measurements and can deduce its potential knowing how to correct the faults.

 

[ROOSKIE]

Unfortunately that is not something we as consumers can do something about. Our best bet is to just buy the best loudspeakers we can afford for the money we want to spend on it and hopefully recording/mastering standards follow. The alternative is to introduce yet another random factor in the chain.

Start with great speakers and be prepared to adjust. I expect to see many more speakers like the Buchardt A500/700 with multiple tunings available via DSP and a click of a remote. Wouldn't be hard to have something like the JBL 708P with a ton of presets.

This way if you know the album/tracks well you can adjust your EQ a bit in a basic way to compensate. (Of course this can be cumbersome.)

Mastering standards may be a future reality however I bet you will still want to listen to content already produced that has not been nor ever will be remastered to the new standard.
Additionally there are many wonderful live recording made with various skill level that are well worth listening to for variety - many of those might require some sort of EQ tweak.

Plus in no way are all albums mastered for maximum sound quality - many compromises are made so the "art" sounds good on many playback devices. In no way does what is on the album have to be considered what the artist really intended.

I'm just glad that both sides have a voice and maybe this will start more dialogue because clearly there's some kind of gap starting to form. And if you don't like what a youtuber is saying, then don't watch them. I've many that I'll never click on again.

I am a measurement guy and as Sean Olive said in that quote, "measurements get you 86% of the way there". A listener better pay attention to what takes you the rest of the 14% of the way to bliss.
I like objectivity but this divide between "subjectivist" and "objectivist" is absurd.

Really it is unfortunately as if a big group of the objective folks completely forgot what objectivity means and are now dogmatists.

Definition of objectivity

: the quality or character of being objective : lack of favoritism toward one side or another : freedom from bias

 

[whazzup]

I am a measurement guy and as Sean Olive said in that quote, "measurements get you 86% of the way there". A listener better pay attention to what takes you the rest of the 14% of the way to bliss.

Actually now that you mention it, how was this 86% derived?

 

[Inner Space]

I like objectivity but this divide between "subjectivist" and "objectivist" is absurd.

Yes, it is. Assuming for a moment good faith on both sides, we have one group who prefer to describe things with numbers, graphs, measurements and reference to understood concepts; and another group who prefer to describe things with words, analogies, and reference to experiences shared or imagined within the group.

We may be right in guessing the numbers folk will be more correct more of the time, but we should acknowledge the words people can do a reasonable job, and even, from time to time, a surprisingly good one.

I think the divide is not side to side, but up and down. There are bad words people, who end up shilling and scamming. And there are bad numbers people too - at least in the sense of making their lives so painfully rational, they suck out any vestige of frivolity or joy. You should meet my brother.

For me personally, I hope I'm a good numbers person, but given a choice of dinner with either a bad numbers person or a good words person, I'd choose the words person in a heartbeat.

 

[vavan]

how was this 86% derived?

I believe it was discussed many times before on ASR. from the 3rd edition:

5.7.2 The Olive Experiments—Part Two
As impressive as this result is, there is a common factor that prevents the model from being
generalized—all 13 of the loudspeakers were bookshelf models, a natural basis for a comparative
product review in a magazine. All suffered from a lack of low bass, and many of them had common
features: enclosure size, driver complement and configuration, and so on. The population needed to
be expanded to include larger products, using different driver configurations and types. This was
done in a second test that involved 70 loudspeakers from many origins, covering large price and size
ranges. The penalty in this test was that the listening had been done in a fragmented manner—19
independent tests over an 18-month period—as happens in the normal course of business. All
loudspeakers were evaluated in comparison with some other models, but there was no overall
organization to ensure that the comparisons were balanced, each model against all other models. It
would be unreasonable to expect the same high precision in the subjective-objective correlations as
has just been seen.
Still, the result was impressive: predicted preference ratings correlated with those from listening
tests with a correlation of 0.86, with a very high statistical significance (p ≤ 0.0001). These are
remarkable numbers given the opportunities for variation in the listening tests, meaning that the
listeners themselves were highly stable “measuring instruments” and that the strategy of always
doing multiple (usually four products) comparisons is a good one

@Sean Olive blog also has lots of interesting articles on the topic

 

[whazzup]

I believe it was discussed many times before on ASR. from the 3rd edition:


@Sean Olive blog also has lots of interesting articles on the topic

Great, thanks!

 

[981CS]

We don't know, because you didn't tell us which speaker models you didnt like, nor did you tell us under what conditions you listened to them or what you compared them with. It's also doubtful you listened or compared blinded and level-matched, so who know what other factors came into play.

The specifics of what *I* like/don't like aren't at all relevant to this topic. The fact that you bring this into question is exactly my point and where things start to turn sideways.

From what you're implying, I'm the one at fault because there is absolutely no way I couldn't like a perfectly engineered and "scientifically superior" speaker. Well, at least in an environment that favors them (aka absolutely nothing like 99.5% of the living rooms of the folks in here).

Yeah...that's not entirely how it works in the real world with real people. And I think this was the point of that video that has gotten everyone's britches itchy in here.

 

[Beave]

The specifics of what *I* like/don't like aren't at all relevant to this topic. The fact that you bring this into question is exactly my point and where things start to turn sideways.

They absolutely ARE relevant. The research never claimed that Revel speakers will win out in sighted, non-level matched listening comparisons, or that everybody will like them in sighted listening in every environment.

From what you're implying, I'm the one at fault because there is absolutely no way I couldn't like a perfectly engineered and "scientifically superior" speaker. Well, at least in an environment that favors them (aka absolutely nothing like 99.5% of the living rooms of the folks in here).

No, what I'm implying is that other things besides actual sound quality matter when doing scientifically UNcontrolled listening.

 

[StefaanE]

guess the problem is that many audiophiles don't see their hardware as devices but as music instruments and thus as part of the "art" (which of course is wrong in the original concept of High Fidelity) and "don't want to chose something so emotional from a bunch of numbers".

Like buying a car, which is often more about emotions than specifications. Great insight, guys.

 

[981CS]

They absolutely ARE relevant. The research never claimed that Revel speakers will win out in sighted, non-level matched listening comparisons, or that everybody will like them in sighted listening in every environment.

Again with having to justify something based on one set of "ideal" circumstances.

Have you listened to Harman speakers? A lot of them sound different from one another. Why is that again?

No, what I'm implying is that other things besides actual sound quality matter when doing scientifically UNcontrolled listening.

Do these "other things" include humans and their predisposition for liking all sorts of different stuff, no matter how "illogical" it may be to someone else?

 

[Aerith Gainsborough]

According to this guy, speaker measurements are not good indicators of sound quality because of other variables not accounted for in measurements that influence what we like including our hearing, our taste, our listening rooms, our program material.

Unpopular opinion inc:

I kind of agree with the man on the front, that OBSESSING about specs as an end-user is the wrong way to go about it.
It is really unrealistic to look at a FR in isolation and expect the speaker to sound like that in your room.

The room will basically nuke the carefully crafted anechonic FR of any given speaker and if you employ room correction DSP, you will most likely move even further away from the manufacturers target curve.

Naturally, designers should measure. Absolutely. They need to find all the little flaws and correct them as best they can. No argument there.

As an end user, measurements are somewhat helpful in the selection process, especially the ones that go beyond the simple frequency response.

In general, I do prefer a flat response, because then, DSP only has to work against the room and not work against an arbitrary house curve in addition to that.

Edit: clarification that I distinguish between the designers and the end-users.

 

[napilopez]

Again with having to justify something based on one set of "ideal" circumstances.

Have you listened to Harman speakers? A lot of them sound different from one another. Why is that again?



Do these "other things" include humans and their predisposition for liking all sorts of different stuff, no matter how "illogical" it may be to someone else?

It is crucial to note that several speakers having good measurements with a flattish frequency response does not at all mean these speakers will sound the same same. As has been discussed many times in this forum (I don't blame you for not seeing all the discussions!), speakers can have a similar looking Spinorama and still sound very different.

Long story short is that it's generally a good idea for most speakers to aim for a flattish frequency response free from resonances, but there's some wiggle room within 'flattish.' Perhaps more importantly, there are many ways directivity can modify the sound of a speaker and its perceived spatial presentation.

Having good measurements is important, but sometimes people take that to mean a homogenization of speakers, which needn't be true. There are many ways to achieve 'good.'

I currently have the KEF LS50 Meta, Focal Chora 806, and JBL-HDI-1600 in my apartment. All three have good, flattish measurements, but all three sound quite different due to their varying directivity. But again, they all sound good.

I don't think most people here want every speaker to sound exactly the same; we just want some basic standards and more honesty in how products are promoted with regards to their performance per price. Measurements provide much needed transparency.

Personally, I don't care if a manufacturer makes a speaker with a wacky frequency response, so long as it is clear on its intent and doesn't promote the speaker as the pinnacle of neutrality. But that rarely happens.

Unpopular opinion inc:

I kind of agree with the man on the front, that OBSESSING about specs as an end-user is the wrong way to go about it.
It is really unrealistic to look at a FR in isolation and expect the speaker to sound like that in your room.

The room will basically nuke the carefully crafted anechonic FR of any given speaker and if you employ room correction DSP, you will most likely move even further away from the manufacturers target curve.

Naturally, designers should measure. Absolutely. They need to find all the little flaws and correct them as best they can. No argument there.

As an end user, measurements are somewhat helpful in the selection process, especially the ones that go beyond the simple frequency response.

In general, I do prefer a flat response, because then, DSP only has to work against the room and not work against an arbitrary house curve in addition to that.

Edit: clarification that I distinguish between the designers and the end-users.

It really depends on what you mean by "frequency response." Just an on-axis graph? Sure that's not very useful. But a full on spin most definitely tells you what to expect in your room - certainly if you know how to do some basic measurements of your own and know how a speakers response (mostly the low end) will be modified by it.

More importantly, there's evidence to support the idea that while a speaker may sound somewhat different in different rooms, the better speakers are still preferred in direct comparisons among speakers within a given room.

As such, we musn't give the room too much credit either. It has a profound effect on the low frequency response, but above 400-500Hz or so, speakers tend to behave as expected. This is both in accordance with research and my own experience, having measured and listened to dozens of speakers in a several locations in a couple of apartments. Yes, some speakers may be slightly better in some rooms than others, but for the most part, the better speakers are just better.

Per Toole, people "have a remarkable ability to 'listen through' rooms." See Olive etc al 1995, or Toole's book Chapter 7.6.2:

"It appears, therefore, that we can acclimatize to our listening environment to such an extent that we are able to listen through it to appreciate qualities intrinsic to the sound sources themselves. It is as if we can separate the sound of a spectrum that is changing (the sounds from the different loudspeakers) from that which is fixed (the colorations added by the room itself for the specific listener and loudspeaker locations within it). This appears to be related to the spectral compensation effect noted by Watkins (1991, 1999, 2005) and Watkins and Makin (1996)."

It's not so dissimilar from the fact that you can clearly recognize a familiar voice or instrument in a variety of different rooms. Our brains can to a significant degree tell apart the room and the speakers. Which is why it's so crucial to get good anechoic or quasi-anechoic data.

 

[Feelas]

The specifics of what *I* like/don't like aren't at all relevant to this topic. The fact that you bring this into question is exactly my point and where things start to turn sideways.



From what you're implying, I'm the one at fault because there is absolutely no way I couldn't like a perfectly engineered and "scientifically superior" speaker. Well, at least in an environment that favors them (aka absolutely nothing like 99.5% of the living rooms of the folks in here).

Yeah...that's not entirely how it works in the real world with real people. And I think this was the point of that video that has gotten everyone's britches itchy in here.

Measuring is merely knowing what you like after you measure, so why are we starting a shouting match here? Actually the guy in vid has shown nothing being in favour of his argument that measurements don't work in real living spaces - I hope you're going to follow up with some actual cases of living rooms not interacting well with good measuring speakers, right? Please stop turning the debate into a fight for a free world, where noone will be held responsible for straying from the standard, since it's not the case here at all. We're arguing against the fascism of forced subjectivity, where one doesn't even know what they might be losing from not trying to use the technical possibilites of the time.

Still, one needs not to abide to double testing protocols, yet claiming a technical superiority of something needs quantifiable metrics so one can be held to their opinion on some objective matter, not merely a changing state of mind. FYI: the point of preferring the "better" loudspeakers is the one which claims, that a flatter response with less adjacent frequency masking due to frequencies overpowering eachother, is statistically preferred in blind testing scenarios and the point holds even when you remove either the best, or worst speakers; the preference order is kept in Toole's research into this. Why are we arguing against something logical and steming from psychoacoustics?

It's just ridiculous... I mean, I know it's fashionable to be seen as being "against the system", but gosh, we've went so far as to write these posts by means of of properly guided science.

 

[Aerith Gainsborough]

It really depends on what you mean by "frequency response." Just an on-axis graph? Sure that's not very useful.

In terms of sound, yes, some speakers may be somewhat better in some rooms than others, but for the most part, the better speakers are just better.

Yup that was mostly what I was referring to, because in most reviews there is little information beyond "here's a FR graph". The Klippel measurements of our Senpai are a completely different ballgame though.

I am curious: define "better". Speaking in the context of the YT video, which limited measurements to a single FR graph, barring any absurd curves, I doubt there is such a thing as "better frequency response".

 

[Feelas]

Yup that was mostly what I was referring to, because in most reviews there is little information beyond "here's a FR graph". The Klippel measurements of our Senpai are a completely different ballgame though.

I am curious: define "better". Speaking in the context of the YT video, which limited measurements to a single FR graph, barring any absurd curves, I doubt there is such a thing as "better frequency response".

If two curves are measured using the same smoothing on graphs) then I'd be close to calling the smoother one the better, mainly because you can mold the FR with EQ, but breakups & resonances can't be treated this way.

 

[ctrl]

I like objectivity but this divide between "subjectivist" and "objectivist" is absurd.

Really it is unfortunately as if a big group of the objective folks completely forgot what objectivity means and are now dogmatists.

: the quality or character of being objective : lack of favoritism toward one side or another : freedom from bias

But to be objective is precisely not to juxtapose all divergent points of view fifty-fifty.

If one side proves certain properties with scientific methods and the other side makes counter claims, without any scientific substantiation, then it is objective when BS is called BS.
Only in this way does scientific work make sense. One can and should critically question scientific work, but the criticism of certain points of view does not make one's own point of view a "true statement" or just as meaningful as a scientific paper.

Are there speakers that show good measurement results but don't sound very good? For sure. Are there sometimes big differences in the evaluation of a loudspeaker in the listening test? Yes. Should we critically question individual aspects of the evaluation of loudspeaker measurements (such as the evaluation of lateral reflections or the effect of vertical reflections)? Absolutely.

But the correlation of loudspeaker measurements and listening test ratings is clearly there.
But if you then go and make the exception (individual taste, hearing damage, my specific room,...) the rule, then this should be objectively called BS.

The big problem in audio is that to refute a completely fact-free claim with sources and facts, the effort to do so is greater by a factor of 10 or 100.
In the time the original claim was already slightly modified or simply another absurd reason invented why it is " true " and spread further.
Therefore, it often takes decades for perceptions to slowly change.

 

[TimVG]

Are there speakers that show good measurement results but don't sound very good? For sure.

In my experience, if it measures good, but sounds bad.. You probably need more, or better, measurements

 

[Vini darko]

wow... so, @joentell pinned my comment which you'll see below. Not because of what I said about AR blocking me from his channel but because I thought his video (Joe's) was rational and I appreciated that.

Then AR's girlfriend jumps in and ... well... I'll let you guys see...


View attachment 111813

View attachment 111814




At least we know exactly what we are dealing with now. I just didn't think they would stoop to the level of actually lying. And I'm damn glad I took screenshots of my comments (which were posted here on Monday).

And the kids in playground chant "fight fight fight". Poor joe hes now gotta join in.

 

[Aerith Gainsborough]

wow... so, @joentell pinned my comment which you'll see below. Not because of what I said about AR blocking me from his channel but because I thought his video (Joe's) was rational and I appreciated that.

Then AR's girlfriend jumps in and ... well... I'll let you guys see...


View attachment 111813

View attachment 111814




At least we know exactly what we are dealing with now. I just didn't think they would stoop to the level of actually lying. And I'm damn glad I took screenshots of my comments (which were posted here on Monday).

Wow, this is depressing.
Your posts were polite, factual and well articulated.

I'm sorry you got this kind of reaction.

 

[Gregm]

My current speakers I bought strictly by how they measured, (...) Might not be the best way but nowadays you do what you can and hope for the best. I figured less risk knowing the measurements and how they should sound.

Exactly -- with some measurement, one can at least speculate at the sonic results at home...
I for one enjoy auditioning. When it comes to speakers, however, I find that even bog-standard (quasi)anechoic FR measurements can be a very easy and useful yardstick to guess the actual listening experience. And surely we have understood a thing or two about our rooms, as well

Not to wax philosophical but, rather, practical, esp when it comes to speakers -- why not simply opt for a combination of both listening and measuring rather than pitching one against the other. Indeed in the example above

I've heard a few Harman speakers that I absolutely do not like and wouldn't own if given to me for free.

why not check whatever measurements are available for these speakers (harman usually has a bunch) and try to identify elements in them that correlate to what you like (or do not like).
For example, I don't go crazy over speakers displaying a horizontally flat unechoic FR (usually 1/3 octave) -- personal preference... what can I say