Master Thread: Are measurements Everything or Nothing?

[thewas]

The one that leaded us to "harman curve" which "everyone enjoys"?
The pic is interesting tho. Even in non-blind testing the diference is within 1.5 of 10 and shortens to 0.5 - but even 1.5 means very small difference to me.
I hope that testing was not as laughable as iconic "headhones testing" where Stax 909 were EQ-emulating other headphones.
Hope it also was not 15s solo violin fragments (music, regardless of hardware).

Must be a very poor speaker, oh, I see, it's not European anyway two normal speakers kept their lead

If you had really had read and understood Toole he is the one that says that such a curve doesn't make sense for loudspeakers, how loudspeaker listening tests are done at Harman, that 15s violin segments are not what Toole recommends and are used at such but it seems you are not interested in real understanding or discussion but rather spew out your irony like many audiophiles do here as their favourite devices don't fair well under controlled conditions.

 

[MAB]

The one that leaded us to "harman curve" which "everyone enjoys"?

You seem to dislike the curve, or is it research?

The pic is interesting tho. Even in non-blind testing the diference is within 1.5 of 10 and shortens to 0.5 - but even 1.5 means very small difference to me.
I hope that testing was not as laughable as iconic "headhones testing" where Stax 909 were EQ-emulating other headphones.
Hope it also was not 15s solo violin fragments (music, regardless of hardware).

Yet you clearly have no idea how the research was done, or the source of the result you quote.

At least study the subject you are deriding. 'Mere glances' don't help.

 

[uwotm8]

it seems you are not interested in real understanding or discussion but rather spew out your irony like many audiophiles do here as their favourite devices

I just don't want to be the kind of "anti-audiophile" who is so "anti" that he becomes a graph-freak who ignores everything else. To me this is only slightly better (if better at all) than those who deny the importance of measurements, tune speakers by ear, and prefer to burn in cables and choose the wood material for the stands for these cables, claiming that this is no less important than the choice of speakers themselves.

You seem to dislike the curve

After playing a lot with it on different headphones, I ended up rather enjoying good yet not perfect ones with no EQ involved at all. I do hear some flaws but they rather seem natural to my ears. EQing to Harman curve seems not. I however do have a pair of chinese IEMs which are very close w/o any EQ and yes the overall tune is okay but that doen not make me like their sound much. Again, no complains by the tuning, but nothing exciting as well. Just... bland sound. For my current daily driver 6xx I found "Optimum Hi-Fi curve" to be somewhat balanced, opposite to "mass-market-bassy" sound which Harman tends to produce (and yes I did try adjusting bass level). Yet still prefer it pure with all flaws. There's some concept in it as is.

Regarding the book:
- I haven't read it in its entirety, it's not in the public domain
- I'm familiar with its main theses. Regarding the evaluation of the speakers, the information is public due to the fact that it was discussed almost everywhere and more than once and I refreshed my memory (at the same time remembering why I can't help but perceive it without a sober share of skepticism), this is the same testing of one speaker in mono behind a curtain with an indefinite set of music tracks "of all genres"
- The name and idea itself is to identify the "PREFERRED character of sound", not the "maximum accurate transmission of the phonogram"
- This comparison lacks a reference as such, it is focused on identifying preferences among what you have in fron of you (the least flawed among flawed)
- The result, however, well, not much surprises here: linear speakers without big surprises in dispersion pattern at least within +-15 degrees (the listening window, known and obtained long before the spinorama-on-Klippel) are preferred

That said, I have no complains at all looking at that research from that point: it was partially made for an audio company and partially used as a part of marketing strategy. "Hey, we've found that sound what most of the people prefer". Well, BBC tune with BBC dip (TM) is a earlier approach and a lot of people like it as well.

So, a mass-preferrable tune is what have been found.
Let's accept this as a fact. Then there are two ways:
1) start fussing around it and pointing at everything that is different and calling it bad, sometimes ecstatically shaking if that's an expensive device
2) take the results into account and in your assessment of the speakers' performance say that from such a point they are imperfect, but in fact this has a strong effect, some effect or no effect at all on subjective listening.
The latter is what I appreciate.

P.S. Also wondering how mono listening of MBL or even MartinLogan static panels can give listener an idea of how it works in real room as a pair.

 

[Jim Taylor]

I just don't want to be the kind of "anti-audiophile" who is so "anti" that he becomes a graph-freak who ignores everything else.

You have control over what you are and what you become. You (and people in general) are not puppets.

I think you will agree that Amir is one of the firm proponents of measurements. Yet he values measurements for what they mean, not what they are. This takes understanding. His comments show a very good correlation between what he measures and what he hears (with a few notable exceptions).

So the understanding is not that some people are "graph-freaks who ignore everything else", but that they value measurements first, and use those as a guideline to further investigation.
After all, if a person wears size 11 shoes, why should they investigate shoes marked size 8?

 

[MAB]

P.S. Also wondering how mono listening of MBL or even MartinLogan static panels can give listener an idea of how it works in real room as a pair.

This claim is made often, not just regarding omni or planar speakers, both which produce sound that obeys the same physical laws as other speakers.
The principle is called superposition. It applies to MBL and Martin Logan in rooms too, and outdoors as well.

A member has actually worked out the Near Field Scanner solutions explicitly, which correctly explains in the first sentence:

At a given frequency, the acoustic pressure field generated by a loudspeaker can be expressed as a summation of a set of basis functions multiplied by their weighting factors. For our problem at hand, a very suitable choice for these basis functions is the spherical wave functions in the spherical coordinate system.

The sound field of two speakers in a room can be described as the sum of the fields of the individual elements. It's a common theme in physics, and exceptions are often noteworthy. Speakers obey this principle.

Also, regarding the rest of what you wrote; your preferences are your own.

I did want to clear up your misunderstanding on a simple principle of sound though.

 

[uwotm8]

The sound field of two speakers in a room can be described as the sum of the fields of the individual elements

I did want to clear up your misunderstanding on a simple principle of sound though

So my "misunderstanding" here is that you completely ignore the whole idea of stereo reproduction? It's not about the difference between playing a mono recording on one or two speakers, it's about a real, three-dimensional stereo picture. About how such speakers interact with the room and how useless it is to judge an omnidirectional system like the MBL if it's just one speaker playing mono - it's tonality (what can be judged) is just a part of whole story.

 

[uwotm8]

His comments show a very good correlation between what he measures and what he hears (with a few notable exceptions)

There's enough exceptions but what's more important is that flat and linear speakers are rather similar while not so flat speakers are all different and may be acceptable and pleasant (or not).
Very linear Dynaudio LYD 5 with its PS of 5.7 wasn't liked much.
Tannoy XT6 with PS of 1.7 (utter garbage, right? $1.2k btw) wasn't disliked even remotely as
PMC Result6 with PS of 2.7
And then $10k Wilson TuneTot (2.6) was described as "suprisingly good despite bad graphs".
Adam T5Vs to sum this "cherrypicking" up: 4.6 or so PS, described as a very pleasing speaker, "a bit bright w/o EQ"; that "bit" is a big BIT if you're on axis. If same PS was caused by another deviation, say, classic BBC dip, it would be a way safer choice "to suggest a random person not familiar with EQ".

All in all what mentioned above is already enough for me not to judge speaker by PS except some ruler flat ones with a top score. Not el cheapos with 2 mm plastic front baffle rattling all the way tho

You (and people in general) are not puppets

A very philosophic question, what's happening around the world from the COVID hit makes me doubt seriously on that statement

 

[Jim Taylor]

There's enough exceptions but what's more important is that flat and linear speakers are rather similar while not so flat speakers are all different and may be acceptable and pleasant (or not).
Very linear Dynaudio LYD 5 with its PS of 5.7 wasn't liked much.
Tannoy XT6 with PS of 1.7 (utter garbage, right? $1.2k btw) wasn't disliked even remotely as
PMC Result6 with PS of 2.7
And then $10k Wilson TuneTot (2.6) was described as "suprisingly good despite bad graphs".
Adam T5Vs to sum this "cherrypicking" up: 4.6 or so PS, described as a very pleasing speaker, "a bit bright w/o EQ"; that "bit" is a big BIT if you're on axis. If same PS was caused by another deviation, say, classic BBC dip, it would be a way safer choice "to suggest a random person not familiar with EQ".

All in all what mentioned above is already enough for me not to judge speaker by PS except some ruler flat ones with a top score. Not el cheapos with 2 mm plastic front baffle rattling all the way tho

A very philosophic question, what's happening around the world from the COVID hit makes me doubt seriously on that statement

'For every complex problem there is an answer that is clear, simple, and wrong.'
H. L. Mencken

It's unfortunate that many people nowadays try to treat rule-of-thumb generalizations as if they were hard-and-fast specifics.. SINAD is one of those, and Preference Score is another. Speakers present what you might call a "soup" of differing characteristics which all interact to varying degrees. If they are to work passably well, electronics present a much more constrained set of characteristics.
Speakers can exhibit a good on-axis response, but exhibit what I call horrible off-axis response. Because speakers radiate in different ways (and to different degrees) in different rooms, PS is the most general of generalities. This is important to me because I sit relatively close to the speakers. I take it that most people do not.
No matter the listening distance, we are immersed in a reverberant field, coming at us from many directions. Markedly different speaker designs, with markedly different off-axis peaks and cancellations, can sound quite different in normal (reverberant) residential rooms. People who listen to a speaker on-axis may experience negative characteristics to a lesser degree than people who lsten off-axis. For some, the various annoyances are slight ... for others, more so.

That's why I said in my last post, regarding Amir's take on measurements ...

... he values measurements for what they mean, not what they are. This takes understanding.

The quote from H.L. Mencken indicates the same. Measurements are not just numbers to be lumped together and dumbed-down. They are data to be understood, taken in context of their varying degrees of interaction with each other, and then appreciated for the lessons they can teach us.

Generalizations can't do that.

 

[uwotm8]

Measurements are not just numbers to be lumped together and dumbed-down. They are data to be understood, taken in context of their varying degrees of interaction with each other

From my understanding of measurements LYD 5 is a very good speaker except bass limited; despite lack of waveguide nothing crimial happens in terms of directivity.
OTOH PS calculation model is complex and very far from just applying a ruler to on-axis FR graph.
The score is high, subjectively however speaker described not just as disappointing, but also "not sounding right" somewhat. Listening was performed in nearfield so there was less room involved that it might be.
So, how should a reader rely on that data, and on what exactly in case of such contradiction?
IMO the adequate point is close to Audioholics reviewer - rely on both and don't exagerrate much.

For every complex problem there is an answer that is clear, simple, and wrong

What makes you think that I'm searching the simple answer? It's exactly opposite. Bold statements and dividing everything into black and white is actually what makes me generate tons of irony.

SINAD is one of those

As far as Path of Harman says us - no listener can define anything below -40 dB, I just don't see any reason to bother about it. Even the worst integrated audio card exceeds that level easily. Well, OK, some irony intended again. But even -90 and -100 dB is now considered bad. Of course the lower the better, but isn't it on that level, after -100 dB, where we should seriously take into account that 0.1-0.2 dB FR difference a speaker cables can cause (Alpha Audio has nice video about it)?

 

[Jim Taylor]

Edit: I've just deleted two posts. It seems I have allowed myself to be manipulated, in my gullibility, by a clever little troll. I apologize to all our members for wasting their time.

 

[MAB]

So my "misunderstanding" here is that you completely ignore the whole idea of stereo reproduction?

No, it's that you sum the sound fields, not ignore.

It's not about the difference between playing a mono recording on one or two speakers, it's about a real, three-dimensional stereo picture.

Mono recording on two speakers (I'm trying to take you seriously), yes those also obey superposition. As does the stereo implementation.

About how such speakers interact with the room and how useless it is to judge an omnidirectional system like the MBL if it's just one speaker playing mono - it's tonality (what can be judged) is just a part of whole story.

You seem to believe that there is something unique about physical properties of sound from MBL vs. sound from another source. There is not.

Do you have a point, or are you just arguing for argument's sake? If you do have a point, please restate it.

 

[uwotm8]

there is something unique about physical properties of sound from MBL vs. sound from another source

It might be German Physics or anything else "true omnipolar", emulating that "ideal sphreical horse in a vaccum source" - one of the (unobtainable) theoretical concepts of an ideal speaker. Not sure if B&O or Bose approaches can be really compared.
So, the point is: one omni speaker will give you some strange effect of sound cloud, but in pair it will actually make an immersive and holographic soundstage. It might be not as precise tonally as a studio monitor, but listener can be blown away by that "artificial" 4D stage and prefer it instead of the rightest ever tonality. Pretty much the same as some people prefer a speaker that goes LOUD and clean, ignoring some tonal issues as an acceptable tradeoff; the very best nearfield monitor will just never make them happy.
Second point: such speakers require special placement as far as room gets way more involved than in typical directional case of "box with cones". But the latter require different placement too sometimes as well. As far as I understand that test conditions, it was about evaluating all 4 speakers placed in the same point, not special/optimal for each exact model. Could affect results a bit as well.

 

[MAB]

It might be German Physics or anything else "true omnipolar", emulating that "ideal sphreical horse in a vaccum source" - one of the (unobtainable) theoretical concepts of an ideal speaker. Not sure if B&O or Bose approaches can be really compared.
So, the point is: one omni speaker will give you some strange effect of sound cloud, but in pair it will actually make an immersive and holographic soundstage. It might be not as precise tonally as a studio monitor, but listener can be blown away by that "artificial" 4D stage and prefer it instead of the rightest ever tonality. Pretty much the same as some people prefer a speaker that goes LOUD and clean, ignoring some tonal issues as an acceptable tradeoff; the very best nearfield monitor will just never make them happy.
Second point: such speakers require special placement as far as room gets way more involved than in typical directional case of "box with cones". But the latter require different placement too sometimes as well. As far as I understand that test conditions, it was about evaluating all 4 speakers placed in the same point, not special/optimal for each exact model. Could affect results a bit as well.

Please reply to my entire post or comment, not just a sentence fragment. This is sloppy and context got lost. What I said is below:

You seem to believe that there is something unique about physical properties of sound from MBL vs. sound from another source. There is not.

So, regarding your examples of speakers that have spherical radiation... The whole point of nearfield scanners, and 3D sound field measurements in general, is to measure the exact same properties of a speaker that you are describing, or any speaker. Since the method typically solves spherical wave equations as I linked earlier, it occurs to me the math gets even simpler if you have a true spherical radiation source.

 

[uwotm8]

So, regarding your examples of speakers that have spherical radiation... The whole point of nearfield scanners, and 3D sound field measurements in general, is to measure the exact same properties of a speaker that you are describing, or any speaker. Since the method typically solves spherical wave equations as I linked earlier, it occurs to me the math gets even simpler if you have a true spherical radiation source

The point is that scanning such speaker and listening it in mono can give you idea of its basic like tonality, distortion etc but not of that uncommonly (hyper)realistic 3D image a pair of it is capable to produce. By testing it in mono you're just missing the whole sense of such a design. One speaker will not "disappear" and so on. Kinda interesting tho how such (almost) uniform directivity across all range will affect PIR in that score-calculating model.
This is not about MBL as a brand or "uniqueness" as a target by itself. I'm just pointing some questionable - to me - aspects of that "mechanized" speaker evaluation, and mostly because that subjective "exceptions" exist and with a lot of common speakers - when what's measured awful is not that bad and the opposite.

I see a very good coreallation in Amir subjective reviews tho - corellation with LF performance. If it's uneven by some means, even relatively high-PS speaker can oftenly be described from not exciting much (KEF LS50 or mentioned LYD 5) to totally terrible (there were some cinema satellites tested as "normal" speakers, all bad PLUS no bass so...).

 

[Jim Taylor]

Kinda interesting tho how such (almost) uniform directivity across all range will affect PIR in that score-calculating model.

Our member @flipflop posted this on 21 November, 2021:

"I guess it's worth pointing out that the "Preference Score" is the predicted preference score and not an actual preference score gathered from listening tests.
Saying that the "value [ranges] from 1-10" is not entirely correct either. Sure, the speakers recommended in this guide all fall into that interval, but there's no theoretical minimum as we've seen with for example Bose 141 and Magnepan LRS receiving scores below 0. A max score of 10 also assumes a LFX of 14.5 Hz."

This might lend a little clarity to the preference scores.

 

[uwotm8]

Our member @flipflop posted this on 21 November, 2021:

"I guess it's worth pointing out that the "Preference Score" is the predicted preference score and not an actual preference score gathered from listening tests.
Saying that the "value [ranges] from 1-10" is not entirely correct either. Sure, the speakers recommended in this guide all fall into that interval, but there's no theoretical minimum as we've seen with for example Bose 141 and Magnepan LRS receiving scores below 0. A max score of 10 also assumes a LFX of 14.5 Hz."

This might lend a little clarity to the preference scores.

I totally understand that (that's why I wrote that first post in OPAL thread where I said that I appreciate Audioholics approach of testing).
Amir said, and not even once, that critical listening expertise does matter at least as much as measurements.
You can actually check it and see that my "trolling" is nothing but pointing caveats in some statements.
Marking speakers as bad due to FR deviations within classic +-3 dB just by graphs, not even trying to analize what's actually bumped/recessed, is a classic case of that. Such speakers are "in grey zone", they might be pleasant or not, and may and will fit or not some individual tastes.
Also, OPAL is sort of exotic project and evaluating is as a typical loudspeaker is arguable to me. Designer does not say that it's any kind of neutral of reference, that was not the idea. I think that should be kept in mind as well. And for me it's actually a no-go already, just by developer description
Speaking subjectively, I could easily get used to BBC dip etc I think, as far as classic Dynaudios have some type of it, but I radically hate speakers with recessed/hollow lower midrange, severe saw-FR/coloration and overall "bright" tilt. So, even it's a LOL OMG BAT SHAPE!!1 but upper bass and lower midrange has its place it's far from the worst for me personally.
To sum this up, I don't think the reviewer is wrong if he politely describes something like that as a specific product not for everyone. Same as Stereophile calls B&W FR tailored. I hate their tune, but a lot of people seem to like it.

 

[MAB]

I totally understand that (that's why I wrote that first post in OPAL thread where I said that I appreciate Audioholics approach of testing).
Amir said, and not even once, that critical listening expertise does matter at least as much as measurements.
You can actually check it and see that my "trolling" is nothing but pointing caveats in some statements.
Marking speakers as bad due to FR deviations within classic +-3 dB just by graphs, not even trying to analize what's actually bumped/recessed, is a classic case of that. Such speakers are "in grey zone", they might be pleasant or not, and may and will fit or not some individual tastes.
Also, OPAL is sort of exotic project and evaluating is as a typical loudspeaker is arguable to me. Designer does not say that it's any kind of neutral of reference, that was not the idea. I think that should be kept in mind as well. And for me it's actually a no-go already, just by developer description
Speaking subjectively, I could easily get used to BBC dip etc I think, as far as classic Dynaudios have some type of it, but I radically hate speakers with recessed/hollow lower midrange, severe saw-FR/coloration and overall "bright" tilt. So, even it's a LOL OMG BAT SHAPE!!1 but upper bass and lower midrange has its place it's far from the worst for me personally.
To sum this up, I don't think the reviewer is wrong if he politely describes something like that as a specific product not for everyone. Same as Stereophile calls B&W FR tailored. I hate their tune, but a lot of people seem to like it.

You blur measurement with preference. The point here is to not do that, but here you are, blurring the conversation additionally with your own preference.

Your examples of MBL etc. are measurable on a near-field scanner. If you measured and dug into the data, it will be incredibly obvious you measured an MBL or similar speaker. If another speaker measured the same as an MBL model, it will have similar distribution of listener's perceived performance, and preference. The measurement data allow you to calculate how that speaker will interact with a room, but not if you will like the speaker. Issues with the speaker, like an untamed resonance will also be evident, your perception of the resonance is another matter. With the data, you can even identify issues that make a speaker more or less performant to your preference in a specific room and setup.

You confuse preference and measurements in this long back and forth you have manifested. It's good your commentary got moved to this catch-all thread. Your tendency to hijack other threads injecting your personal preference along the way with poorly reasoned caveats is likely what makes you come across as a troll.

 

[krabapple]

The one that leaded us to "harman curve" which "everyone enjoys"?

No, the actual book.

What it 'leaded' you to, I can't say, as I'm not convinced you ever read it, based on nonsensical takes you offer in this post.


tl;dr you don't know what you're snarking about. You're shaping up to be yet another waste of time on ASR.

 

[uwotm8]

You blur measurement with preference. The point here is to not do that, but here you are, blurring the conversation additionally with your own preference.

Your examples of MBL etc. are measurable on a near-field scanner. If you measured and dug into the data, it will be incredibly obvious you measured an MBL or similar speaker. If another speaker measured the same as an MBL model, it will have similar distribution of listener's perceived performance, and preference. The measurement data allow you to calculate how that speaker will interact with a room, but not if you will like the speaker. Issues with the speaker, like an untamed resonance will also be evident, your perception of the resonance is another matter. With the data, you can even identify issues that make a speaker more or less performant to your preference in a specific room and setup.

You confuse preference and measurements in this long back and forth you have manifested. It's good your commentary got moved to this catch-all thread. Your tendency to hijack other threads injecting your personal preference along the way with poorly reasoned caveats is likely what makes you come across as a troll.

I wonder why you're bluring some facts (which will not change whether I posted it or not) with personal preferences which I always mark as personal preferences.
And why you blur my opinion on test approach with what you're trying to put here as "my" opinion about whole idea of measurements.
Fact: preference score calculation is measurement based.
Fact: the very idea of it to predict subjective opinion (with arguable accuracy).
Then, not sure if fact or my interpretation: one of the goals (if not main) of Harman research was finding that "average listener" preference target curve; also finding corellations between measurements and subjective experience, and finding how much do measurable characteristics affect that/what matters much (unsurprisingly FR).

Now let me quote what I'm actually answering:

The measurement data allow you to calculate how that speaker will interact with a room, but not if you will like the speaker

Wasn't that one of the goals? To predict that with a good chances. To design speakers that will be liked by the most of people and selling well as a result.

If you measured and dug into the data, it will be incredibly obvious you measured an MBL or similar speaker. If another speaker measured the same as an MBL model, it will have similar distribution of listener's perceived performance, and preference

Somehow you're missing again and again that in case of omnipolars I'm appealing to the fact that listening to one will not allow you to hear their real spatial capabilities. I did not state that omnipolars can't be measured (but AFAIR John Atkinson stated that measuring even simplier things - MartinLogan ELS models - with traditional approach is incorrect). Still, all mono vs stereo talk in case of omnipolars was about listening only. Where you simply can't get that (hyper)realism using only one speaker, so you're missing their main feature if doing so. Can be worse for sure, just imagine if one judges such non-traditional design by measurements only, with no listening at all.

you don't know what you're snarking about. You're shaping up to be yet another waste of time on ASR

The thing what makes me smile more and more reading such is that what you call "snarking" mostly does not contradict - in essence - with what has been stated many times by a lot of experts who do not strive for polar assessments and prefer restraint.
Critical importance for subjective listening is among this.
If you count "hijacking" something like "I'd avoid buying this blindly" or "better double-check if you will like that switching from 1.5m height 3-way towers to 5" monitors with a sub" - well, as you wish.

P.S. Just curious if posting something generic "party approved" like "OMG that $100 D-class just blown out the water all hi-end amps" (yet it's barely on par with 20 y.o NAD 320 you could buy used fot the same money, get a lot of inputs etc) or, for example, "wow just realize how much money you saved not buying this" (mostly about the devices even a true audiophool will barely buy) is considered as creating good, needed content of a good quality, informative and useful.

 

[krabapple]

The thing what makes me smile more and more reading such is that what you call "snarking" mostly does not contradict - in essence - with what has been stated many times by a lot of experts who do not strive for polar assessments and prefer restraint.
Critical importance for subjective listening is among this.

All listening is subjective. Truly critical importance would be blind listening.

Absent that, temper your claims. Unless of course you have measurements to back them up.

You seem to be regurgitating tropes that have been splashed on ASR many times before. Do you flatter youreself that you are doing otherwise?