"Bias" of some members towards headphone measurements?

[Old_School_Brad]

Point #10, the one that Toole highlighted, is what you are disagreeing with. And while it may seem possible to EQ every speaker to better performance, that's just not the case, as explained in point 5. You really can't EQ a resonance. Perhaps you can EQ one aspect of it's radiation pattern, like an on-axis peak at a given frequency, but you will then introduce other on- and off-axis artifacts. Even if you can get one slice of the field EQ'ed with on-axis performance that is OK in one position in the room, you are left with other a host of other problems, sometimes worse than the on-axis performance you EQ'ed.

Resonances are hard to deal with in any system, not just speakers. They uniformly defy equalization, require good design up front. I tested an old Yamaha speaker with a rigidly mounted Styrofoam woofer that operates as a resonant membrane, and is a good example of a speaker that cannot have it's issues EQ'ed.

I recommend thinking deeply about point # 5 in Toole's post, and then about EQ.

I’m not saying EQ can work miracles and fix every speaker. We’re not living in a world where a Jamo 365 magically can be made to sound like a Revel Salon. That said, EQ is still a super useful tool for improving sound quality. Sure, it can’t solve physical problems like resonances, cabinet vibrations, or bad transient response, but it’s great for fixing frequency imbalances. It can smooth out peaks and dips, make the sound more balanced, and even help with room issues.

No, EQ won’t fix a poorly designed speaker or turn a cheap one into a high-end model, but it can still make a big difference, even with budget setups. Most speakers whether they’re good or bad will sound better with some EQ, especially when it’s adjusted to fit your room, preferences, and listening position. This ties back to my earlier point about active speakers that come with different tunings or "house curves" built in.

 

[SuicideSquid]

I'm specifically referring to the practice of looking at a single 2-D FR measurement of a transducer (headphones, in particular) and attempting to make strong predictions about SQ. I'm sure you've undoubtedly seen headphones "measured" as a single 2-d freq vs amplitude measurement with a superimposed Harman curve meant for "comparison."

This seems like a strawman. I think most people around here agree that FR is the most important measurement, I don't think you'll find a single person who will take the position that it's the only measurement that matters.

 

[preload]

With headphones 2D is actually pretty predictive (maybe not 100.0%)

Yes, particularly for IE headphones, but ONLY if you are measuring with the same/identical rig that Harman uses (when trying to assess compliance to target) AND you are applying a regression formula to those measurements.

The high frequencies are pretty hard to measure accurately but other than that... FR is definitely still the most important measure I'd use to consider headphone sound quality if I could only have one, would you not agree?

Oh I agree 100% with this, but I also would not expect the measurements to predict SQ beyond a certain point, for reasons previously stated.

Also, think about it, how many headphones have you listened to that are very very close to each other and the Harman target on an FR chart, yet you clearly still have preferences among them?

 

[preload]

This seems like a strawman. I think most people around here agree that FR is the most important measurement, I don't think you'll find a single person who will take the position that it's the only measurement that matters.

You'd be surprised. Notwithstanding, there still appears to be a general overestimate of just how predictive simple FR measurements are of SQ, accompanied by a lack of appreciation that FR charts of headphones (particular OE) have significant comparability concerns due to the use of different measurement rigs.

 

[markanini]

This seems like a strawman. I think most people around here agree that FR is the most important measurement, I don't think you'll find a single person who will take the position that it's the only measurement that matters.

I've seen a lot of audio enthusiasts dismiss or praise speakers and headphones after only viewing a FR graph. That's practically the same as thinking that only measurements matter. This happens on a number or audio tech channels, including ASR. Browse a review thread here and you will be guaranteed to read multiple opinions like that. Even for products that are clearly developed for niche applications where scrutiny of FR would not necessarily apply.

 

[kemmler3D]

Yes, particularly for IE headphones, but ONLY if you are measuring with the same/identical rig that Harman uses (when trying to assess compliance to target) AND you are applying a regression formula to those measurements.

I guess I don't pay much attention to the regression scores, they're more of interesting trivia?

Oh I agree 100% with this, but I also would not expect the measurements to predict SQ beyond a certain point, for reasons previously stated.

Also, think about it, how many headphones have you listened to that are very very close to each other and the Harman target on an FR chart, yet you clearly still have preferences among them?

TBH I've heard a lot of headphones but I don't think I've heard that many that were legitimately super close to the harman target.

I will say that when I've heard a headphone or IEM and seen an FR plot, my heard impressions and the plot seem to correspond pretty well. I used to be able to listen to an IEM or headphone, make notes, and if I got to see an FR plot later I would usually congratulate myself for getting things right in the notes. I used to test a good number of IEMs for my job.

"SQ" is often conceived of as being this gestalt where FR is only one component among equals, but in my experience, changes in FR in a critical listening scenario are more dominant, subtle and multifarious than people realize.

A bump at 200-400hz can make the whole thing sound wooden, the highs seem to suffer at the same time. A little peak between 2-6 Khz can give everything a thin, metallic, sharp, type of sound. These subjective impressions feel like there should be more factors going into them, but it's just FR and perhaps some interaction with the THD characteristics of the headphone.

This only became really apparent to me after I spent a full workweek tweaking the EQ of a single headphone by ear.

So, bottom line, in my experience, FR is as big a determinant of SQ as they say. I don't think people should pay much attention to preference scores though. A preference score is useful for marketing or product planning, because it predicts what the average person will think - not as useful for individual consumers because it doesn't tell you what you will think.

 

[pablolie]

Accusing ASR of "bias towards measurements" is like 'accusing' Ducati of having a bias towards fast acceleration and fast circuit lap times in their sports motorcycle portfolio. If you're a vendor that claims to make audiophile gear, you better pass basic performance benchmarks to validate that status.

 

[markanini]

Accusing ASR of "bias towards measurements" is like 'accusing' Ducati of having a bias towards fast acceleration and fast circuit lap times in their sports motorcycle portfolio. If you're a vendor that claims to make audiophile gear, you better pass basic performance benchmarks to validate that status.

At the same time, this thread has generated lots of quality discussion and sharing of knowledge. How cool is to have an outcome like that from questioning an obvious position?

 

[MAB]

I’m not saying EQ can work miracles and fix every speaker. We’re not living in a world where a Jamo 365 magically can be made to sound like a Revel Salon. That said, EQ is still a super useful tool for improving sound quality. Sure, it can’t solve physical problems like resonances, cabinet vibrations, or bad transient response, but it’s great for fixing frequency imbalances. It can smooth out peaks and dips, make the sound more balanced, and even help with room issues.

No, EQ won’t fix a poorly designed speaker or turn a cheap one into a high-end model, but it can still make a big difference, even with budget setups. Most speakers whether they’re good or bad will sound better with some EQ, especially when it’s adjusted to fit your room, preferences, and listening position. This ties back to my earlier point about active speakers that come with different tunings or "house curves" built in.

I'm not talking about EQ'ing a bad speaker and ending up with sound like a Revel Salon.
I'm talking about EQ'ing resonances and ending up with worse performance. Or no better, just a different form of bad.
That's the point, many speakers will sound worse after an attempt (for instance) to flatten the on-axis performance.

 

[Verig]

"SQ" is often conceived of as being this gestalt where FR is only one component among equals, but in my experience, changes in FR in a critical listening scenario are more dominant, subtle and multifarious than people realize.

Excellent point. The magnitude of deviation may be very different looking than its effect.

With headphones this becomes even more interesting. For example I very much enjoy Hifiman Ananda and Arya. I don't really perceive the problems in highs as problems at all. I do hear it's not really neutral but the errors come as spacious feel and certain slight grain. I find this grain similar to a vinyl setup played through SET tube amps. So basically something weird in response + very subtle distortion and background which I really only seem to notice in high-mids/highs.

Quite subjective. But I like analyzing myself, what I hear and what I like. After learning a bit about myself it's far easier to see through the measurements what I'd perhaps enjoy even if it's not target curve. Bass is the easiest, I don't prefer Harman level but a bit lower. I find Harman messing with mids too much. There, done, but the rest is far more tricky.

 

[preload]

I've seen a lot of audio enthusiasts dismiss or praise speakers and headphones after only viewing a FR graph. That's practically the same as thinking that only measurements matter. This happens on a number or audio tech channels, including ASR. Browse a review thread here and you will be guaranteed to read multiple opinions like that. Even for products that are clearly developed for niche applications where scrutiny of FR would not necessarily apply.

Thank you! Glad someone else has noticed this phenomenon. 100%

 

[preload]

I guess I don't pay much attention to the regression scores, they're more of interesting trivia?

Regression analysis and preferences scores are the fundamental basis of Toole and Olive's statements about what measurements correlate with good sound quality. So it's more than "interesting trivia" IMO.

TBH I've heard a lot of headphones but I don't think I've heard that many that were legitimately super close to the harman target.

I have, and despite very very similar matching FR measurements, I still perceive clear differences in SQ, and I clearly have preferences. Perhaps you might want to talk to folks who have made these comparisons, I'm willing to bet nobody will say "they all sounded the same and I didn't care which ones I listened to."

So, bottom line, in my experience, FR is as big a determinant of SQ as they say

Great glad you believe this. Many people do.

 

[kemmler3D]

I have, and despite very very similar matching FR measurements, I still perceive clear differences in SQ, and I clearly have preferences.

How much of this do you think could be due to unaccounted-for variations in high frequencies?

Regression analysis and preferences scores are the fundamental basis of Toole and Olive's statements about what measurements correlate with good sound quality. So it's more than "interesting trivia" IMO.

Well yeah, how else are you going to develop a standard based on a large dataset, eyeball it?

My point is just that these scores aren't as useful in individual decision-making, for exactly the reason @Verig's describing - individual preference matters, especially in high frequency performance where measurements aren't as accurate.

I think that FR plots can give you a pretty clear idea of how something (headphone or speaker) will sound, if you are very accustomed to comparing them to what something actually sounds like. Granted not everyone has that experience.

But I think we would probably agree that even if you know how something is going to sound, it's ironically hard to predict if you will LIKE how it sounds. The broad (maybe even medium-sized) trends are predictable, but small 1-2dB deviations and their impact on nuanced subjective reactions to them are not very predictable IME.

Like if I see a 4dB hump at 400hz I know I'm going to hate it. I just personally hate the boxy, honky sound that comes from that. But if I see +1.5dB at 2khz, I expect to hear something, but I can't tell you if I would like it or not. Could be "shrill", could be "shimmer".

 

[MAB]

I've seen a lot of audio enthusiasts dismiss or praise speakers and headphones after only viewing a FR graph. That's practically the same as thinking that only measurements matter. This happens on a number or audio tech channels, including ASR. Browse a review thread here and you will be guaranteed to read multiple opinions like that. Even for products that are clearly developed for niche applications where scrutiny of FR would not necessarily apply.

I haven't taken a poll, but it seems to me an order of magnitude greater portion of people dismiss measurements outright, even when the measurements indicate a severely flawed product. So yeah, some people do misinterpret a single measurement as being important, and you can easily see it while browsing review threads, but way more people make excuses for bad performance than single out isolated areas of good performance.
I think you already know this, but a spinorama isn't a FR graph, it's a set of measurements of the 3D sound field from which a number of predictions and estimates can be made, like the FR in a room. Also, I agree we should leave out niche items (like in-wall speakers), since it is a distraction. Also, some products do come down to a single figure of merit, like the sound of tube amps is typically captured in one single figure of merit, the FR irregularities introduced when driving a low impedance speaker.

 

[Jim Taylor]

despite very very similar matching FR measurements, I still perceive clear differences in SQ,

How, exactly, do you define "sound quality"?

 

[preload]

How, exactly, do you define "sound quality"?

In the context of what I was saying, it's my subjective preference for the perceived sound reproduction when listening to familiar material

 

[Geert]

And even then, the predictive value isn't perfect, and primary validation of the Harman-funded research was done using single musical genre (rock).

Not only using rock music, but their research on the effect of the style of music on people’s performance to determine the quality of speakers and identify the speakers or headphones they prefer most showed rock music performed the best.

SoundStage! Access | SoundStageAccess.com (GoodSound.com) - Dr. Sean Olive on 35 Years of “Fast Car”

"We had the statistical evidence that people are better able to do the tests—they come up with answers much faster and much more consistently and with much more discrimination".

 

[kemmler3D]

In the context of what I was saying, it's my subjective preference for the perceived sound reproduction when listening to familiar material

Well... I'll just say there is no such thing as a number or graph that captures individual preference, unless you graph your preferences yourself.

I think FR is easily the single most important tool we have for guessing at personal preference, other than ears-on listening, but that doesn't make it perfect.

For the most part ASR defines sound quality as fidelity, and according to scientifically derived standards (Harman) when there is not a linear relationship between input and output. This is useful because fidelity can be directly quantified, but as you note, you might want something else / more.

The data can take you pretty far but I don't think most members here believe the data is the destination.

 

[preload]

Not only using rock music, but their research on the effect of the style of music on people’s performance to determine the quality of speakers and identify the speakers or headphones they prefer most showed rock music performed the best.

Correct! And this is why rock music was chosen for the purposes of Harman research studies, because listeners could more reliably differentiate their speakers preference. This means a smaller sample size and fewer trials were needed to obtain usable results.

HOWEVER, this also unfortunately limited the generalizability of the research because it is then not clear that their conclusions also apply to non-rock music. That's the problem.

 

[MAB]

Correct! And this is why rock music was chosen for the purposes of Harman research studies, because listeners could more reliably differentiate their speakers preference. This means a smaller sample size and fewer trials were needed to obtain usable results.

Yes.

HOWEVER, this also unfortunately limited the generalizability of the research because it is then not clear that their conclusions also apply to non-rock music. That's the problem.

This doesn't follow from the above. Please give examples. Also, please extend this concept to measurements, like should there be rock and roll settings as well as orchestra settings for a Klippel? And if so, what would those settings and measurements be?