"Bias" of ASR community towards measurable parameters?

[Dazerdoreal]

I worry about being misunderstood on this one. I do not believe in magic.
But while the ASR community is partly a bastion against snake oil and subjectivism, I think some ASR users tend to overrate the meaningfulness of measurable data when it comes to headphones. (Maybe this also goes for other audio devices, but I almost only read the headphone topics here.)

a) Most of us agree that the frequency response is the most important parameter. But it is all about the frequency response in your ear, not on some measurement rig (you cannot know it exactly in beforehand). I have quite a few IEMs and Over-Ears, and when I tune them to Harman, they all sound different - some quite significantly - as they interact with my ear in a different way (also HRTF, hair, glasses etc.). Soundstage also seems a bit random.
b) The Harman Target is a very helpful standard, but it is not the perfect target for everyone.
c) Distortion is important if it exceeds a certain amount. But many people completely overestimate how well they can hear it. Besides, it is irrelevant if a headphone has high distortion on 114dbspl if you never listen to it at 114dbspl anyway.

So basically, my point is that, while all this data surely is more helpful than highly subjective reviews, our ears still are not measurement rigs. The things you hear come from an interaction between headphone and your ear, and not everything which is measurable does really influence your listening experience.

- If a headphone does exactly hit the Harman target that doesnt mean that it will sound perfect to you (or that there is something wrong with your ear if it does not sound perfect to you).
- You probably cannot hear in a blind test if a headphone has less distortion unless one of them performs badly.
- If you like a headphone which was reviewed with average/mediocre results, you were not necessarily fooled. It is not necessarily a good idea to buy a "better" headphone if you didnt feel something was wrong before you read the review.

 

[majingotan]

Objective measurements are objective and by default not biased. Your preference/opinions may or may not align with what's objective and that's perfectly fine. The point is to have a reference of which products are objectively performing and which ones are broken or poorly performing/distorting under excursion.

 

[Barrelhouse Solly]

Measurements provide the best view we can get of signal quality. I think of them as a valuable decision support tool. The basic question measurements answer is, "How close is the output of a device to the input?" What you hear and your personal preferences are a different question.

 

[InfiniteJester]

Yes. That we don't have a standard to measure something yet doesn't mean that that thing necessarily doesn't exist.

Absence of proof doesn't mean proof of absence.

At least some users here like to think that soundstage doesn't exist. I prefer the Rtings approach, because they are trying to measure it; with more or less success.

 

[markanini]

I worry about being misunderstood on this one. I do not believe in magic.
But while the ASR community is partly a bastion against snake oil and subjectivism, I think some ASR users tend to overrate the meaningfulness of measurable data when it comes to headphones. (Maybe this also goes for other audio devices, but I almost only read the headphone topics here.)

a) Most of us agree that the frequency response is the most important parameter. But it is all about the frequency response in your ear, not on some measurement rig (you cannot know it exactly in beforehand). I have quite a few IEMs and Over-Ears, and when I tune them to Harman, they all sound different - some quite significantly - as they interact with my ear in a different way (also HRTF, hair, glasses etc.). Soundstage also seems a bit random.
b) The Harman Target is a very helpful standard, but it is not the perfect target for everyone.
c) Distortion is important if it exceeds a certain amount. But many people completely overestimate how well they can hear it. Besides, it is irrelevant if a headphone has high distortion on 114dbspl if you never listen to it at 114dbspl anyway.

So basically, my point is that, while all this data surely is more helpful than highly subjective reviews, our ears still are not measurement rigs. The things you hear come from an interaction between headphone and your ear, and not everything which is measurable does really influence your listening experience.

- If a headphone does exactly hit the Harman target that doesnt mean that it will sound perfect to you (or that there is something wrong with your ear if it does not sound perfect to you).
- You probably cannot hear in a blind test if a headphone has less distortion unless one of them performs badly.
- If you like a headphone which was reviewed with average/mediocre results, you were not necessarily fooled. It is not necessarily a good idea to buy a "better" headphone if you didnt feel something was wrong before you read the review.

I agree. It's irrational to ignore the variability factors of IEMs and headphones and fixate on what appears flat on a graph. Unless someone is ignorant of FR variability data, but this is frankly not hard to come by.

 

[ZolaIII]

There is bias on ASR but it's not related to "objective data". Methodology is evolving especially regarding ear chanel shape and outer pina impact on revived data, microphone also (for new HATS and ear shapes). Harman preference is up to date it whose done and to the used SPL influenced/not influenced participating subjects tone controls alike (and not greatly done) prefered levels which where then averaged. However Sean Olive with it's resources and Harman ongoing research is still a leading point to a right direction.

It's not set in stone nor will it ever be, psy accustic adjustments as equal loudness apply on all.

 

[DVDdoug]

Most of us agree that the frequency response is the most important parameter.

Count me in that camp.

But it is all about the frequency response in your ear, not on some measurement rig (you cannot know it exactly in beforehand).

True, but it can usually give you a good relative measure. If a headphone has a +6dB bass boost over another, everybody should hear the difference. (That's assuming good "sealing" which can be different on different heads, and glasses can really mess that up.)

If you're doing reviews the only alternative would be to have a consistent listening panel, and for that panel to listen "blind". That's not always practical and not necessarily reliable/repeatable.

It doesn't hurt to listen for yourself before buying.

The Harman Target is a very helpful standard, but it is not the perfect target for everyone.

True. It's a statistical preference. I think Amir prefers a little more bass... and I think I do too but I'm not a high headphone guy.

c) Distortion is important if it exceeds a certain amount. But many people completely overestimate how well they can hear it. Besides, it is irrelevant if a headphone has high distortion on 114dbspl if you never listen to it at 114dbspl anyway.

The problem usually is when you boost the bass, to better-match the Harman or just because you like it.

I have quite a few IEMs and Over-Ears, and when I tune them to Harman, they all sound different - some quite significantly - as they interact with my ear in a different way (also HRTF, hair, glasses etc.).

It's not easy to exactly-match the Harmon curve. And it's hard to do an unbiased blind comparison because even if someone helps you they probably feel different in your ear, and weight, etc.

Soundstage also seems a bit random.

Most people don't get anything like a realistic soundstage illusion with headphones (or IEMs). Headphone soundstage survey

 

[sam_adams]

. . . not everything which is measurable does really influence your listening experience.

Do you have any measurements to support this statement?

 

[InfiniteJester]

Another peculiar thing is that we simultaneously seek Harman preference response curve and low distortion levels, despite moderately high distortion being, apparently, preferred by many listeners.

 

[MAB]

I worry about being misunderstood on this one. I do not believe in magic.
But while the ASR community is partly a bastion against snake oil and subjectivism, I think some ASR users tend to overrate the meaningfulness of measurable data when it comes to headphones. (Maybe this also goes for other audio devices, but I almost only read the headphone topics here.)

a) Most of us agree that the frequency response is the most important parameter. But it is all about the frequency response in your ear, not on some measurement rig (you cannot know it exactly in beforehand). I have quite a few IEMs and Over-Ears, and when I tune them to Harman, they all sound different - some quite significantly - as they interact with my ear in a different way (also HRTF, hair, glasses etc.). Soundstage also seems a bit random.
b) The Harman Target is a very helpful standard, but it is not the perfect target for everyone.
c) Distortion is important if it exceeds a certain amount. But many people completely overestimate how well they can hear it. Besides, it is irrelevant if a headphone has high distortion on 114dbspl if you never listen to it at 114dbspl anyway.

So basically, my point is that, while all this data surely is more helpful than highly subjective reviews, our ears still are not measurement rigs. The things you hear come from an interaction between headphone and your ear, and not everything which is measurable does really influence your listening experience.

- If a headphone does exactly hit the Harman target that doesnt mean that it will sound perfect to you (or that there is something wrong with your ear if it does not sound perfect to you).
- You probably cannot hear in a blind test if a headphone has less distortion unless one of them performs badly.
- If you like a headphone which was reviewed with average/mediocre results, you were not necessarily fooled. It is not necessarily a good idea to buy a "better" headphone if you didnt feel something was wrong before you read the review.

A "parameter" by definition is a measured quantity.

Definition of PARAMETER

an arbitrary constant whose value characterizes a member of a system (such as a family of curves); also : a quantity (such as a mean or variance) that describes a statistical population… See the full definition

www.merriam-webster.com

So, yes on that.

Targets are about preference. Measurements define that target and allow us to achieve it, rather than just guess. If you like a different target, at least you have a starting point, and a standard. And the Harman is a pretty good starting point. Nobody here seriously is saying you need to listen to EQ that matches the Harman parameter, Amir and others state this over and over. Many headphones get poor reviews because they can't be EQ'ed to any reasonable curve because of resonances, distortions, etc. Not because they can't do Harman right.

 

[RDoc]

This is Audio Science Review. Science doesn't claim to understand everything, or even to be completely correct about the theories that are current. What it does claim is to be rational and reproducible in it's claims and to modify itself when new evidence is presented. The idea is to produce better and better models of reality, not to find some sort of ultimate understanding.

I've no doubt at all that our current set of measurements isn't sufficient to really capture what people perceive of as "good sound". However, I also believe that the way forward is with more scientific measurements, blind, reproducible, large samples. These are very difficult to do and interpret but are likely to be more useful than any other methodology I'm aware of.

 

[terryforsythe]

Measurements are extremely important.

However, sometimes we make judgments based upon an incomplete set of measurement data. For example, some amplifiers exhibit different characteristics when measured using a purely resistive load as compared to a load with a complex impedance. If a complex load is not used when taking the measurements, this may be missed. Nonetheless, measurement equipment and techniques continue to improve.

 

[markanini]

This is Audio Science Review. Science doesn't claim to understand everything, or even to be completely correct about the theories that are current. What it does claim is to be rational and reproducible in it's claims and to modify itself when new evidence is presented. The idea is to produce better and better models of reality, not to find some sort of ultimate understanding.

I've no doubt at all that our current set of measurements isn't sufficient to really capture what people perceive of as "good sound". However, I also believe that the way forward is with more scientific measurements, blind, reproducible, large samples. These are very difficult to do and interpret but are likely to be more useful than any other methodology I'm aware of.

No one is questioning science though. OTOH a multitude of users will insist that they know all that's needed to be known based on an incomplete understanding of science. I've addressed users who complained about target adherence with updated data, and they don't acknowledge it and return to raise the same concern in short time. Or they demonstrate ignorance of statistical significance. It's remarkable behavior by those who claim to be in service of science, and suggests that the level of knowledge will vary no matter what.

 

[akamatsu]

Measurements are extremely important.

However, sometimes we make judgments based upon an incomplete set of measurement data. For example, some amplifiers exhibit different characteristics when measured using a purely resistive load as compared to a load with a complex impedance. If a complex load is not used when taking the measurements, this may be missed. Nonetheless, measurement equipment and techniques continue to improve.

Extremely important to whom? I've never measured nor looked at measurements when selecting kit. However, the engineers that designed my kit probably did some measurements along the way.

 

[muza_1]

The thing with IEMs is that because ear anatomy you cannot trust that you are going to hear what the measurements suggest BUT for the same reason you cannot trust subjective reviews or opinions, I personally take the measurement* every time, is not perfect but it takes me closer to my preference, of course you have to try it yourself in the end.

The thing that I don't understand is the push back the Harman curve gets, even if you don't like it at all it's a very useful tool to have, once you know your preference you can compare it against it, it's way easier than just look at curves without a point of reference or against each other.

*as flawed as it might be.

 

[terryforsythe]

Extremely important to whom?

To me, and probably to a large portion of ASR members.

I've never measured nor looked at measurements when selecting kit.

If that is what makes you happy, nothing wrong with that.

 

[EERecordist]

I think you are asking too much of transducer measurements.

Transducers vary more than electronics. So measuring IEMs, headphones, speakers, phono cartridges, and microphones is hard because you have to measure a transducer with a transducer. Transducers are far from linear compared to electronics.

Our inexact headphone & IEM measurements are a starting point. Then find what you like with your reference tracks. What people like in transducers is as varied as what people like in food. Maybe someday we will have the Taste Precision analyzer? A friend of mine worked for the government and a smell transducer is probably be possible.

There is a lot of fundamental research by Fraunhofer, Bell Labs, universites, industrial labs of companies big enough to support them, small companies that subcontract testing, and results published in academic journals subject to peer review.

 

[akamatsu]

To me, and probably to a large portion of ASR members.


If that is what makes you happy, nothing wrong with that.

"Measurements are extremely important" to some people. But completely unnecessary to assemble a good hifi system.

 

[Purité Audio]

That rather depends upon your definition of ‘good’.
Keith

 

[muza_1]

"Measurements are extremely important" to some people. But completely unnecessary to assemble a good hifi system.

It might as well be for Electronics but they are way different than IEMs, headphones or speakers, now a days you know you are supposed to get a functional and transparent unit (in most cases), if you do that with speakers, IEMs or headphones you wouldn't have the faintest idea of how they will sound and given the number of options you can spend a lifetime choosing at random for something you like.