"Bias" of some members towards headphone measurements?

[kyuu]

A brief perusal of those papers doesn't really seem to include anything to support your position. They also don't focus on actual playback systems. I am inclined to defer to the studies where these sorts of things were actually studied in the context of playback systems, where phase distortion has generally been found to only be detectable with headphones and test tones. Even then, there was no discernible preference for any particular phase relationship. This makes sense if you think about it: in any actual real life situation, the combination of direct and reflected sounds is going to be a veritable "phase soup".

If my recollection of the studies is incorrect or out-of-date, hopefully someone will correct me.

 

[Jaxx1138]

I worry about being misunderstood on this one. I do not believe in magic.
But while the ASR community is partly a bastion against snake oil and subjectivism, I think some ASR users tend to overrate the meaningfulness of measurable data when it comes to headphones. (Maybe this also goes for other audio devices, but I almost only read the headphone topics here.)

a) Most of us agree that the frequency response is the most important parameter. But it is all about the frequency response in your ear, not on some measurement rig (you cannot know it exactly in beforehand). I have quite a few IEMs and Over-Ears, and when I tune them to Harman, they all sound different - some quite significantly - as they interact with my ear in a different way (also HRTF, hair, glasses etc.). Soundstage also seems a bit random.
b) The Harman Target is a very helpful standard, but it is not the perfect target for everyone.
c) Distortion is important if it exceeds a certain amount. But many people completely overestimate how well they can hear it. Besides, it is irrelevant if a headphone has high distortion on 114dbspl if you never listen to it at 114dbspl anyway.

So basically, my point is that, while all this data surely is more helpful than highly subjective reviews, our ears still are not measurement rigs. The things you hear come from an interaction between headphone and your ear, and not everything which is measurable does really influence your listening experience.

- If a headphone does exactly hit the Harman target that doesnt mean that it will sound perfect to you (or that there is something wrong with your ear if it does not sound perfect to you).
- You probably cannot hear in a blind test if a headphone has less distortion unless one of them performs badly.
- If you like a headphone which was reviewed with average/mediocre results, you were not necessarily fooled. It is not necessarily a good idea to buy a "better" headphone if you didnt feel something was wrong before you read the review.

Headphone metrics are yet to be properly standardized. Repeatability is paramount. If one rig measures differently than another when measuring the same thing then there is some discrepancy. Loudspeaker metrics are highly repeatable with very tiny differences. Headphone metrics are still in debate as to which ones are more accurate. When headphone metrics meet these same standards then this discussion will become moot. The argument is fundamentally centered around the disagreement in what sounds good and then what some arbitrary metric displays on a graph. When the science is properly conducted and understood, only then will we be able to look at a series of measurements and know within a high level of certainty which devise is more accurate. The primary problem (IMO) is that the conflation of how we hear verses how the headphone preforms seems to be the crux of this issue. How we hear is irrelevant. No one considers HRTF's for measuring loudspeakers. And yet we can define their performance quit well with current standards. Headphones do provide the conundrum of using a human head as part of the system ( providing the particular acoustic impedance). Outside of that, characterizing the distortion and frequency response should be a more straightforward process. In other words the measuring rig need not duplicate a human head but only the proper loading for the acoustic impedance which is part of the overall tuning.

 

[pablolie]

Headphones will always be a very personal preference, why is that so hard to admit, and why the debates about what is "right"? The fit, the shape of our ear channels... the primary interacting conduits to headphone listening are *not* identical for any of us. And this is not just about personal preference, it's about physics... which is the reason many headphone apps test for fit and ear channel and what not.

 

[NTK]

Standard loudspeaker measurements are free field (i.e. measured in an anechoic chamber or are quasi-anechoic measurements), and the measurements can be replicated by anyone with the necessary skill and resources.

Headphones measurements are always measured in a reverberant "room" (and a very finicky one at that where the "room modes" cover the whole audible frequency range and can change drastically with small variations in the seating and sealing of the headphones). The measurements can be only replicated by someone with an exact replica of that "room" (test fixture). Your ear/headphone combination isn't that "room".

If you take the speakers room EQ optimized for someone else's room, and you move those speakers to your room which can be very different, what is the likelihood that those room EQ parameters will also be optimal for your room?

 

[Jaxx1138]

Headphones will always be a very personal preference, why is that so hard to admit, and why the debates about what is "right"? The fit, the shape of our ear channels... the primary interacting conduits to headphone listening are *not* identical for any of us. And this is not just about personal preference, it's about physics... which is the reason many headphone apps test for fit and ear channel and what not.

There are definitely personal preferences for individuals. This has nothing to do with physics or sound reproduction. That is all science and fact based which is all objective. What is "right' would equate to being accurate. That has nothing to do with your opinion about it. That is often the conflation with many audiophile enthusiast. Just because you prefer something does not make it correct. Music is art which is a subjective, in that it has no reference for being accurate. Sound reproduction is either as close to accurate as possible or it is colored, edited, distorted or just inaccurate. Audio reproduction may be deemed "personal preference" but true audiophile reproduction and the tools used to achieve it are trying to recreate an original as true to form as possible. Not to what one individual prefers.

 

[Jaxx1138]

Headphones measurements are always measured in a reverberant "room" (and a very finicky one at that where the "room modes" cover the whole audible frequency range and can change drastically with small variations in the seating and sealing of the headphones)

I am not certain where you may have sourced this. But I do not believe this is "always" the case. And there is no standard that I am aware of that supports this claim. In headphone metrics the "room" has little to do with the metric. The room is insignificant (up to a point) in that unless there is extreme background noise the room should not impart much to the measurement of a headphone. Many metrics for headphones are done in an isolation booth or even in a full anechoic chamber in some cases.

 

[kyuu]

I am not certain where you may have sourced this. But I do not believe this is "always" the case. And there is no standard that I am aware of that supports this claim. In headphone metrics the "room" has little to do with the metric. The room is insignificant (up to a point) in that unless there is extreme background noise the room should not impart much to the measurement of a headphone. Many metrics for headphones are done in an isolation booth or even in a full anechoic chamber in some cases.

You have completely failed to grasp what NTK was saying. The clue is in this sentence, pay particular attention to the paranthetical:

The measurements can be only replicated by someone with an exact replica of that "room" (test fixture).

 

[Jaxx1138]

Had not seen this thread before and somehow landed on page 8 where member who used to work at Harman was asking why we use a headphone fixture whereas we don't for speakers.

There is a simple answer: the output of a headphone is very low as to try to measure it in an anechoic chamber or Klippel NFS. The fixture solves that problem by sealing the space between the headphone/IEM and putting the microphone very close to it. This in turn causes acoustic coupling issues due to frequencies involved. To solve this standardized fixtures were developed which while not perfect, is all we have. The fixture also removes the external aspects of the headphone from interfering with the measurements.

Given what we are stuck with, then research was performed to see what measurements correlate with preference. To the extent that was successful, then we have nullified the negative effects of using a fixture.

Same thing happened with speakers where an anechoic measurement succeeded as reference as opposed to say, always measuring a speaker in certain room.

It's not that a fixture is being utilized, it is the type of fixture using pinna that introduces an acoustic filter which is an aberration to the metric itself. The pinna effect and all of the HRTF's should not be part of the measurement because it is a variable and has little to nothing to do with the acoustic performance of the headphone. Only the acoustic impedance must be duplicated in order to discern the frequency response and distortion characteristics. The reason an anechoic chamber is used (or free field/4pi) for loudspeakers is to eliminate the variable acoustic effects of room boundaries. This same principle should be applied with headphone metrics.

 

[GGroch]

Standard loudspeaker measurements are free field (i.e. measured in an anechoic chamber or are quasi-anechoic measurements), and the measurements can be replicated by anyone with the necessary skill and resources.

They are standardized and replicable for sure. But (almost) no one listens to speakers in anechoic chambers. My guess is that real listening rooms vary from each other far more than do differences between individuals pinna and ear canals. Also, headphone measurement rigs come much closer to mimicking ear canals than anechoic chambers mimic living rooms.

Harman's target curves for speakers then, are likely to be significantly less accurate in the real world than are headphone targets if the headphone measurements were made in a well designed (if not standardized) measurement rig. In either case, measurements of the transducer's output is just a useful estimate, a valuable starting point.

Fortunately Room EQ (for speakers) and Headphone personalization software (including Sony's Ear Shape Analysis; Denon's Nura & Bose Custom Tune Ear Canal "Sonar", and everyone else's app based hearing tests) can improve accuracy.

 

[Jaxx1138]

Yeah, well... this is Audio Science Review... if it wasn't heavily biased toward science that would be odd. Communicating about sound quality using primarily non-scientific ideas has proved very detrimental to the industry and consumers over the past 30 years or so.

I'm not one of those people that will argue it's inappropriate and worthless to speak subjectively about audio, but I will argue that there is still a dearth of scientific thinking in the consumer audio world, and the pendulum has not swung nearly far enough toward scientific / quantitative rigor and away from magical thinking and self-delusion. Maybe someday, but not yet.

Until people like Mark Levinson and Rob Watts are laughed off the internet instead of idolized for spouting their brand of nonsense, we should ask for more science, not less.

Also, it's worth noting that recorded audio is a completely quantifiable entity until it hits the loudspeaker, and even after that we can do pretty well with Klippel and other measurements. The idea that we might prefer non-quantitative modes of analysis of audio electronics is like wanting reviews of CPUs or GPUs without any benchmarks. You can tell me the framerate looked high or "silky smooth" or whatever, but we'll all save time if you just tell me what the framerate WAS.

Hear, hear ( No pun intended)

 

[NTK]

It's not that a fixture is being utilized, it is the type of fixture using pinna that introduces an acoustic filter which is an aberration to the metric itself. The pinna effect and all of the HRTF's should not be part of the measurement because it is a variable and has little to nothing to do with the acoustic performance of the headphone. Only the acoustic impedance must be duplicated in order to discern the frequency response and distortion characteristics. The reason an anechoic chamber is used (or free field/4pi) for loudspeakers is to eliminate the variable acoustic effects of room boundaries. This same principle should be applied with headphone metrics.

I used the term "room" for headphone testing very loosely, and that's why I put them in quotes (" "). What I meant was this:

Compared to this (BK 5128):

Or this (Gras RA040X):

 

[Jaxx1138]

I used the term "room" for headphone testing very loosely, and that's why I put them in quotes (" "). What I meant was this:

View attachment 511602

Compared to this (BK 5128):
View attachment 511603

Or this (Gras RA040X):
View attachment 511604

In that I agree with you. And this is why this is such a mess. Because of the differing attempts to creating the atomy of a human(which is variable). We are not evaluating the human hearing mechanism. We are attempting to evaluate the headphone and not how humans hear. Just like using an anechoic chamber to eliminate the acoustical room boundary variables. But that does not mean we listen to loudspeakers in an anechoic chamber because it is simply a tool.

 

[pablolie]

There are definitely personal preferences for individuals. This has nothing to do with physics or sound reproduction.

No one claimed otherwise.

That is all science and fact based which is all objective. What is "right' would equate to being accurate.

It is also a fact that different people will hear the accurate result differently for a variety of reasons your somewhat simplistic "science based" statement fails to acknowledge, especially with headphones. You need to read up on the Head-Related Transfer Function (HRTF), specifically in regard to headphones.

That has nothing to do with your opinion about it...

And that is (a) unrelated (b) downright rude.

 

[usern]

your somewhat simplistic "science based" statement fails to acknowledge, especially with headphones. You need to read up on the Head-Related Transfer Function (HRTF), specifically in regard to headphones.

You make it sound as if HRTF is not science based.

 

[pablolie]

You make it sound as if HRTF is not science based.

I stated quite the contrary. Quite clearly. Any truly educated discussion about headphones needs to consider it, especially when preferences (aka "bias" as in the title of this very topic) are discussed.

 

[Jaxx1138]

It is also a fact that different people will hear the accurate result differently for a variety of reasons your somewhat simplistic "science based" statement fails to acknowledge, especially with headphones. You need to read up on the Head-Related Transfer Function (HRTF), specifically in regard to headphones.

It does not matter the differences in each individuals hearing. The reference will stay the same. If a trumpet is played in a concert hall with two individuals listening, it will still be the same reference.

 

[Jaxx1138]

And that is (a) unrelated (b) downright rude and (c) silly.

My apologies, I didn't intend to personalize it. I was simply stating the differences between opinions (subjective) vs factual data (objective).

 

[pablolie]

It does not matter the differences in each individuals hearing. The reference will stay the same. If a trumpet is played in a concert hall with two individuals listening, it will still be the same reference.

I am in no way disputing that the original signal is the reference, and that we can measure whether the chain that provides us with said signal does deliver it accurately.

However, headphone measurements -while a very useful reference- are also a bit limited when it comes to capture our entire headphone experience. That's why I mentioned HRTF.

My personal experience is that things like accuracy, linearity etc are more reliable as a guarantee of balanced sound when it comes to speakers, rather than headphones. A well-measuring speaker always sounds good to me. With headphones, I am never as sure. And the reason is probably HRTF, I would venture to think. I find it fascinating though: with headphones, quite often the recommended PEQ sounds horrific to me. That never happens with speakers.

The fact is that headphones are -and for the time being will stay- a very personal experience. Whatever recommended, widely approved reference one goes for... it'll probably travel from the headphones through our ear canal into our eardrums differently. To me, that explains the variability in personal preference even among very experienced, bias-free listeners.

It's a wordy way to say that I agree with where the OP of this topic is coming from - the reference measurements on measurement gear are probably not as reliable to predict what we'll hear with headphones. With speakers, the spatial cues we're trained on arrive as they are meant to, with headphones probably not so... if they would, we would somehow *have* headphone technology that present us with a perfect "out of ear" experience... and that tech oddly enough doesn't exist (afaik).

[And please don't say binaural, because I have never heard a binaural track that sounds like it's completely outside my head. ]

 

[pablolie]

My apologies, I didn't intend to personalize it. I was simply stating the differences between opinions (subjective) vs factual data (objective).

Thanks, no offense taken whatsoever, then! I respect your argument; I was just nuancing it some with HRTF.

 

[markanini]

It does not matter the differences in each individuals hearing. The reference will stay the same. If a trumpet is played in a concert hall with two individuals listening, it will still be the same reference.

For headphones individual coupling matters more than you think, because of individual variability massively affecting FR, the value of a reference needs to be tempered to some extent. I'm more of an objectivist than most, even among ASR members, and will happily acknowledge the limitations of headphones measurement systems and their data. This is what Amir does in practice, by complementing his headphone measurements with practical listening tests, and EQ-ing manually, not aiming perfectly match a target curve.

DAC, amps, interconnects etc are linear systems where absolute references have more meaning. OTOH transducers coupled to artificial or biological ears are almost the opposite of a linear system. On top of that they carry anthropomorphic variability at the per-listener level. It stands to reason that you have to treat them as divergent systems.

Let me recommend Amirs excellent video that goes in depth on the subject.