ASR Headphone Testing and BK 5128 Hats Measurement System

[March Audio]

If you'll allow me, for a moment, I think that your misconception here is that your experience does not align with what I am saying rather than anything else.

Thats a fair point, I have the same argument with people regarding speakers. Their personal experience doesnt equal "the truth" for lack of a better phrase.

However I dont have this in-congruent experience with speakers.

 

[Mad_Economist]

Thats a fair point, I have the same argument with people regarding speakers. Their personal experience doesnt equal "the truth" for lack of a better phrase.

However I dont have this in-congruent experience with speakers.

Kind of gets down to what's incongruous though, doesn't it? Like, I know I'm being pedantic here, but I legitimately am not sure that there's an incongruity between the experiences y'all are having, and what is to be expected from the headphone metrology side, and I'm not sure that's been made sufficiently clear.

 

[March Audio]

Perhaps it is the dosage that's differing, then Put it this way: at present, I would not necessarily expect a headphone's response to even strongly resemble an HRTF when it's on a head, even if it's a highly priced model, in the present day. It's comparatively rare to see a speaker that's even close to as far away from the anechoic targets as almost all headphones are from the on-HATS response targets we have.



Ah, yes, this clarifies it - physiological variation will absolutely occur, but its impact on subjective response depends on how correlated HpTF (headphone eardrum response) and HRTF-under-target-conditions (I didn't used to have to use these annoying constructions when DF-HRTF was the only target...) differ on an individual basis.

Perhaps it is the case that so many headphones have bonkers response curves. Why that would be considering the technology and knowledge we have now is a curious question. The knowledge gained about speakers isnt in isolation.

So the question I pose is how much do the physiological differences change the "response" of an enclosed space system (headphone on ear) compared to that of listening open "to the room"?

 

[Mad_Economist]

Perhaps it is the case that so many headphones have bonkers response curves. Why that would be considering the technology and knowledge we have now is a curious question.

I invite you to take a random sampling and consider how you'd feel about speakers whose frequency responses were as far from anechoic flat as the samples are from target.

As far as the bolded section, the answer is probably more in the domain of some sort of abnormal psychologist, given my experiences within the headphone field.

So the question I pose is how much do the physiological differences change the "response" of an enclosed space system (headphone on ear) compared to that of listening open "to the room"?

A very good question, and unfortunately one that can only really have incomplete answers. Surprisingly little work has been done directly correlating individual HRTF and individual HpTF, but it's a specific interest point for me precisely for this reason - if HRTF varies between two people one way, but HpTF for a given headphone varies between them another way, you're obviously going to have a subjective disconnect in what they hear. What I've been able to establish thus far (from a few different measurement heads and a couple of very unenthused volunteers - people don't like probe mics - so take it with all appropriate grains of salt) is that for larger circumaural designs, particularly low impedance ones such as acoustically open electrodynamic headphones and planar headphones, variation in HpTF between heads clusters in ways that has similarity to differences in diffuse field HRTF.

If I'm ever profoundly rich enough to take time off from trying to make money to do science, I fully intend to do a properly sized study of specifically this. Until then, however, we can rest with the uneasy reassurance that given that the IEM preference model, which tautologically doesn't include individual HRTF elements correlates extremely robustly with user preference, we can at least figure out a likely order of preference even when individual variation is disregarded.

 

[March Audio]

A very good question, and unfortunately one that can only really have incomplete answers. Surprisingly little work has been done directly correlating individual HRTF and individual HpTF, but it's a specific interest point for me precisely for this reason - if HRTF varies between two people one way, but HpTF for a given headphone varies between them another way, you're obviously going to have a subjective disconnect in what they hear. What I've been able to establish thus far (from a few different measurement heads and a couple of very unenthused volunteers - people don't like probe mics - so take it with all appropriate grains of salt) is that for larger circumaural designs, particularly low impedance ones such as acoustically open electrodynamic headphones and planar headphones, variation in HpTF between heads clusters in ways that has similarity to differences in diffuse field HRTF.

If I'm ever profoundly rich enough to take time off from trying to make money to do science, I fully intend to do a properly sized study of specifically this. Until then, however, we can rest with the uneasy reassurance that given that the IEM preference model, which tautologically doesn't include individual HRTF elements correlates extremely robustly with user preference, we can at least figure out a likely order of preference even when individual variation is disregarded.

This is precisely what Im alluding to. Of course its very weak evidence, borderline no evidence at all, but what I have experienced and observed in others regarding headphone preferences seems to suggest there is a disconnect. With speakers I seem to hear what others hear. With headphones Im confident that I am not hearing what others hear with the same can.

The IEM model makes sense - it (generally) lacks the physiological variation and one would intuitively expect it to conform strongly to a specific response curve.

 

[Frank Dernie]

You're correct that the results are highly variable in situ, but the same can of course be said of an unwindowed in-room measurement of a speaker at an arbitrary position. The different with speakers is that we've got a fairly nifty toolkit for divorcing ourselves from the reality of the playback situation for practical measurements, whereas with headphones, you must take the whole kit and caboodle as well. This reflects the reality of headphones on heads, however - the audible effects of positional variation in headphones are meaningful, just as moving around within a listening room is meaningful to our subjective perceptions.

Sorry to be OT but what is your view on the AKG N90 approach here?
It seemed a sensible idea to me.

 

[Mad_Economist]

This is precisely what Im alluding to. Of course its very weak evidence, borderline no evidence at all, but what I have experienced and observed in others regarding headphone preferences seems to suggest there is a disconnect. With speakers I seem to hear what others hear. With headphones Im confident that I am not hearing what others hear with the same can.

How do you square that with the relatively consistent headphone preference rankings per Olive? For that matter, while I don't want to sound invective-y, I have heard precisely the same argument regarding variations in what people perceive regarding speakers, headphones, amplifiers, digital-to-analog converters marshaled by folks who were skeptical of the correlation of measurements with perception there too...

The IEM model makes sense - it (generally) lacks the physiological variation and one would intuitively expect it to conform strongly to a specific response curve.

Not when you consider that subjective response is HpTF minus listener HRTF (yada yada circumstances yada). If a headphone measures the same on all listeners, and their HRTFs vary, the subjective response will vary equivalently to their HRTFs.

Sorry to be OT but what is your view on the AKG N90 approach here?
It seemed a sensible idea to me.

It has been a while since I read the AES paper but my memory is that it seemed to make some not-entirely-justified conjectures about what we can infer regarding eardrum response from a measurement right in front of the headphone driver. Nonetheless, I think that it's a good idea, I just have some misgivings about how well it would work.

 

[March Audio]

I invite you to take a random sampling and consider how you'd feel about speakers whose frequency responses were as far from anechoic flat as the samples are from target.

.

I have Senn HD6xx, HiFiMan 4xx, Oppo PM1 and PM3. I will try those EQ curves. Shame there isnt one for the PM3 which is my preference.

 

[Mad_Economist]

I have Senn HD6xx, HiFiMan 4xx, Oppo PM1 and PM3. I will try those EQ curves. Shame there isnt one for the PM3 which is my preference.

There is a PM3 EQ, assuming you mean the Oppo.

The PM3 is often held up as a standard-bearer for the Harman research, incidentally

 

[March Audio]

How do you square that with the relatively consistent headphone preference rankings per Olive? For that matter, while I don't want to sound invective-y, I have heard precisely the same argument regarding variations in what people perceive regarding speakers, headphones, amplifiers, digital-to-analog converters marshaled by folks who were skeptical of the correlation of measurements with perception there too...


Not when you consider that subjective response is HpTF minus listener HRTF (yada yada circumstances yada). If a headphone measures the same on all listeners, and their HRTFs vary, the subjective response will vary equivalently to their HRTFs.


It has been a while since I read the AES paper but my memory is that it seemed to make some not-entirely-justified conjectures about what we can infer regarding eardrum response from a measurement right in front of the headphone driver. Nonetheless, I think that it's a good idea, I just have some misgivings about how well it would work.

Yes absolutely, as I mentioned I make the same arguments myself so I am very well aware of the risk of self delusion here. I have also seen a lot of people say the olive response is too bassy.

Its not quite that simple. Yes everybodies HRTF is different, but they still prefer a flat response speaker. An IEM doesnt have the variation caused by the individual anatomy (well to a much smaller extent) so its essentially the response of the driver. Its a more "consistent" system. However with an enclosed system of the headphone over the ear you are going to get quite disparate effects dependant upon that anatomy. Which leads us back to the previous question of the difference between "listening to the room" and listening to the "closed system".

 

[March Audio]

There is a PM3 EQ, assuming you mean the Oppo.

The PM3 is often held up as a standard-bearer for the Harman research, incidentally

Sorry I meant PM1, but yes I have the PM3 also.

FWIW I prefer the PM1 over the PM3 followed by HD6xx followed by an utter dislike of the H4xx . Another interesting one is that I have tried the HD800 multiple times - tried my best because it has been so lauded, but I really dont like it at all. As an observation I seem to have an aversion to any headphone that measures (relatively) bright.

 

[pozz]

As you wish.

You're correct that the results are highly variable in situ, but the same can of course be said of an unwindowed in-room measurement of a speaker at an arbitrary position. The different with speakers is that we've got a fairly nifty toolkit for divorcing ourselves from the reality of the playback situation for practical measurements, whereas with headphones, you must take the whole kit and caboodle as well. This reflects the reality of headphones on heads, however - the audible effects of positional variation in headphones are meaningful, just as moving around within a listening room is meaningful to our subjective perceptions.


This is where you're going off the reservation. The inverse HRTF filtering process we apply to speakers we apply as well to headphones (albeit we don't use the inverse filter set for a frontal sound source; see Theile 1986, 2016, etc), and the same rules for timbral accuracy apply: we need eardrum sound power to approximate the listener's head in the sound field of the perceived acoustic source (arguably a diffuse field, see Theile if you care but honestly nobody does these days) to perceive the sound as "correct", or "flat", or what have you.

In a scenario where HRTF varied wildly - which, to some degree, it does, at least at some frequencies - and the response at the eardrum of headphones was absolutely constant across users, you would be quite correct that we would be unable to predict subjective tonality above perhaps 2khz or so. In such a world, each headphone would effectively be different to each user because it was the same and the users differed. I've spent a fair amount of time measuring headphones on different pinnae, however, and I simply do not see this effect - there are meaningful differences between the same headphone measured on two different sets of ears, and two different headphones on two sets of ears tend to differ in similar ways, at least for the circumaural designs that make up the majority of the high-end market. I'll freely acknowledge this as a caveat of any design that bypasses the ear anatomy - and IMO it contributes to the higher variation of preference in in-ear designs, which bypass the ear almost entirely - but it's my opinion that you just can't reconcile the degree of skepticism you're voicing with the degree of consistency in ex. Sean Olive's correlation of headphone response on a mannequin to subjective assessments.

I'll second this,

The variability in headphone/IEM measurement so far has been due to technique and tools. We see that readily enough in the inconsistencies in published measurements but lack an open database of HRTFs or preference studies matching individual or ethnic HRTFs to headphone choice.

Sean Olive's published work on the headphone choice and culture/country showed consistency in preference:

So it seems that even with differences in HRTF the neural/cognitive mechanism of aural adaptation is so consistent that a person could be said to "listen through headphones" the same way Toole said that people "listen through rooms".

Where HRTF does matter is binaural simulation, which has less to do with the headphone than precise individualized EQ done digitally at the source.

What constitutes a good headphone may have less to do with compensating for apparent anatomical differences than certain basics of construction and tuning, which are still being worked out, again depending more on a consistency in FR trends similar to those we've grown to expect in speakers, where smoothness and predictability dominate the response for an otherwise unknown room and individual listener.

With all that in mind a headphone/IEM measurement rig should model a person without too closely resembling any specific person. No precise target means that the elimination of known problems like resonances becomes the deciding factor, including realistic pinna compliance for accuracy in fit and so on.

The best thing that could be said about ASR adopting a 5128 as part of the standard set of gear is that with it almost the whole of the consumer audio industry will be covered. Assessing the quality of some new, exciting model when one comes out wouldn't be up to someone else. Real leadership in that sense. The downsides are obvious and practicality would counsel against it, but the ability to test claims and keep up seems more valuable than the idea that the best use of gear is as many measurements as possible for as low a cost as possible.

 

[Mad_Economist]

Yes absolutely, as I mentioned I make the same arguments myself so I am very well aware of the risk of self delusion here. I have also seen a lot of people say the olive response is too bassy.

I would be in that subgroup - listener class 3 in Olive, Welti, & Khonsaripour 2019. I prefer a milder rise at low frequencies, and past a certain level of boost, linear is definitely preferable.

Sorry I meant PM1, but yes I have the PM3 also.

Ah, unfortunate. You might want to ask Oratory or check his wiki regarding the pads used on the PM2 that he measured - the PM1 and 2 measured essentially identically in frequency response with the same earpads.

Its not quite that simple. Yes everybodies HRTF is different, but they still prefer a flat response speaker. An IEM doesnt have the variation caused by the individual anatomy (well to a much smaller extent) so its essentially the response of the driver. Its a more "consistent" system. However with an enclosed system of the headphone over the ear you are going to get quite disparate effects dependant upon that anatomy. Which leads us back to the previous question of the difference between "listening to the room" and listening to the "closed system".

Unfortunately, it is not quite as you are making out. People do not hear the behavior of drivers, they hear whatever is at their eardrums, and their perception of that is in relation to the perceived acoustic source. I've lost the free link to the paper - it's somewhere on Hauptmikrofon.de that Google seems to find poorly - but Theile 1986 is a reasonable place to start in the headphone case.

 

[March Audio]

Unfortunately, it is not quite as you are making out. People do not hear the behavior of drivers, they hear whatever is at their eardrums, and their perception of that is in relation to the perceived acoustic source. I've lost the free link to the paper - it's somewhere on Hauptmikrofon.de that Google seems to find poorly - but Theile 1986 is a reasonable place to start in the headphone case.

Sorry, poor phrasing on my behalf, I inherently meant what is received at the eardrum. However that is obviously correlated to the output of the driver.

 

[Mad_Economist]

Sorry, poor phrasing on my behalf, I inherently meant what is received at the eardrum. However that is obviously correlated to the output of the driver.

Indeed, but it must also correlate to the HRTF of the listener, otherwise you get subjectively non-flat frequency response/"tone colour error" in Theile's strange preferred lexicon. This is the problem of binaural recordings mentioned above: the HRTF of the recording head and the listening head differ, and the subjective effect is worsened or ruined.

 

[March Audio]

Indeed, but it must also correlate to the HRTF of the listener, otherwise you get subjectively non-flat frequency response/"tone colour error" in Theile's strange preferred lexicon. This is the problem of binaural recordings mentioned above: the HRTF of the recording head and the listening head differ, and the subjective effect is worsened or ruined.

So *all* headphones will always sound coloured because they are not tuned to the individual, which is sort of where we came in

 

[pozz]

People do not hear the behavior of drivers, they hear whatever is at their eardrums.

+1

 

[Mad_Economist]

So *all* headphones will always sound coloured because they are not tuned to the individual

To some degree, yes - that's part of what I was rambling at length about to Blumlein up thread, the headphone playback case has some inherent flaws in fidelity that cannot be addressed without some fairly involved digital signal processing work.

Equally, within the flawed paradigm of headphones, we see very consistent hierarchies of preference, so it would appear that we are still far away from a world where inter-individual tone-colour-defect is the major thing that makes people prefer headphone A over headphone B - indeed, I dearly dream of that world.

 

[Blumlein 88]

We are thinking very much along the same lines and have very similar experience - except that we wouldnt agree on which can sounded right . However I would wager we have similar thoughts about which speaker sounded right.

This gets at the crux of the matter I think.

The other thing is the in your head sound. If you can't get an out of your head sound, then phones seemed destined to be an altogether different experience that can't link the two conditions together. If the Smyth realizer manages the trick, then despite its seemingly too high cost, it is onto something important and should be examined to see how it works.

 

[Mad_Economist]

The other thing is the in your head sound. If you can't get an out of your head sound, then phones seemed destined to be an altogether different experience that can't link the two conditions together. If the Smyth realizer manages the trick, then despite its seemingly too high cost, it is onto something important and should be examined to see how it works.

The way it works isn't overly complicated so much as it's just sort of a pain to implement. The Realizers come with a pair of small insert microphones

which are inserted by the intended listener, and used to record a set of head-related impulse responses (HRIRs) in the intended listening room. The listener records the impulse response of the playback headphones with the same microphones, and mounts the head tracking fixture to them. The Realizer's DSP then applies the difference of the HRTF for the relative direction of the listener's head and the intended source to the measured on-head headphone response. Because the head tracking allows for this to be continuously updated with head movement, and because the measurements are individual to the listener-loudspeaker-room and listener-headphone systems, the effect is quite convincing.

Impulcifier is a somewhat humbler project, to my understanding of it, and somewhat inhibited due to the lack of suitable intraaural microphones available commercially (most "binaural" mics protrude past the omnidirectional section of the ear, so the transfer functions they capture won't produce the ideal effect), but it's also not $4000, so, hey.