ASR Headphone Testing and BK 5128 Hats Measurement System

[Mad_Economist]

With headphones sound is coming from the side, totally different result so not usable.

Headphone sound is not approximate to ipsilateral incident sound - either in acoustics, or perception. The small, closed volume of earpads means that for much of the audio band, the driver is directly coupled to the eardrum, and above that you move into something more analogous to the modal range of a listening room - only at the very highest frequencies do you see something like conventional room sound propagation in headphones.

Subjectively, headphone sound is also not localized to the sides outside the head - if it were, we would require radically different target curves to achieve accurate timbre, and they would be very bright ones indeed by the standards of what we use presently. Rather, headphone sound is "in-head localized", outside of edge cases like head tracking+hrtf synthesis or well-matched HRTF binaural recordings.

The challenge for HATS users is to create (I assume HATS manufacturers by now will provide a correction file for this use case) an 'average' correction for headphones.
[...]
The problem is there is no 'reference' headphone like there are reference speakers. Large, small drivers, angled or not, pads etc. differ from headphone to headphone so that's the biggest issue.

Outside of the Harman target, there is no per se "headphone target" compensation for HATS, just the HRTFs of the HATS under different circumstances. However, this is ultimately neither more or less reasonable than the situation with speakers - after all, a speaker that measures flat with an omni mic in an anechoic room will measure like the free field HRTF of the HATS if you used that to measure it in said room. The "references" we have are either grounded in tests of subjective preference (e.g. the Toole and Olive experiments in speaker preference) or simply referenced to sound pressure under some acoustic circumstance, likely in an anechoic chamber or a facsimile thereof.

I'd generally argue that, to translate the headphone case to the world of speaker-philes, it's best to consider all measurements to be "in-room" - there is no analogue to the anechoic free field condition for headphones, because headphones are inherently designed as a system with the head and ear, and their behavior depends on that pairing. This is basically congruent with Olive's research, which indicated that people want, generally, about the same response at the eardrum from speakers and headphones (with a tad more bass and treble from speakers).

That would be the room correction. The reason for adding that to the correction is because speakers in a room that radiate a 'flat' response from their position(s) in the room will be changed due to room modes, reflections and distance (small effect) to the listener.
That 'change' in general will be a downward slope from higher frequencies and some 'boost' of lower frequencies.

Please note that the chain of causation doesn't necessarily go this way - "good" speakers that are anechoically flat do slope downwards in real rooms, but it's not clear to me that it isn't this aspect that makes them "good". Part of Olive, Welti, & McMullin 2013 including equalizing a pair of Revel speakers flat, in room, at the listening position (with an omni mic), then measuring them with a 43AG and equalizing a set of headphones to the same response. When users were then allowed to adjust the response, they applied similar adjustments to both - and these adjustments do resemble the way that the Revels naturally slope in-room, but my argument would be that people tend to like the directivity characteristics which produce this in-room response in part because it results in this power response...

 

[Robbo99999]

They can be used as target curves, but ultimately they are simply measurements of the eardrum response of the head under certain acoustic conditions. The 0 elevation, 0 azimuth free field HRTF, if applied as compensation like a mic calibration file, will result in a flat line from a frontally located sound source with a flat frequency response, in an anechoic chamber or other situation where there are no other incidences for sound.

Yes, cool, that's how I understand it now based on Jimbob's post earlier, however I don't like to think it of a mic compensation file because that complicates it in my mind, I'd rather visualise it as the actual measured response at the mic combined with the actual target that's being aimed for at the mic, along the lines of the the type of plots seen in these Oratory1990 plots for instance, the portion that I circled in the pic below, which is why I find Solderdude's posts so confusing because he sees it from a "mic compensation file" perspective which I find deeply unhelpful & unintuitive:

1: All standards-compliant (the pertinent ones being ITU-T P57/58, IEC60318-4/7) mannequins and ear simulators have transfer functions that fall within certain tolerance bands given in the pertinent standards. Independent measurements of mannequin HRTF show that the commercially available models generally conform to these specifications, although Head Acoustics occasionally pushes the limits a bit. Thus while the transfer functions of the mannequins or ear simulators may differ somewhat - and model and unit-specific compensation data is preferable for this reason - there are limits to the range of variation. You cannot have an apples-to-apples comparison of uncompensated KEMAR data to uncompensated 5128 data, but the same data compensated to each unit's diffuse field HRTF should be comparable, and without compensation it should still be "close".

I bolded that section in your post.....yeah cool I can visualise why that would be so.

2: There are two targets in broad use for compensated headphone response: The first is diffuse field (drawn, as said, either from population averages, or the specific mannequin in question's HRTF in a diffuse field), which is a well defined sound field that should be possible to approximate (for example via the free field summation method, see Hammershoi & Moller 1995) by pretty much anyone willing to put in the work with any anthropomorphic system (and which most manufacturers will supply compensation data for).

The second is the Harman target, which is based on a set of in-room measurements within Harman's listening room, conducted with a 43AG set into a mannequin head, adjusted by a pair of shelf filters that a large population of testers were allowed to adjust to their preferences. The "baseline" of the Harman response is highly smoothed due to its origins as in-room unwindowed measurement, so the 43AG/KEMAR pinna specific HRTF elements it contains are fairly smoothed down, but applying it directly to data from another HATS would be improper. I generally champion the premise of either directly applying the Harman shelf filters to the DF-HRTF of the HATS in question, or if one feels particularly motivated, technically something even closer could be synthesized via Chris Struck's target methodology given that the acoustic parameters of the Harman room and the directivity of the Revels used in the Olive-Welti 2013 paper are known.

The first bit of your post that I put in bold is what I was suggesting as an idea in my previous post, so that makes sense to me. I'm gonna have to get some sleep and then have a look at your Chris Struck's target methodology.

Broadly, if you want something that's got a well established statistical correlation with subjective perception, a GRAS unit and the direct use of the Harman target is the best validated approach - this would be the closest analogy to the speaker preference calculations already featured on this site. However, as said, Sean may be able to provide compensation data for the 5128, as he has had one for a while, and I see little reason to believe that the statistical robustness of the GRAS derived results would be absent with the 5128 (bar the aforementioned in-ear issues).

Ah, cool, so essentially Sean can provide us with a modified Harman Curve Target specific for the B&K 5128 that we're getting (again you see I don't like talking about compensation files because I find them confusing & less intuitive, so I'm talking about the actual target measured frequency response at the mic). I also intuitively believe that Harman is the way to go given the research & my experience with it's EQ's, therefore I think if our B&K 5128 can be molded to fit that then I think that is ideal! Hell, you could also try it against the other target curves (so to speak) that come with the B&K out of interest.
Did Sean say when he would provide the compensation file?

 

[Mad_Economist]

Yes, cool, that's how I understand it now based on Jimbob's post earlier, however I don't like to think it of a mic compensation file because that complicates it in my mind, I'd rather visualise it as the actual measured response at the mic combined with the actual target that's being aimed for at the mic, along the lines of the the type of plots seen in these Oratory1990 plots for instance, the portion that I circled in the pic below, which is why I find Solderdude's posts so confusing because he sees it from a "mic compensation file" perspective which I find deeply unhelpful & unintuitive:
View attachment 77179


I bolded that section in your post.....yeah cool I can visualise why that would be so.


The first bit of your post that I put in bold is what I was suggesting as an idea in my previous post, so that makes sense to me. I'm gonna have to get some sleep and then have a look at your Chris Struck's target methodology.


Ah, cool, so essentially Sean can provide us with a modified Harman Curve Target specific for the B&K 5128 that we're getting (again you see I don't like talking about compensation files because I find them confusing & less intuitive, so I'm talking about the actual target measured frequency response at the mic). I also intuitively believe that Harman is the way to go given the research & my experience with it's EQ's, therefore I think if our B&K 5128 can be molded to fit that then I think that is ideal! Hell, you could also try it against the other target curves (so to speak) that come with the B&K out of interest.

I can see where you're coming from regarding compensation files and nomenclature - "target response" is really the best description. Of course, with HRTF stuff, our brains actually do something analogous to subtracting a mic compensation file in terms of our subjective timbre/FR, so things can be argued many ways...

The TL;DR of Chris Struck's approach is a way to synthesize a "combined" diffuse and free field HRTF based on the acoustics of a room and the speakers used in it; essentially a way to try to approximate an eardrum response in-room, but using higher quality anechoic data and thus forgoing the substantial smoothing necessitated by Olive and Welti's approach.

Well, I believe Sean might be able to, but I'm suggesting reaching out to him - he's responsive on Twitter to inquiries, so perhaps Amir might want to drop him a line? This is the last I heard from Sam Vafaei on the subject, so I'm out of date

 

[Robbo99999]

I can see where you're coming from regarding compensation files and nomenclature - "target response" is really the best description. Of course, with HRTF stuff, our brains actually do something analogous to subtracting a mic compensation file in terms of our subjective timbre/FR, so things can be argued many ways...

The TL;DR of Chris Struck's approach is a way to synthesize a "combined" diffuse and free field HRTF based on the acoustics of a room and the speakers used in it; essentially a way to try to approximate an eardrum response in-room, but using higher quality anechoic data and thus forgoing the substantial smoothing necessitated by Olive and Welti's approach.

Well, I believe Sean might be able to, but I'm suggesting reaching out to him - he's responsive on Twitter to inquiries, so perhaps Amir might want to drop him a line? This is the last I heard from Sam Vafaei on the subject, so I'm out of date View attachment 77181

Well I think reaching out to Sean for his calibration file aka modified Harman Target Response specific for the 5128 is an excellent idea. I think whoever knows him best with the most influence should be the person that contacts him.....I have no idea if that would be Amir or you or anybody else......but I think it's an excellent idea. Hopefully some relevant people could volunteer to contact him about it.

I'll read up about your Chris Struck's approach tomorrow so I can understand it.

 

[AdamW]

Despite what is available out there, people ask me that we do that kind of testing. There is style and substance to what we do that is of value to people. Now whether it is worth significant investment will depend on whether we can do a better job of measuring.

Just to clarify a bit: I picked the word "value" carefully. I do think you could possibly give us *better* measurements than existing sites, and there'd certainly be *interest* in any measurements you get. I just don't think that it's yet possible to measure headphones 'enough better' that it actually will make a change to how people buy headphones.

Let's put it this way - we have sites that give us, I dunno, say 5/10 quality measurements. That's good enough to get a vague idea of the basic character of a pair of headphones, but not good enough to declare "these are clearly Good Headphones" or "Bad Headphones", or for anyone to rush out and buy a pair based solely on the measurements.

Let's say you can buy this new thing and give us 7/10 quality measurements. That would be...still good enough to get an idea of the character, but not good enough to declare they're Good or Bad Headphones, or for anyone to safely go buy a pair based purely on the measurements People might *do* it, but it probably wouldn't be a good idea.

That was what I was driving at. Unless we can somehow get to, I dunno, 9/10 or 10/10 quality measurements, taking into account variation between individual physiology!, I don't think it'll change how anyone buys headphones much. Whereas accurate measurement of DACs and amps and speakers absolutely *has* changed how at least some people buy them.

 

[amirm]

Just to clarify a bit: I picked the word "value" carefully. I do think you could possibly give us *better* measurements than existing sites, and there'd certainly be *interest* in any measurements you get. I just don't think that it's yet possible to measure headphones 'enough better' that it actually will make a change to how people buy headphones.

Let's put it this way - we have sites that give us, I dunno, say 5/10 quality measurements. That's good enough to get a vague idea of the basic character of a pair of headphones, but not good enough to declare "these are clearly Good Headphones" or "Bad Headphones", or for anyone to rush out and buy a pair based solely on the measurements.

Let's say you can buy this new thing and give us 7/10 quality measurements. That would be...still good enough to get an idea of the character, but not good enough to declare they're Good or Bad Headphones, or for anyone to safely go buy a pair based purely on the measurements People might *do* it, but it probably wouldn't be a good idea.

That was what I was driving at. Unless we can somehow get to, I dunno, 9/10 or 10/10 quality measurements, taking into account variation between individual physiology!, I don't think it'll change how anyone buys headphones much. Whereas accurate measurement of DACs and amps and speakers absolutely *has* changed how at least some people buy them.

Again, we are back to saying science is helpless in directing us on what headphone we should buy. This is something I like to investigate first hand using this new HATS. I have likes and dislikes in the headphone set that I have. My hope is that I can figure out objectively which is better.

By the way, I expect frequency response to be only a part of this analysis. I hope to drill deeply into distortion characteristics and figure out its role. In speakers distortion easily causes brightness/sharpness that could be mistake for frequency response errors. It is quite non-intuitive to filter out the bass yet find that the sound becomes warmer and cleaner!

Is this going to happen? I don't know. I spent months studying this last year before deciding to not go the headphone route. I am hopeful that there may be more now than there was then. If there isn't, then we won't proceed of course.

I should give another motivation for this work. With the weather getting warmer, I can't wear my Ether CX closed back headphones to work and get away from fan noise of PC/instruments around me. So I switched to an IEM someone had sent me. Boy, this thing does not sound pleasant and again, it is not all tonality. There is got to be a way to quantify problems like this.

 

[Mad_Economist]

I fear you may not find very much that is interesting in terms of distortion characteristics - "typically" headphones have very low distortion, but even some relatively highly nonlinear models are quite subjectively acclaimed. There's an exception here when you get into driver compression/"clipping" at the low end, of course, but the correlation is far from robust.

Incidentally @amirm, it does look like Sean has a target response for you if you'd like it:

https://twitter.com/i/web/status/1292218883785363458

As an aside, I recall AP has added coherence measurements to their functionality list - any plans for measuring headphones with music?

 

[solderdude]

Headphone sound is not approximate to ipsilateral incident sound - either in acoustics, or perception. The small, closed volume of earpads means that for much of the audio band, the driver is directly coupled to the eardrum, and above that you move into something more analogous to the modal range of a listening room - only at the very highest frequencies do you see something like conventional room sound propagation in headphones.

That's why I mentioned that anything above 5kHz is rig dependent and one should be careful relying on that data to provide accurate EQ.
Below 5kHz there should be not much difference unless the pinnae differ too much. As you say the coupling is direct, yet the pinna does also have an amplitude changing effect above 1kHz so that would indicate that above 1kHz the coupling already isn't direct anymore.

Do you know at what frequency the coupling starts to become less 'direct' and what would be the reason for this. I mean when coupling is direct why would a pinna affect the incoming sound ?

Subjectively, headphone sound is also not localized to the sides outside the head - if it were, we would require radically different target curves to achieve accurate timbre, and they would be very bright ones indeed by the standards of what we use presently. Rather, headphone sound is "in-head localized", outside of edge cases like head tracking+hrtf synthesis or well-matched HRTF binaural recordings.

Not sure what you are getting at. I hear the mono part inside my head and on the better headphones the sounds that were panned right or left (or binaurally recorded) outside of my head. Just not to the front.

With things like headtracking the signal applied to the headphones is altered in amplitude and phase response to mimic the effect the pinna has because sounds from the front of us have different properties than sounds from the sides or as you put it are 'direct coupled'.
What's interesting about this (you will probably know this and is why I ask) is that headtracking thus should work equally well with IEMs.
I hate to ram things in my ear canal so never use IEMS.

The direct coupling of higher freq bypassing the influence the pinna has thus could/should result in better spatial sound when the correct HRTF of an individual would be mimic-ed. As not all HRTF are the same that may well be the reason it works better for some than others.
For instance I have never experienced sound coming from the front or around me using headphones. To me it is always L-R and inbetween regardless of the trickery/binaural recordings I use. Never tried headtracking though. I am fine with L-R, it is a compromise which I learned to accept over time to me.

Outside of the Harman target, there is no per se "headphone target" compensation for HATS, just the HRTFs of the HATS under different circumstances.

Aside from the rather unnatural bass boost (driven by the desire to boost bass, lets forget the reasons for this for the moment) and the downward sloping FR response which deviates just slightly from other room corrections that is added to the 'corrected' flat response, what makes the target response of Harman very different from 'flat' with other room responses. Is this the slightly different slope only ?

However, this is ultimately neither more or less reasonable than the situation with speakers - after all, a speaker that measures flat with an omni mic in an anechoic room will measure like the free field HRTF of the HATS if you used that to measure it in said room.

Of course it will. But not in your typical room. That is the reason to also apply 'room correction'. Agreed that Harman response comes closer to speakers in room than the response to mimic large concert halls or other derived room compensations. To me Golden Ears already used a 'similar-ish' bass boost in its plots.

I'd generally argue that, to translate the headphone case to the world of speaker-philes, it's best to consider all measurements to be "in-room" - there is no analogue to the anechoic free field condition for headphones, because headphones are inherently designed as a system with the head and ear, and their behavior depends on that pairing. This is basically congruent with Olive's research, which indicated that people want, generally, about the same response at the eardrum from speakers and headphones (with a tad more bass and treble from speakers).

Indeed... because of my technical background I thus 'pull apart' the correction for the measurement mic and the room correction. The advantage of looking at it this way is that one can (electrically) experiment with various types of room correction where the end result for the HATS + room correction becomes the total correction which some folks thing is 'the Harman' correction applicable to all HATS.
A lot of confusion is generated by the nontechnical minded folks about the differences between the Harman correction plot (for their HATS) and the Harman target.
A question you are likely to know. Is there a difference between the actual final correction correction (Harman in this case) between the overly smoothed 'target' shown in virtually all publications, as an example the one Robbo99999 is referencing to in the Oratory plot and the actual one used per HATS. I expect this too be the case and is why one needs a different Harman correction plot tailored to each specific HATS.
That's what the interesting part (to me) is all about and how did they determine this.

I will elaborate my question a bit. The room Harman determines its Harman correction in (taking their findings in consideration) will be essential to do make the calibration files in. I mean FF and DF will yield different results per HATS.
Is that room (its aspects) specified and formalized so that others can use that info as well or is it essential to place each specific HATS in the Harman listening room to obtain the final Harman target for that HATS.
I suppose the latter is the case ?

 

[amirm]

Is that room (its aspects) specified and formalized so that others can use that info as well or is it essential to place each specific HATS in the Harman listening room to obtain the final Harman target for that HATS.

It is an IEC compliant room and Sean has published a full paper on its construction and measurement compliance:
A New Reference Listening Room For
Consumer, Professional and Automotive
Audio Research

Here is a picture of it while I was there:

The projection screen of course is normally up and hidden.

 

[zenki]

I should give another motivation for this work. With the weather getting warmer, I can't wear my Ether CX closed back headphones to work and get away from fan noise of PC/instruments around me. So I switched to an IEM someone had sent me. Boy, this thing does not sound pleasant and again, it is not all tonality. There is got to be a way to quantify problems like this.

Which IEM?

 

[amirm]

Which IEM?

I don't remember. It is something a company sent me. Not a major brand.

 

[Rja4000]

Tried a lot of headphones at a hifi shop a while ago and the Grado were the best all-rounders price/quality wise.

I had 2 Grado: SR60 and SR80i.
I prefer the first one.
But they are fragile pieces !

 

[Robbo99999]

I fear you may not find very much that is interesting in terms of distortion characteristics - "typically" headphones have very low distortion, but even some relatively highly nonlinear models are quite subjectively acclaimed. There's an exception here when you get into driver compression/"clipping" at the low end, of course, but the correlation is far from robust.

Incidentally @amirm, it does look like Sean has a target response for you if you'd like it:

https://twitter.com/i/web/status/1292218883785363458

As an aside, I recall AP has added coherence measurements to their functionality list - any plans for measuring headphones with music?

Nice job Mad_Economist, that's the way to get things moving! (Sean) I think that dramatically increases the testing possibilities and validity of the 5128 that's gonna be on trial.

 

[JohnYang1997]

Using compensation file is certainly interesting to me. We will need to see the results. The result will definitely explain some things. And from what I know from Amir in this forum, if something doesn't work for him, he would not tolerate. It will definitely be interesting to see Amir's opinion about Harman's IE targets. I believe in this community, it's definitely different from other forums like headfi or reddit. We have a number of people who have their background and really know their stuff.

 

[JohnYang1997]

BTW why is Sean so fascinated about predicted score? I definitely don't like it. If any I prefer binary scoring system. Sound like shit or sound acceptable.

 

[Robbo99999]

The TL;DR of Chris Struck's approach is a way to synthesize a "combined" diffuse and free field HRTF based on the acoustics of a room and the speakers used in it; essentially a way to try to approximate an eardrum response in-room, but using higher quality anechoic data and thus forgoing the substantial smoothing necessitated by Olive and Welti's approach.

As promised I attempted to read about Chris Struck's approach using that link you gave (https://www.aes.org/e-lib/browse.cfm?elib=17041 ), but that was just to an Abstract of the paper, not viewable without purchase. However I did Google around the subject, and came across a mostly unrelated set of slides from him that are nonetheless interesting & informative so I thought I'd link them, I've skimmed them at this point rather than interpreted all of it.....don't look at this next link in detail (or at all), it's just a by-the-by point of interest mostly unrelated but sufficiently interesting to include: http://www.cjs-labs.com/sitebuildercontent/sitebuilderfiles/HeadphoneMeasurements-CJSLabs.pdf

But anyway, going back to the Chris Struck approach that you linked, from the small bits I've read is this essentially the same as the Harman Headphone Approach for calculating a Headphone Frequency Response Target except it's theoretically calculated rather than measured? And the only measurement it uses is the anechoic data of the speaker, all with a view to creating a Frequency Response Target for headphones that is to mimic any specific model (eg JBL 305p Mkii for example) of speaker that has had it's frequency response anechoically measured.....and then you add in theoretical room variables of your choice (to describe the room you wish to theoretically listen in) to combine with the supplied Diffuse & Free Field HRTF files from the HATS you are using to measure headphones......and that's how the Frequency Response Target is created? I'm making a lot of assumptions here because there's not a lot to read on this subject and I've not spent hours on it. I'm assuming we could also add in things like Low Shelf Bass Boosts on top of that to account for the Harman preference research. Do we have everything at our disposal to try the Chris Struck approach with the B&K 5128? We'd need access to the formulas though I imagine, in order to combine all of this to create the Target Frequency Response?

EDIT: as an aside, isn't this related to the Smyth Realiser system, albeit Chris Struck approach is using more theoretical measurements (dummy head HRTF & theoretical room & 'theoretical' anechoically measured speakers) rather than actual measurements?

 

[solderdude]

It is an IEC compliant room and Sean has published a full paper on its construction and measurement compliance:

Here is a picture of it while I was there:

View attachment 77252

In that case a reference file created by Olive would be a good reference as the same rig was used in that specific room.

This begs the next question.
How far 'off' is your listening room with respect to the standard room ?

I ask this in case you want to relate measured headphone response (when EQ'ed acc to the measured results) with how your speakers sound at the listening position.
When assuming the HATS is accurate it may still seem 'off' when the speaker/room does not adhere to Harman target.
That seems to me as a logical step to do at your home.

I usually EQ based on my own measurements (= crap acc. to some) and look at Rtings to compensate for missing pinna in the 1-5kHz range and then compare it to a known reference (my speakers in room or headphone with EQ I know works for me).
Would be nice if I can change to my own + your measurements once your measurement library is extended enough after a few years when you decide to pursue this.

 

[solderdude]

For the same source, same dac+amp (E1DA 9038S), what is it that makes my 6xx sound different from the IEM I bought for balanced 2.5 mm connection?

Is that when both are EQ'ed to the same target or without any correction ?

 

[Robbo99999]

Is that when both are EQ'ed to the same target or without any correction ?

(off topic much....)

 

[solderdude]

PSO seeked for an explanation for the perceived differences for which he would like to see Amirs (future?) measurements explain this.
Is my question a stupid thing to ask... if so why ?
The 2 very different headphones will measure/sound different from each other on any rig.
When they are EQ'ed to the same target (based on whatever is currently available) then one would expect the differences to be much smaller.
So... that's why I asked. It is not off-topic at all.
When PSO expects the differences he perceived are when EQ'ed to a specific target and yet they still sound different (which I would not find weird at all) and Amir's HATS + correction is closer to the reality PSO perceives (we don't hear exactly as HATS, but close to) then Amir doing a better job than others in this business makes sense.
not off topic at all.