ASR Headphone Testing and BK 5128 Hats Measurement System

[solderdude]

so an HD600 on two different heads will have two different frequency responses.

Also we would have to consider production spread and changes over time. Look at the plots from Sonarworks, Beowulf (Hrodulf) worked there for a while and posted Sonarworks info on SBAF:
https://www.superbestaudiofriends.org/index.php?threads/solving-the-hd650-mystery.8026/

I found a similar thing but with a much smaller sample size. 3 different HD650. Do note the dB scale as it is different from ones I usually make

So apart from different measurement methods showing different results (with correction) you may get different plots depending on the 'version' you have.

Also note Sonarworks and my plots have different 'corrections' and also are both flatplates.
However, before folks starts shooting at the necessity to have a pinna otherwise these plots are worthless. Do note that this post is merely to indicate spread in production also exists and what the magnitude is for HD650. As these headphones are all built exactly the same the absolute accuracy is not important at all as the goal is comparison between the same models and show unit to unit variations on the same rig under the same circumstances.

 

[Tks]

This is the #1 reason I may not get into headphone testing. So we need to identify the greater good against this onslaught of grief that is guaranteed to come with it.

Honestly, if I were you, I'd make this nearly one of the top reasons to now get into such testing.

 

[Tks]

I fear you may not find very much that is interesting in terms of distortion characteristics - "typically" headphones have very low distortion, but even some relatively highly nonlinear models are quite subjectively acclaimed. There's an exception here when you get into driver compression/"clipping" at the low end, of course, but the correlation is far from robust.

Incidentally @amirm, it does look like Sean has a target response for you if you'd like it:

https://twitter.com/i/web/status/1292218883785363458

As an aside, I recall AP has added coherence measurements to their functionality list - any plans for measuring headphones with music?

Tbh, FR and Distortion metrics are mainly what I'd want form headphones/IEMs (and perhaps tangential things like comfort if possible). With distortion in the .01% realm, and a spread across 20Hz to 20kHz that is fairly consistent, I feel comfortable I have enough tools to then EQ the thing to whatever preference, and not be caught off guard by anything too crazy without the tools to validate what the EQ did to any potential blowback from the distortion side of the operation.

Naturally I set my desires for these two basic things, simply because everything else you've been talking about along with the other members like John about the history of measurements and technicalities/discoveries over the years goes WAY over my head. You should get Amir to rename you to Mad_Scientist, as it seems your knowledge on these matters is quite wide and comprehensive (as is the material to posses as reference). Real pleasure reading since you started posting.

 

[Mad_Economist]

Tbh, FR and Distortion metrics are mainly what I'd want form headphones/IEMs (and perhaps tangential things like comfort if possible). With distortion in the .01% realm, and a spread across 20Hz to 20kHz that is fairly consistent, I feel comfortable I have enough tools to then EQ the thing to whatever preference, and not be caught off guard by anything too crazy without the tools to validate what the EQ did to any potential blowback from the distortion side of the operation.

Naturally I set my desires for these two basic things, simply because everything else you've been talking about along with the other members like John about the history of measurements and technicalities/discoveries over the years goes WAY over my head. You should get Amir to rename you to Mad_Scientist, as it seems your knowledge on these matters is quite wide and comprehensive (as is the material to posses as reference). Real pleasure reading since you started posting.

Thanks! I was kind of worried that I was overly responding to this thread, but headphone metrology is, like, my thing, and there's a lot of really handy information out there that isn't widely used, so it's hard to resist spamming a bit

Vis-a-vis distortion, the really key question there is the playback level. Some headphones (e.g. much of Audeze's lineup) will be essentially insignificant in their nonlinearities out to insane volumes - 110dBSPL or higher, for example. But with as humble of maximum output levels as are used in Amir's speaker tests, I'd be surprised to see many competently engineered headphones fall down - designing headphones, I often regard 90dBSPL as a "low" level from a standpoint of making sure that transducers are behaving themselves, and a look around Tyll Hertsen's database (if you can find it) will generally show that for many headphones, the distortion remained below the noise floor of his measurements out to 100dB at most frequencies.

 

[Tks]

Thanks! I was kind of worried that I was overly responding to this thread, but headphone metrology is, like, my thing, and there's a lot of really handy information out there that isn't widely used, so it's hard to resist spamming a bit

Vis-a-vis distortion, the really key question there is the playback level. Some headphones (e.g. much of Audeze's lineup) will be essentially insignificant in their nonlinearities out to insane volumes - 110dBSPL or higher, for example. But with as humble of maximum output levels as are used in Amir's speaker tests, I'd be surprised to see many competently engineered headphones fall down - designing headphones, I often regard 90dBSPL as a "low" level from a standpoint of making sure that transducers are behaving themselves, and a look around Tyll Hertsen's database (if you can find it) will generally show that for many headphones, the distortion remained below the noise floor of his measurements out to 100dB at most frequencies.

LCD2C's are great speaking first hand. And noticed no distortion at all. HD800S' on the other hand didnt sit right as I tried to bump up the sub bass. IEMs as a whole seemingly neber distort in my experience which is good. And the only times I noticed thr HD800S slip, was when I was listening at deafening levels pretty much.

Btw have you designed any phones yourself. You seem the type id want working at every manufacturer for validation and product inception.

 

[Fluffy]

LCD2C's are great speaking first hand. And noticed no distortion at all.

Second that. These cans just effortlessly go louder and louder without problems. DIY-Audio-Heaven specs their max SPL at 137 db (at 5W of power!) and I do not doubt this at all.

 

[Robbo99999]

The Harman room's acoustic characteristics are published in the AES, so that wouldn't be hard to come by. I could knock together a little spreadsheet for this purpose, I suppose - it'd be really messy though, I'm terrible at Excel.

Ideally I'd really need the B&K data rather than just scraping from the plots - we're going to need to do enough interpolation on the directivity and RT60 as is...

Regarding the bolded section, since I keep seeing this concept in this thread, I'd like to specifically disclaim the idea that headphone frequency response/headphone transfer functions measured on a HATS are necessarily "one size fits none" (e.g. that the frequency response you get with one HATS will be predictive of subjective timbre only insofar as your HRTF matches the HATS'). The ear plays a meaningful role in HpTF, so an HD600 on two different heads will have two different frequency responses.

Well a spreadsheet sounds great, and maybe a few words about what "weighting" you use to combine all the elements, so that we don't have to trawl through the spreadsheet to work out the most important relationships. Yeah, I mean I would welcome that massively if you provide a spreadsheet to us for the purpose of the calclulation, and I'm sure others would think that's a great idea too. So thankyou if you do this indeed!

About needing the raw data for the graphs in this post from Amir: https://www.audiosciencereview.com/...128-hats-measurement-system.15147/post-479999. @amirm , could you post up a file containing the raw data that describes those plots that you posted (the ones at the link in this paragraph)? It's needed as part of the process to create a Target Response Curve using the "Paul Struck Method".

About the bit I put in bold in your post - isn't that what I was saying? I was referring to closeness of the HRTF dummy head to an individual's own HRTF to be an important factor in how applicable & accurate any headphone EQ will be for that individual. When I say HRTF I understand that to mean all affects of the head shape influencing incoming sound from speakers, as well as outer ear shape, and also the shape of the ear canal. So I'm a bit confused about the distinctions you're making in your last paragraph there?

EDIT: thinking about my last paragraph whilst cooking dinner, I think I know what you mean now....I didn't know what HpTF means, but I'm guessing that means Headphone Transfer Function. So, I think you're saying that two people could have very similar HRTF plots but have massively different HpTF plots due to the way a headphone reacts with small differences on the outer ear that wouldn't have such a great effect when it comes to listening to speakers in a room. So you were pointing out that in terms of terminology we should be referring mainly to HpTF differences to the dummy rather than HRTF when describing how accurately a headphone EQ (created from the dummy head measurements) might sound to any given individual in the population......and you say this because you think HRTF is not as different amoungst the population as HpTF is. Is that about right?

 

[Mad_Economist]

LCD2C's are great speaking first hand. And noticed no distortion at all. HD800S' on the other hand didnt sit right as I tried to bump up the sub bass. IEMs as a whole seemingly neber distort in my experience which is good. And the only times I noticed thr HD800S slip, was when I was listening at deafening levels pretty much.

Btw have you designed any phones yourself. You seem the type id want working at every manufacturer for validation and product inception.

I'm presently working on my first entirely-designed-by-me headphone, as a matter of fact! I had expected to already have it on the market by now, but the COVID logistics issues and the woes of building planar magnetic drivers from scratch have set me back a fair bit...

Well a spreadsheet sounds great, and maybe a few words about what "weighting" you use to combine all the elements, so that we don't have to trawl through the spreadsheet to work out the most important relationships. Yeah, I mean I would welcome that massively if you provide a spreadsheet to us for the purpose of the calclulation, and I'm sure others would think that's a great idea too. So thankyou if you do this indeed!

Well, my current version of the sheet is a complete trash fire, but I make no pretensions of being good at Excel in the slightest, so I'll attach it.

The Hopkins-Stryker equation gives us the actual sound pressure level at a given frequency as a sum of the direct and reverberant sound. It's application in Chris's method, as best I understand it, is to provide a weight in an average between two inputs: an HRTF representing the "direct sound" (this being a free field HRTF, 30 degrees would likely be best but I used 0 since it's what I had), and an HRTF representing the reverberant sound (this being a diffuse field HRTF). The other variables - speaker directivity as a function of frequency, room reverberation time as a function of frequency, room volume, source distance - serve to determine the ratio of the direct and indirect HRTFs in the resulting average.

About needing the raw data for the graphs in this post from Amir: https://www.audiosciencereview.com/...128-hats-measurement-system.15147/post-479999. @amirm , could you post up a file containing the raw data that describes those plots that you posted (the ones at the link in this paragraph)?

I suspect Amir got those plots from the manual, in which case they will be generic - but yes, if @amirm could get a copy of the unit-specific HRTF data posted, that'd be great! It may only be in disk/USB drive form with the unit that's stranded on the Fedex truck, however, so we may need to wait.

About the bit I put in bold in your post - isn't that what I was saying? I was referring to closeness of the HRTF dummy head to an individual's own HRTF to be an important factor in how applicable & accurate any headphone EQ will be for that individual. When I say HRTF I understand that to mean all affects of the head shape influencing incoming sound from speakers, as well as outer ear shape, and also the shape of the ear canal. So I'm a bit confused about the distinctions you're making in your last paragraph there?

EDIT: thinking about my last paragraph whilst cooking dinner, I think I know what you mean now....I didn't know what HpTF means, but I'm guessing that means Headphone Transfer Function. So, I think you're saying that two people could have very similar HRTF plots but have massively different HpTF plots due to the way a headphone reacts with small differences on the outer ear that wouldn't have such a great effect when it comes to listening to speakers in a room. So you were pointing out that in terms of terminology we should be referring mainly to HpTF differences to the dummy rather than HRTF when describing how accurately a headphone EQ (created from the dummy head measurements) might sound to any given individual in the population......and you say this because you think HRTF is not as different amoungst the population as HpTF is. Is that about right?

Well, your interpretation is a reasonable one, but the argument I was making is actually the converse: Since subjective FR is equal to on-head HpTF (which is indeed headphone transfer function) minus individual HRTF (which HRTF being set aside for the moment), the only case where varying subjective FR will be significant is if HRTFs vary in ways that exceed or differ from the ways HpTFs vary.

In some cases, this is incontrovertibly true - an IEM can't tell what the volume of my concha bowl is, or how my pinna is shaped, so one could reasonably expect a spread of subjective frequency response at higher frequencies for headphones that bypass much of the ear anatomy. I'm arguing, however, against the premise that this is necessarily the case - I've measured headphones on nigh on a half dozen ears at this point, and while unit-ear specific interactions do exist, generally speaking the differences between any two headphones measured on any two ears have parallels; that is, the HRTF and HpTF are linked. Which, I mean...our ears are still a part of it for circumaural headphones, after all!

 

[Robbo99999]

I'm presently working on my first entirely-designed-by-me headphone, as a matter of fact! I had expected to already have it on the market by now, but the COVID logistics issues and the woes of building planar magnetic drivers from scratch have set me back a fair bit...


Well, my current version of the sheet is a complete trash fire, but I make no pretensions of being good at Excel in the slightest, so I'll attach it.

The Hopkins-Stryker equation gives us the actual sound pressure level at a given frequency as a sum of the direct and reverberant sound. It's application in Chris's method, as best I understand it, is to provide a weight in an average between two inputs: an HRTF representing the "direct sound" (this being a free field HRTF, 30 degrees would likely be best but I used 0 since it's what I had), and an HRTF representing the reverberant sound (this being a diffuse field HRTF). The other variables - speaker directivity as a function of frequency, room reverberation time as a function of frequency, room volume, source distance - serve to determine the ratio of the direct and indirect HRTFs in the resulting average.



I suspect Amir got those plots from the manual, in which case they will be generic - but yes, if @amirm could get a copy of the unit-specific HRTF data posted, that'd be great! It may only be in disk/USB drive form with the unit that's stranded on the Fedex truck, however, so we may need to wait.




Well, your interpretation is a reasonable one, but the argument I was making is actually the converse: Since subjective FR is equal to on-head HpTF (which is indeed headphone transfer function) minus individual HRTF (which HRTF being set aside for the moment), the only case where varying subjective FR will be significant is if HRTFs vary in ways that exceed or differ from the ways HpTFs vary.

In some cases, this is incontrovertibly true - an IEM can't tell what the volume of my concha bowl is, or how my pinna is shaped, so one could reasonably expect a spread of subjective frequency response at higher frequencies for headphones that bypass much of the ear anatomy. I'm arguing, however, against the premise that this is necessarily the case - I've measured headphones on nigh on a half dozen ears at this point, and while unit-ear specific interactions do exist, generally speaking the differences between any two headphones measured on any two ears have parallels; that is, the HRTF and HpTF are linked. Which, I mean...our ears are still a part of it for circumaural headphones, after all!

Awesome, thanks for creating that spreadsheet & attaching it! Regarding the bits I put in bold in your post.....it's late now so I'm gonna get some sleep, think about it tomorrow and hopefully ask you some intelligent questions to aid my understanding....unless it turns out I end up understanding it tomorrow after thinking about it! I'll also take a look at your spreadsheet in more detail tomorrow when my brain is awake!

It's great to have you here in this forum taking part in this project because you have loads of directly relevant knowledge & experience, I certainly appreciate it and I'm sure others do too.....and impressive re designing your own headphone, congratulations on that big time, plus I hope it can get restarted for you once COVID situation allows better progress.

 

[amirm]

At long last, the unit arrived! Spent the last couple of hours trying to reformat the calibration files to something AP software can accept.

Here is the raw response of the HD-650:

It seems to generally agree with other measurements out there until about 5 kHz or so. How trustworthy the rest is, is the question.

I took the free-field correction from 5128 (0 degree elevation and azimuth) which looks like this:

I inverted it, used it as EQ and remeasured to get the green curve:

That peak around 9 kHz is not like anything shown with other gear. There may be pilot error but this is what I have in the first 2 hours of messing with it.

Comments welcome.

 

[amirm]

I suspect Amir got those plots from the manual, in which case they will be generic - but yes, if @amirm could get a copy of the unit-specific HRTF data posted, that'd be great! It may only be in disk/USB drive form with the unit that's stranded on the Fedex truck, however, so we may need to wait.

I have them but I am not sure if they are supposed to be confidential or not. Will ask.

 

[solderdude]

At long last, the unit arrived! Spent the last couple of hours trying to reformat the calibration files to something AP software can accept.

Here is the raw response of the HD-650:

View attachment 77842

It seems to generally agree with other measurements out there until about 5 kHz or so. How trustworthy the rest is, is the question.

I took the free-field correction from 5128 (0 degree elevation and azimuth) which looks like this:

View attachment 77844

I inverted it, used it as EQ and remeasured to get the green curve:

View attachment 77843

That peak around 9 kHz is not like anything shown with other gear. There may be pilot error but this is what I have in the first 2 hours of messing with it.

Comments welcome.

Doesn't look right to me and FF is not the correct compensation anyway.
Perhaps a DF is a better match or 'mostly' DF and a little bit FF will give better results.

The 7.5dB peak at 9kHz certainly does not come from the driver. The 25dB dip at 15kHz also isn't there in reality with an absolute certainty.

Hoping you can find a decent correction plot.

Rtings also shows such huge dips but stop trusting the measurements (with good reason) above 10kHz.
This plot shows raw measurements of Rtings and shows why above 10kHz isn't accurate and this will not be very different from this rig.
Funnily enough the GRAS Jude is using shows a 15dB dip at 9kHz instead of a peak. There is >20dB delta between these measurements.

Ofcourse my cheap pinna and earcanal less totally incorrect and not trustworthy measurements of what comes from the driver tells a different story.

This is exactly how I hear it and when sweeping, alas my hearing tops out well before my mic does , I don't hear any peaks or dips at around 10kHz. Do note this is pinna less so I would expect that the area between 1 and 6kHz in reality could be perceived even higher than measured (I am quite certain it doesn't) but certainly not lower. When plots show the FR to be lower there it must because of over-compensation which will surely be the case with FF.

There is more pinna activation with sounds coming from the front than with headphones so that's why FF correction is way off in the area where the pinna makes the most difference. (between 1kHz and 6kHz )

Best bet would be a mix of DF and FF by lack of actual headphone-suited correction (like Olive's)

The raw measurements seem plausible. Would be interesting to see overlaid a bunch of measurements of the same HD650 but moved forward, backwards etc. (as far as the pinna allows) to get a feel of the magnitude of errrors (like Tyll, Rtings and Oratory also does)
When this is done with FP one only sees more treble roll-off but not wildly varying responses.
I could make a plot of this as well for comparison. Most measurement extremists will say that is pointless so will only poor time in this when this is found interesting as the method is flawed but IMO more accurate above 6kHz.

 

[amirm]

Rtings also shows such huge dips but stop trusting the measurements (with good reason) above 10kHz.
This plot shows raw measurements of Rtings and shows why above 10kHz isn't accurate and this will not be very different from this rig.
Funnily enough the GRAS Jude is using shows a 15dB dip at 9kHz instead of a peak. There is >20dB delta between these measurements.

Thanks. I found a few of those. It is interesting that there is no standardization on scales or aspect ratios. This makes them hard to compare.

I will try the diffused field next. The one I looked at is much more gentle and doesn't have the big troughs.

On the raw measurements, should we not get agreement to a few Kilohertz with other HATS???

 

[solderdude]

Thanks. I found a few of those. It is interesting that there is no standardization on scales or aspect ratios. This makes them hard to compare.

I will try the diffused field next. The one I looked at is much more gentle and doesn't have the big troughs.

On the raw measurements, should we not get agreement to a few Kilohertz with other HATS???

There seems to be a standard (REW can generate) but not many seem to use it. I won't as I have the same dilemma as Tyll had... revisit all measurements and update all info.
Should you decide to start measuring I would suggest that standard.
Before I made my own calibration file I took all plots of HD650 I could find, used paint to stretch all plots to the same scale (log scale sometimes was 'off' in some cases) and overlaid to find what others measured (weighted more towards official measurements) and corrected (below 1kHz) for what I measured 'raw' and 'baked' that correction in a hardware pre-amp for my rig.
Then checked with nearfield monitor if this was close to how I heard it.

Most likely DF will give more truthful results.

Yes, raw untill 5kHz should be pretty similar.
Measuring and correlating is interesting. Too bad I don't want to throw money on it and more time than I can spare now.

 

[amirm]

Should you decide to start measuring I would suggest that standard.

Oh, I definitely will. That is what I do in my speaker measurements. And followed the same 50 dB scale in above graphs.

 

[Blumlein 88]

There seems to be a standard (REW can generate) but not many seem to use it. I won't as I have the same dilemma as Tyll had... revisit all measurements and update all info.
Should you decide to start measuring I would suggest that standard.
Before I made my own calibration file I took all plots of HD650 I could find, used paint to stretch all plots to the same scale (log scale sometimes was 'off' in some cases) and overlaid to find what others measured (weighted more towards official measurements) and corrected (below 1kHz) for what I measured 'raw' and 'baked' that correction in a hardware pre-amp for my rig.
Then checked with nearfield monitor if this was close to how I heard it.

Most likely DF will give more truthful results.

Yes, raw untill 5kHz should be pretty similar.
Measuring and correlating is interesting. Too bad I don't want to throw money on it and more time than I can spare now.

web plot digitizer probably works better than stretching graphs in paint. It will turn graphs you have into results you can drop into a spreadsheet as a csv file. It is surprisingly precise.

https://automeris.io/WebPlotDigitizer/

 

[thewas]

An interesting presentation about headphone measurements

From Rocky Mountain International Audio Fest
Moderator: Jude Mansilla, Head-Fi.org
Panelists: Dan Foley, Audio Precision; Jacob Soendergaard, G.R.A.S. Sound & Vibration

The grouping of international standards - IEC 60318-x - specifies the electro acoustic equipment for testing headphones, earphones, hearing protectors, hearing aids etc. This has been the benchmark of audio testing for years, but as markets and products evolve, the need for test equipment to evolve in tandem is increasingly urgent. G.R.A.S. is addressing this and proposing a new set of headphone measurement equipment. RMAF/CanJam attendees will be the first to get a sneak preview of the new pinnae, ear canal and inner ear simulator – the next generation replica of the human auditory system.

 

[solderdude]

Hmm the video is 5 years old already, even Tyll was starring in it. A lot has changed since that video came out.
Is the new HATS Amir is testing what they were talking about ?
Absolute accuracy and target curves sorted yet ?

 

[MZKM]

At long last, the unit arrived! Spent the last couple of hours trying to reformat the calibration files to something AP software can accept.

Here is the raw response of the HD-650:

View attachment 77842

It seems to generally agree with other measurements out there until about 5 kHz or so. How trustworthy the rest is, is the question.

I took the free-field correction from 5128 (0 degree elevation and azimuth) which looks like this:

View attachment 77844

I inverted it, used it as EQ and remeasured to get the green curve:

View attachment 77843

That peak around 9 kHz is not like anything shown with other gear. There may be pilot error but this is what I have in the first 2 hours of messing with it.

Comments welcome.

Comparison:


InnerFidelity

ClarityFidelity

RTINGS, left ear

RTINGS, right ear

Crinacle:

Oratory1990:

Photopea link if you want to play around with them:
https://www.photopea.com/#iABP8MFRg

Crinacle & RTINGS' right ear are very similar <3kHz

InnerFidelity & ClarityFidelity are very similar <4kHz

ASR & InnerFidelity are very similar between 70Hz and 4kHz
ASR & ClarityFidelity are very similar between 70Hz and 4kHz
ASR & Oratory1990 are very similar between 50Hz and 3kHz

That peak around 9 kHz is not like anything shown with other gear.

RTINGS' shows a 9kHz peak in both ears, with the right ear having a similar amplitude.

What ear did you measure (or is this the average?)? Is this a single measurement or an average?

Showing each ear separately and having at least 5 measurements of slightly different positions (including tilting the headband) should be done. And what I like that RTINGS does is that one/few of the measurements are with glasses being worn.

__________

Your horizontal scale is different between 10kHz & 20kHz, whereas all the others are pretty much the same.

 

[thewas]

Hmm the video is 5 years old already, even Tyll was starring in it. A lot has changed since that video came out.

Sure, but I think its still a good overview to the headphone measurements topic and issues.