• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required as is 20 years of participation in forums (not all true). Come here to have fun, be ready to be teased and not take online life too seriously. We now measure and review equipment for free! Click here for details.

ASR Headphone Testing and BK 5128 Hats Measurement System

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
28,211
Likes
72,884
Location
Seattle Area
#1
Hello everyone. I am out of my mind to be thinking about getting into headphone testing. But the itch exists and I am reminded of it often in private communication with people asking me why I am not testing headphones.

I had tabled the whole thing for many reasons, most of which is that I am not happy with state of the measurement systems out there for headphones.

There has been a development which may make the measurement situation better. It is the Brüel & Kjær 5128 HATS (head and torso simulator). Here are a couple of quick promotional video on it:



I like the flexibility of the artificial ear/pinna and better reliability and repeatability that this brings.

There are some big negatives however:

1. The 5128 extends the simulation limit of the older "711" standard substantially. But with it, it also makes the measurements non-standard so existing research may be difficult to apply to it.

2. The cost. Man, oh the cost. The full HATS has a retail cost of $41,000! There is a truncated one that is a bit cheaper (just the head and no torso). This is a stunning amount of money to spend to measure headphones.

I have asked BK to give me an evaluation unit to test. After all, I still don't know if this is a good solution or not. They have been kind enough to say Yes and the unit will arrive soon. I only have a few short days when I get it to test and then return it. Let me know what you think I should be measuring/doing with it.

Whatever you all do, don't mention a word about this to my wife! I honestly don't know how to go and tell her I want to spend $5,000 on this let alone nearly 10X that!

Anyway, any and all feedback is welcome including whether we should even bother doing this.
 

JohnYang1997

Major Contributor
Technical Expert
Manufacturer
Joined
Dec 28, 2018
Messages
4,309
Likes
6,455
Location
China
#4
I don't recommend it. The new one is better than GRAS's HIRES offering but not really better than the original 4128c. The high frequency range 6-10k+ gets really tricky. All new hires head reduces natural ear canal resonances which lead to false measurement results.
Current state of measurement equipment is a sad state.
If this is purchased in the end I won't be surprised. But I won't give hope to the accuracy of the measurement results. I also won't worry too much because at least it's better than GRAS's hires head.

Also get a dedicated bk 4195 or gras ra0045 for earphone measurements. It's essential.
 

Blumlein 88

Major Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
9,656
Likes
12,904
#7
Smart money is the world is mainly personal audio now and in the future. Hence why Samsung has Olive only working on headphones.

Now I'd say no.

Not my torso+not my head+not my pinna+not my ear canal=FAIL!

I don't think this averaging of factors adds up to a useful average. It will still be significantly wrong for most people above 5 khz.

I don't see the state of the art of headphones being anywhere close to as useful as the situation with loudspeakers from the work done by Harman, and devices like the Klippel.

I think one day someone will use impulse testing of individual ears (like done with newborns) to generate personal parameters. Then these will allow a personal calibration file for use with headphones well designed to provide real improvements in headphone fidelity. This whole improving the average approach is wrong-headed for the problem. There may be advances and plenty will be learned, but you'll never break the circle of confusion in headphones without individual measurement of the users torso, head, pinna, and ear canal. Averaging isn't getting you much.
 

solderdude

Major Contributor
Joined
Jul 21, 2018
Messages
6,230
Likes
11,444
Location
The Neverlands
#8
More tests on the already way too full plate ?

Consider that someone like Oratory1990 already tests acc. to a standard and publishes results.
There will certainly be measurable differences with the HATS oratory uses. Even the one that Tyll used simply because the correction was very wrong.
Rtings had their own correction but slowly bends towards standards. The measurements are kind of official. Here too basic specs are missing and not tested for.
What measurements to show ? Only FR and distortion ? Also would need to build a silent room and calibrated mics/speakers in that chamber to measure leakage and isolation. There is way more to it than just a HATS.

So for measurements themselves IF they need to be compliant to a standard I would say go for current standards and not something older.

The thing that is lacking from Oratory is basic specs and subjective evaluations. In any case neatly bundled together and easy to find in one place.
Also there are always simple things (or less simple) that can be done to improve headphones (and verify them).
Include that or only 'check' specs/measurements, make a sound description and call it a day.

So IF you really want to evaluate headphones as well you should test acc. to a standard (I don't have to) then like the AP should buy the best standard and adhere to it even when it seems 'flawed'. Then you would also have to add basic specs and check those. Also recommendations and descriptions should be there. The problem is will this be in technical talk or flowery talk. Log used recordings, SPL used etc. Only then you will do better than others.

Don't do half-baked things, there are plenty doing that already (like me) but at least I try to link measurements to what is being heard.

If it needs to be ASR 'expected' quality this is going to be very costly and time consuming as well. More backlog.
 
Last edited:

JohnYang1997

Major Contributor
Technical Expert
Manufacturer
Joined
Dec 28, 2018
Messages
4,309
Likes
6,455
Location
China
#9
More tests on the already way too full plate ?

Consider that someone like Oratory1990 already tests acc. to a standard and publishes results.
There will certainly be measurable differences with the HATS oratory uses. Even the one that Tyll used simply because the correction was very wrong.
Rtings had their own correction but slowly bends towards standards. The measurements are kind of official. Here too basic specs are missing and not tested for.
What measurements to show ? Only FR and distortion ? Also would need to build a silent room and calibrated mics/speakers in that chamber to measure leakage and isolation. There is way more to it than just a HATS.

So for measurements themselves IF they need to be compliant to a standard I would say go for current standards and not something older.

The thing that is lacking from Oratory is basic specs and subjective evaluations. In any case neatly bundled together and easy to find in one place.
Also there are always simple things (or less simple) that can be done to improve headphones (and verify them).
Include that or only 'check' specs/measurements, make a sound description and call it a day.

So IF you really want to evaluate headphones as well you should test acc. to a standard (I don't have to) then like the AP should buy the best standard and adhere to it even when it seems 'flawed'. Then you would also have to add basic specs and check those. Also recommendations and descriptions should be there. The problem is will this be in technical talk or flowery talk. Log used recordings, SPL used etc. Only then you will do better than others.

Don't do half-baked things, there are plenty doing that already (like me) but at least I try to link measurements to what is being heard.

If it needs to be ASR 'expected' quality this is going to be very costly and time consuming as well. More backlog.
I don't like Oratory. I even like Crinacle more.... And I also think measuring speakers will bring much more value to the field.
 

Robbo99999

Addicted to Fun and Learning
Forum Donor
Joined
Jan 23, 2020
Messages
868
Likes
514
Location
UK
#10
Very cool Amir, good that you're considering going into headphone measurement and applying the same rigour that you do with speaker testing! I agree though that headphones are harder to measure because we all have different HRTF's and different pinnas / ear canals and outer ears, all of which affects how we perceive sound from a headphone, so this is the extra layer of complication on top of speaker measurement.

I noticed you mentioned about it being a different measurement rig to a lot of the standards that are out there, e.g. the GRAS unit that Oratory1990 uses for instance, and therefore the standard Harman Headphone Curve might not be applicable to it due to the physical reasons mentioned in my first paragraph above. As an idea if you wanted to calibrate your unit to the Harman Headphone Curve, you could measure well known headphones that have been measured by Oratory and compare your measured curves to his....you may be able to set up a "calibration file" that would be able to predict the changes between the two and therefore you could create your own Amir Harman Headphone Curve from that. If the results are good you could publish the EQ filters used to fit the Amir Harman Headphone Curve for the headphones you test. This would be extremely useful for the community. If your rig is somehow superior to the measurement rig used by Oratory1990 for example, then that might give you room to improve the curve, but I'm hazy on this and how you would determine that to be true or not.

I'm sure there are other measurements within headphones that are important too, beyond frequency, so you could get involved with those in terms of measurement & interpretation.
 

JohnYang1997

Major Contributor
Technical Expert
Manufacturer
Joined
Dec 28, 2018
Messages
4,309
Likes
6,455
Location
China
#11
Very cool Amir, good that you're considering going into headphone measurement and applying the same rigour that you do with speaker testing! I agree though that headphones are harder to measure because we all have different HRTF's and different pinnas / ear canals and outer ears, all of which affects how we perceive sound from a headphone, so this is the extra layer of complication on top of speaker measurement.

I noticed you mentioned about it being a different measurement rig to a lot of the standards that are out there, e.g. the GRAS unit that Oratory1990 uses for instance, and therefore the standard Harman Headphone Curve might not be applicable to it due to the physical reasons mentioned in my first paragraph above. As an idea if you wanted to calibrate your unit to the Harman Headphone Curve, you could measure well known headphones that have been measured by Oratory and compare your measured curves to his....you may be able to set up a "calibration file" that would be able to predict the changes between the two and therefore you could create your own Amir Harman Headphone Curve from that. If the results are good you could publish the EQ filters used to fit the Amir Harman Headphone Curve for the headphones you test. This would be extremely useful for the community. If your rig is somehow superior to the measurement rig used by Oratory1990 for example, then that might give you room to improve the curve, but I'm hazy on this and how you would determine that to be true or not.

I'm sure there are other measurements within headphones that are important too, beyond frequency, so you could get involved with those in terms of measurement & interpretation.
Harman uses 5128 now.
Oratory's is not hires head with a high res pinna. Worst combination. Not standard at all. Oratory's measurements are one of the worst in terms of truthfulness.
 

Robbo99999

Addicted to Fun and Learning
Forum Donor
Joined
Jan 23, 2020
Messages
868
Likes
514
Location
UK
#13
Harman uses 5128 now.
Oratory's is not hires head with a high res pinna. Worst combination. Not standard at all. Oratory's measurements are one of the worst in terms of truthfulness.
Hmm, interesting, I heard the opposite, that his gear was the best on the market and applicable to the Harman Headphone Curve. He's the only person I heard of that has such gear that makes it truly precise in this way, also with his experience & profession it is directly applicable. Unless all this is nonsense that has been paraded on the internet. I personally find his EQ profiles very good.

As an aside, if you say Harman use different gear now, then wouldn't they have to change the Headphone Harman Curve? And I don't think they've changed it.

EDIT: If what you say is true then that means Amir's 5128 is the same as that used by Harman......have they not published the latest Headphone Harman Curve then, and if the curve doesn't need to be changed for this latest gear, then therefore the Headphone Harman Curve that we have now would still be applicable.
 

JohnYang1997

Major Contributor
Technical Expert
Manufacturer
Joined
Dec 28, 2018
Messages
4,309
Likes
6,455
Location
China
#15
Hmm, interesting, I heard the opposite, that his gear was the best on the market and applicable to the Harman Headphone Curve. He's the only person I heard of that has such gear that makes it truly precise in this way, also with his experience & profession it is directly applicable. Unless all this is nonsense that has been paraded on the internet. I personally find his EQ profiles very good.

As an aside, if you say Harman use different gear now, then wouldn't they have to change the Headphone Harman Curve? And I don't think they've changed it.

EDIT: If what you say is true then that means Amir's 5128 is the same as that used by Harman......have they not published the latest Headphone Harman Curve then, and if the curve doesn't need to be changed for this latest gear, then therefore the Headphone Harman Curve that we have now would still be applicable.
Harman never stated the curve to be an absolute. So allowing large variation. Changing a head is too little for that. The harman's work for headphone targets is good for spreading the message but the results are pretty bad and inconsistent really.

I gave up all the hope after seeing er4 and hd58x measurements and eq. I tried to help him but he was on the high place like all of us don't know shit, while I was involved with the ones that really make good shit, Moondrop. Let him design some earphones to compete....
 

solderdude

Major Contributor
Joined
Jul 21, 2018
Messages
6,230
Likes
11,444
Location
The Neverlands
#16
As an aside, if you say Harman use different gear now, then wouldn't they have to change the Headphone Harman Curve? And I don't think they've changed it.
Only the compensation curve (this is not the Harman curve) for their new test rig would have to be changed. Chances are headphones will have different plots even though they compensate to the same 'average correction'. That's the whole issue of 'standards'.
That is above 1kHz results will not be the same and it still is not wise to apply 'exact EQ' to thos measurements above a few kHz.
 

Robbo99999

Addicted to Fun and Learning
Forum Donor
Joined
Jan 23, 2020
Messages
868
Likes
514
Location
UK
#18
Harman never stated the curve to be an absolute. So allowing large variation. Changing a head is too little for that. The harman's work for headphone targets is good for spreading the message but the results are pretty bad and inconsistent really.

I gave up all the hope after seeing er4 and hd58x measurements and eq. I tried to help him but he was on the high place like all of us don't know shit, while I was involved with the ones that really make good shit, Moondrop. Let him design some earphones to compete....
Only the compensation curve (this is not the Harman curve) for their new test rig would have to be changed. Chances are headphones will have different plots even though they compensate to the same 'average correction'. That's the whole issue of 'standards'.
Ah, each measurement rig has a compensation curve then, kind of like a microphone calibration file. My prior idea about measuring known headphones on this rig and comparing against Oratory1990's could still be a way to get to a known standard though. How valid that is depends on how good you think Oratory's measurements are, at the moment I'm not convinced by a couple of you people saying they're no good, especially if intimated from Solderdude who is measuring them on a flat plate, he's got his own thing going on in his approach which is fine but totally different, it's not comparable.

EDIT: different EQ profiles could be provided from Amir, one set based on "calibration" to a known standard, e.g. by comparing against Oratory to create a "calibration file", and the other set of EQ profiles could be one without calibration and just EQ'd directly to the Harman Curve. Then people could see which they liked better.
 
Last edited:

JohnYang1997

Major Contributor
Technical Expert
Manufacturer
Joined
Dec 28, 2018
Messages
4,309
Likes
6,455
Location
China
#19
Ah, each measurement rig has a compensation curve then, kind of like a microphone calibration file. My prior idea about measuring known headphones on this rig and comparing against Oratory1990's could still be a way to get to a known standard though. How valid that is depends on how good you think Oratory's measurements are, at the moment I'm not convinced by a couple of you people saying they're no good, especially if intimated from Solderdude who is measuring them on a flat plate, he's got his own thing going on in his approach which is fine, but it's not comparable.
I won't worry too much honestly. I wasn't the same person as in 2018. Also remember, it's much much better to make real products people can get their hands on than talking or publishing papers. Revel did well. But what about headphones and earphones. They still failed to hit the target or sound good. It's getting a bit better now but that's based on trial and error. There are just too much to it that consumers can not comprehend. I have talked about these stuff in different time in various places. No one gives a shit. Well. Here I am making electronics.
 
Top Bottom