• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

HBK Headphone Measurement Talks from Head-Fi and Sean Olive

Somafunk

Major Contributor
Joined
Mar 1, 2021
Messages
1,311
Likes
3,027
Location
Scotland

Chester

Senior Member
Joined
May 22, 2021
Messages
434
Likes
1,023
I see nothing wrong with it if he truly likes it though. It's only a matter when we know the person(s) liking them only because they're expensive, or they're trying to guide the crowd into thinking that way.

I think it’s more likely Jude named the HE1 as his favourite headphone as they are not realistically in competition with any of his sponsors. They are a safe bet to keep the sponsors happy.
 

Sean Olive

Senior Member
Audio Luminary
Technical Expert
Joined
Jul 31, 2019
Messages
329
Likes
3,004
Bruel & Kjaer (BK), makers of precision acoustic measurement products, had a nice two day virtual conference. I wanted to listen to a bunch of them but had a major conflict. Fortunately, I watched two talks related to headphone measurements:

First one was by Jude Mansilla of Head-fi with the curious title of, "Audio Measurements As Consumers Content."

The second was from Dr. Sean Olive of Harman, going through his standard presentation on headphone preference but also a new research addition: difference between various headphone measurement gear relative to the one they used in their research. Specifically we were treated to the first analysis of BK 5128 which Head-fi has adopted as their standard.

Jude Presentation
I was very surprised and disappointed in most of this presentation. Bulk of time was spent showing and ridiculing the DIY measurement rigs members of his own forum had created. He showed pictures of them and while correctly stating some issues with them, I just could not figure out what he is after. Hobbyist are creating these measurements because headphone companies are not providing them. If Jude wanted to improve things, he should have complained about lack of such measurements from that sector which would be likely to be in this conference, than DIY people. I personally admired the work and creativity people had put in building their various fixtures.

Fortunately the second half of the talk was better in that he showed a bit about how they measure headphones which seems to be following what Tyll did with use of square wave and such. I am personally of fan of driving headphone or speakers with square wave, especially a low frequency one. This can be hard on the transducer with the long duty cycle essentially being DC. I saw little justification for this method other than a hack to show the frequency response, sort of, using crude FFT.

The other thing he mentioned is that they no longer calibrate at one frequency and instead use white noise. I don't understand the merit of this either as matching the measurements to target needs to be done in a way that relates to the research. Credit to him he asked for feedback from audience but none was provided. Target matching is a visual thing for humans anyway so ultimately it doesn't matter per se.

I finally got an explanation of why they spent so much money on the Herzan isolation chamber. I am able to make measurements with better noise floor than them in my office with no isolation chamber. Answer is that they work in a noisy and busy office building and without noise isolation, they would have fair bit of pollution. In their typical promotion, they always made it sound like this was an essential thing in any measurement system rather than being a requirement in their noisy environment. We are fortunate to live far from civilization so have the quietness most of the time sans the delivery truck and our dogs barking because of that. I do my acoustic testing at night which eliminated this issue. Anyway, having an isolation chamber is not a bad idea but context is important and was not stated until now.

In QA section he was asked what his favorite headphone was. He said the Sennheiser HE-1 ($45,000). He was asked if he had measured it. Shockingly he said no! Gosh that was awkward when you are in a measurement seminar and you don't believe in this stuff to practice it.

Along these lines, he was asked if he felt listening tests were needed in headphones once you have measurements. He said no but could not provide any reasoning why.

Dr. Olive Presentation
As I noted, the meat of the presentation was a new research project to go back and remeasure a 20 headphones against half a dozen HATS and measurement gear. They only disclosed the difference between the modified GRAS 45 that they had used and BK 5128 which happened to be the one I, we are all interested in. The results were fascinating.

The BK 5128 underreported bass frequencies by substantial amount and in a sloping way. The lower the frequency, the less it reported the bass energy. This caused the preference score of a headphone drop 45 points from high 80s down to 40s! There were also some differences in high frequencies but that was not that meaningful or emphasized in the presentation.

They developed a compensation curve for 5128 relative to Harman target but alas, not all headphones showed the same differential. Using this new target, the above headphone showed an error of I think 6 points. But there are others that cannot be fixed this way.

To summarize, there is no way to use the Harman target curve 100% reliably when performing measurements using BK 5128! You can get close for majority of cases if you compensate but not all. And there could be effect on headphones that were not measured in the study.

So the overlaying that Jude does at head-fi with Harman target is totally wrong and inappropriate for bass energy.

As a proof point, Sean actually pulled my measurements of the Dan Clark Stealth and compared it to Jude's:

Mine:
index.php


Judes:

View attachment 156436

He matched the sizes and played with the reference levels to get an apple vs apple comparison. It clearly showed deficiency in bass measurements which he could explain with their research. But Jude's also showed a deficiency in the 3 to 6 kHz which I have circled in read which Sean and team could not explain. The dip does not show up in my measurements. That seems to be a measurement error in Jude's measurement but of unknown cause right now.

This is a fairly big setback for Head-fi headphone measurements based on BK 5128. No benefit was identified in using it, and problems identified with what it does produce. Fortunately some of it can be fixed and I hope Jude adopts the modified curve as to reduce confusion that his measurements can create relative to Harman research and comparison with measurements that the rest of us are performing.

I have been making this point for quite a while that more precision in some measurement doesn't do you any good if you don't have a yardstick to compare it to. And that seems to be the case with BK5128. I think fair bit of the research money in development of that fixture should have gone toward creating realistic target curves for it. Without it, the hype has gotten ahead of its true value putting head-fi in a difficult situation right now.

I felt bad for Jude as Sean's presentation was right after his. He had just finished singing the praises of BK5128 but all he showed for a proof point was the same IEM measurements he has been showing and how an IEM they tested was bright and GRAS measurement didn't show it but BK5128 did (or the reverse, can't recall exactly which).

Sean was also asked about his favorite headphones and he mentioned a couple that were in under $200 range.

When he covered the difference bass preferences, he was asked what they target. He said that the JBL line is boosting the bass a couple of dBs with the assumption that younger crowd were buying so were more inclined to want more bass. AKG line was staying true to the target bass response.

One sour note was Samsung Legal censoring one of his slides. And the fact that the company's policy has become to make sure competitors are NOT taught how to compete with Harman as they disclose their research. This means we will not hear as much as we have heard in the past about their work. I can see why Samsung has taken this position from business point of view, but for the rest of us, it is not a good development.

I hope Samsung management/Legal understand that by sharing their research, their work has become a review criteria which can be very beneficial to them.

That's all I remember to post. I have asked HBK if they will provide on-demand versions of these presentations but I have not heard back.
One more thing: My intention of this presentation was not to disparage the B&K 5128, and I don't think this presentation did that.

We own a 5128 and I hope to use it more in our research. The problem up until now was our research was based on a different test fixtures, and we needed some way to be able to better interpret headphone measurements made on the 5128. That was the motivation of the research.

Jude maintains that the 5128 is more accurate than the other test fixtures. In terms of representing the acoustics of average human ear canals that may well be true. But there still needs to be some way of interpreting headphone measurements made on the 5128 in terms of human perception of sound quality. In my view, it doesn't matter if the 5128 is more accurate, if there isn't any data on how the measurements correlate with listeners' sound quality ratings.

So until someone has done a similar study of subjective and objective headphone measurements made on the B&K 5128, this Harman Target customized for the 5128 is the best solution we have for interpreting what the measurements mean in terms of how the headphones sound.
 

Sean Olive

Senior Member
Audio Luminary
Technical Expert
Joined
Jul 31, 2019
Messages
329
Likes
3,004
Sean, were your notable/favorite headphones referenced in the talk (and Amirm's summary) the K371 and JBL Tune 710, or were there others?

Also, do the Tune 710's still have the same profile when using the 3.5mm jack, or is that a DSP'd profile only available in the active BT mode?
That measurement is based on BT mode. I didn't measure the analog input.
 

Sean Olive

Senior Member
Audio Luminary
Technical Expert
Joined
Jul 31, 2019
Messages
329
Likes
3,004
I think it’s more likely Jude named the HE1 as his favourite headphone as they are not realistically in competition with any of his sponsors. They are a safe bet to keep the sponsors happy.
=
It's really not a good thing that so many people are measuring headphones and there isn't the barest thing close to a standard. Imagine buying or making a fixture and measuring hundreds.of headphones and then finding out the data is not reliable or meaningful? Anyone in that position would be prone to denial.

The irony is that one of the major issues in HP measurements is similar to spliced speaker measurements - bass level. Just as stereophile has no way to get the bass level right ( I've made the same mistake) I see huge variation in bass level from different measurement sources.

It is also a real shame that Harman is clamping down on research sharing. We have benefitted so much from their research on speaker tonality and dispersion, but it is possible that the publication of these standards has decreased the differentiation of their products.

This leaves it up to us to develop standards. I think ASR should lead the way.

I have no idea what this entails but I suspect that building bridges with other reviewers would be a step in the right direction. It would be nice, for example, if different measurement sets (ASR, headfi, crinacle, etc etc) were compared and a compensation curve was developed to get all this data to gel.

Unfortunately it looks as if the measurement protocol and pinnae are so different that much data is simply invalid? Someone correct me if I'm thinking about this wrong.
The thing is, at least these fixtures made by GRAS and B&K DO meet an IEC standard. The 5128 is most different from the IEC 711 couplers and there is a new standard to include it. Beside the HF differences which could be pinnae related is the low frequency response. Some of this could be related to the larger volume of the 5128 coupler. The other difference is how the headphone seals to the shape of the head/cheek which always creates a greater leak than the flat plate of the GRAS 45 CA. We see similar leaks with the KEMAR head compare to the GRAS45CA. The question is which fixture best simulates average headphone leakage measured on humans?
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,376
Likes
234,537
Location
Seattle Area
Jude maintains that the 5128 is more accurate than the other test fixtures. In terms of representing the acoustics of average human ear canals that may well be true. But there still needs to be some way of interpreting headphone measurements made on the 5128 in terms of human perception of sound quality. In my view, it doesn't matter if the 5128 is more accurate, if there isn't any data on how the measurements correlate with listeners' sound quality ratings.
That's been very much my position. I must have said it a hundred times. :) The problem we face is that he has been heavily promoting the 5128, doing infomercials for it, etc. and causing confusion with some people thinking its results are more meaningful.

So while I appreciate your good intentions, the net result has been increased confusion in the market and it needed correcting.
 

617

Major Contributor
Forum Donor
Joined
Mar 18, 2019
Messages
2,404
Likes
5,296
Location
Somerville, MA
=

The thing is, at least these fixtures made by GRAS and B&K DO meet an IEC standard. The 5128 is most different from the IEC 711 couplers and there is a new standard to include it. Beside the HF differences which could be pinnae related is the low frequency response. Some of this could be related to the larger volume of the 5128 coupler. The other difference is how the headphone seals to the shape of the head/cheek which always creates a greater leak than the flat plate of the GRAS 45 CA. We see similar leaks with the KEMAR head compare to the GRAS45CA. The question is which fixture best simulates average headphone leakage measured on humans?
Well, how do you measure that? Would be naive to suggest putting microphones in a bunch of human ears?

In the same way that we strive to create room independent loudspeakers, should we be trying to create cheek independent headphones?

Perhaps a good consumer test of a headphone should include different cheek shapes. Ergonomics are huge in headphone quality and cannot be separated from audio quality.
 

the_brunx

Senior Member
Joined
Apr 7, 2020
Messages
341
Likes
859
I recently deleted my headfi account because of the overwhelming cesspool of obvious shills masquerading as fellow users in that platform, I observed for years and it became obvious to me some of the dirty tricks that goes on on that forum. It seems all a sly manufacturer has to do there is get a band of shills on headfi to post glaring claims and reviews (“honest impressions”) that the product sounds like the best thing since slice bread and the rest of the unaware crowd will follow suit and start buying the product. Or get some shills to go into a competitors Thread and intentionally bash and thrash the competitor’s product with lies/misinformation about how they sound and the rest stop buying it.
And if anyone posts anything factual but potentially damaging to the image of a product the shills are protecting they will have your comments deleted by quickly quoting your comment and personally attack you with vulgar and both comments will get deleted by admin. It’s become a rigged online mafia marketing platform imo. Started from what seemed as honest audio passion when I joined, now when you read between the lines, it’s mostly just about money and lots of snake oil there. Won’t participate anymore.
 
Last edited:

GXAlan

Major Contributor
Forum Donor
Joined
Jan 15, 2020
Messages
3,868
Likes
5,955
@Sean Olive, one of the headphones in my collection is the vintage AKG K-60. It lacks the bass of modern headphones but is actually a very pleasant sounding headphone and I felt as if it did a very good job with imaging music with out-of-phase sound effects. Its FR curve is widely different from what we would consider to be the ideal target, but back in the 1960's, the "subjective evaluation" was relatively flat.

Do you have any comments on some of this early work? How were people trying to quantify the differences between subjective experience and objective measurements back then?



1633067215778.png
 

thewas

Master Contributor
Forum Donor
Joined
Jan 15, 2020
Messages
6,761
Likes
16,227
I have that data. We made 5 reseats for each of the 20 headphones we measured using different amount of pressure on the cups. So a total of 20 headphones x 5 reseats x 3 pressures x 6 fixtures =1800 measurements
Could you say about the magnitude of order how much the delta of the measurements varies between the GRAS and B&K?
 

Jave

Active Member
Joined
Apr 1, 2016
Messages
154
Likes
73
@Sean Olive, @amirm - Couldn't the differences between Amir's and Jude's measurements be explained even by seating the headphones differently on the measurements fixture? These headphones' FR varies significantly with placement. In the sense that a comparison between the measurements can be problematic.
 

MayaTlab

Addicted to Fun and Learning
Joined
Aug 15, 2020
Messages
924
Likes
1,512
Couldn't the differences between Amir's and Jude's measurements be explained even by seating the headphones differently on the measurements fixture? These headphones' FR varies significantly with placement. In the sense that a comparison between the measurements can be problematic.

An illustration of the various individual seatings Jude obtained on the 5128 :
 

oluvsgadgets

Member
Reviewer
Joined
Feb 27, 2021
Messages
29
Likes
99
Here is my quick measurement of the Stealth, done with my own ears, by just playing pink noise target is Optimum Hifi, and I get a heavy 9khz peak. This is averaged over 11 different positions for the Stealth, also averaged for left and right.
Stealth.png

for comparison the Focal Elex (green) and Utopia (blue) also exactly my ears etc:
Elex-Utopia.png

I am not sure how nobody so far didn't notice that 9khz peak, it sounds just horrible to me, also dunno why it doesn't show up in Amir's measurements, but that peak is noticeable in the other measurement to some degree.
To me the headphone sounded just horrible, not sure why many claim it to be the "world's best closed headphone". a complete letdown soundwise.
 

Thomas_A

Major Contributor
Forum Donor
Joined
Jun 20, 2019
Messages
3,423
Likes
2,408
Location
Sweden
If there were 11 measurements made one could take the median curve and give standard deviation. Are the such data published for the devices?

Ha just saw the 11 measurements post after I posted this... :)
 

thewas

Master Contributor
Forum Donor
Joined
Jan 15, 2020
Messages
6,761
Likes
16,227
Here is my quick measurement of the Stealth, done with my own ears, by just playing pink noise target is Optimum Hifi, and I get a heavy 9khz peak. This is averaged over 11 different positions for the Stealth, also averaged for left and right.
View attachment 156525
for comparison the Focal Elex (green) and Utopia (blue) also exactly my ears etc:
View attachment 156528
I am not sure how nobody so far didn't notice that 9khz peak, it sounds just horrible to me, also dunno why it doesn't show up in Amir's measurements, but that peak is noticeable in the other measurement to some degree.
To me the headphone sounded just horrible, not sure why many claim it to be the "world's best closed headphone". a complete letdown soundwise.
But that is due to your own ears and mic compensation, for example on Oratory both the Elex and Utopia show a dip around 9 kHz


and its logical that if those measure neutral there for you the Stealth will have a peak there.
 

oivavoi

Major Contributor
Forum Donor
Joined
Jan 12, 2017
Messages
1,721
Likes
1,935
Location
Oslo, Norway
Here is my quick measurement of the Stealth, done with my own ears, by just playing pink noise target is Optimum Hifi, and I get a heavy 9khz peak. This is averaged over 11 different positions for the Stealth, also averaged for left and right.
View attachment 156525
for comparison the Focal Elex (green) and Utopia (blue) also exactly my ears etc:
View attachment 156528
I am not sure how nobody so far didn't notice that 9khz peak, it sounds just horrible to me, also dunno why it doesn't show up in Amir's measurements, but that peak is noticeable in the other measurement to some degree.
To me the headphone sounded just horrible, not sure why many claim it to be the "world's best closed headphone". a complete letdown soundwise.

Interesting!
Curious: How are you able to measure this while the phones are on your head? I am sure it has been covered on the forum before, but I haven't followed the headphone discussions very closely.
 

111a111sk

Member
Joined
Jul 21, 2020
Messages
10
Likes
12
(good chance I have a way oversimplified idea)
If you put a flat measuring (on a microphone) loudspeaker into an anechoic chamber and measure it using a dummy head&torso rig, don't you get its compensation (neutral target) curve just like that?
 

abdo123

Master Contributor
Forum Donor
Joined
Nov 15, 2020
Messages
7,425
Likes
7,941
Location
Brussels, Belgium
Stupid Question, how do we know that the B&K is inaccurate while the GRAS is more accurate instead of the other way around?

I mean is there like a 'reference' headphone out there that we're absolutely certain matches the Harman Target that we can use to weed out the bad measurement rigs?
 

thewas

Master Contributor
Forum Donor
Joined
Jan 15, 2020
Messages
6,761
Likes
16,227
Interesting!
Curious: How are you able to measure this while the phones are on your head? I am sure it has been covered on the forum before, but I haven't followed the headphone discussions very closely.
With tiny special mics inserted in your ear channels.
 
Top Bottom