Bruel & Kjaer (BK), makers of precision acoustic measurement products, had a nice two day virtual conference. I wanted to listen to a bunch of them but had a major conflict. Fortunately, I watched two talks related to headphone measurements:
First one was by Jude Mansilla of Head-fi with the curious title of, "
Audio Measurements As Consumers Content."
The second was from Dr. Sean Olive of Harman, going through his standard presentation on headphone preference but also a new research addition: difference between various headphone measurement gear relative to the one they used in their research. Specifically we were treated to the first analysis of BK 5128 which Head-fi has adopted as their standard.
Jude Presentation
I was very surprised and disappointed in most of this presentation. Bulk of time was spent showing and ridiculing the DIY measurement rigs members of his own forum had created. He showed pictures of them and while correctly stating some issues with them, I just could not figure out what he is after. Hobbyist are creating these measurements because headphone companies are not providing them. If Jude wanted to improve things, he should have complained about lack of such measurements from that sector which would be likely to be in this conference, than DIY people. I personally admired the work and creativity people had put in building their various fixtures.
Fortunately the second half of the talk was better in that he showed a bit about how they measure headphones which seems to be following what Tyll did with use of square wave and such. I am personally of fan of driving headphone or speakers with square wave, especially a low frequency one. This can be hard on the transducer with the long duty cycle essentially being DC. I saw little justification for this method other than a hack to show the frequency response, sort of, using crude FFT.
The other thing he mentioned is that they no longer calibrate at one frequency and instead use white noise. I don't understand the merit of this either as matching the measurements to target needs to be done in a way that relates to the research. Credit to him he asked for feedback from audience but none was provided. Target matching is a visual thing for humans anyway so ultimately it doesn't matter per se.
I finally got an explanation of why they spent so much money on the Herzan isolation chamber. I am able to make measurements with better noise floor than them in my office with no isolation chamber. Answer is that they work in a noisy and busy office building and without noise isolation, they would have fair bit of pollution. In their typical promotion, they always made it sound like this was an essential thing in any measurement system rather than being a requirement in their noisy environment. We are fortunate to live far from civilization so have the quietness most of the time sans the delivery truck and our dogs barking because of that. I do my acoustic testing at night which eliminated this issue. Anyway, having an isolation chamber is not a bad idea but context is important and was not stated until now.
In QA section he was asked what his favorite headphone was. He said the Sennheiser HE-1 ($45,000). He was asked if he had measured it. Shockingly he said no! Gosh that was awkward when you are in a measurement seminar and you don't believe in this stuff to practice it.
Along these lines, he was asked if he felt listening tests were needed in headphones once you have measurements. He said no but could not provide any reasoning why.
Dr. Olive Presentation
As I noted, the meat of the presentation was a new research project to go back and remeasure a 20 headphones against half a dozen HATS and measurement gear. They only disclosed the difference between the modified GRAS 45 that they had used and BK 5128 which happened to be the one I, we are all interested in. The results were fascinating.
The BK 5128 underreported bass frequencies by substantial amount and in a sloping way. The lower the frequency, the less it reported the bass energy.
This caused the preference score of a headphone drop 45 points from high 80s down to 40s! There were also some differences in high frequencies but that was not that meaningful or emphasized in the presentation.
They developed a compensation curve for 5128 relative to Harman target but alas, not all headphones showed the same differential. Using this new target, the above headphone showed an error of I think 6 points. But there are others that cannot be fixed this way.
To summarize,
there is no way to use the Harman target curve 100% reliably when performing measurements using BK 5128! You can get close for majority of cases if you compensate but not all. And there could be effect on headphones that were not measured in the study.
So the overlaying that Jude does at head-fi with Harman target is totally wrong and inappropriate for bass energy.
As a proof point, Sean actually pulled my measurements of the
Dan Clark Stealth and compared it to Jude's:
Mine:
Judes:
View attachment 156436
He matched the sizes and played with the reference levels to get an apple vs apple comparison.
It clearly showed deficiency in bass measurements which he could explain with their research. But Jude's also showed a deficiency in the 3 to 6 kHz which I have circled in read which Sean and team could not explain. The dip does
not show up in my measurements. That seems to be a measurement error in Jude's measurement but of unknown cause right now.
This is a fairly big setback for Head-fi headphone measurements based on BK 5128. No benefit was identified in using it, and problems identified with what it does produce. Fortunately some of it can be fixed and I hope Jude adopts the modified curve as to reduce confusion that his measurements can create relative to Harman research and comparison with measurements that the rest of us are performing.
I have been making this point for quite a while that more precision in some measurement doesn't do you any good if you don't have a yardstick to compare it to. And that seems to be the case with BK5128. I think fair bit of the research money in development of that fixture should have gone toward creating realistic target curves for it. Without it, the hype has gotten ahead of its true value putting head-fi in a difficult situation right now.
I felt bad for Jude as Sean's presentation was right after his. He had just finished singing the praises of BK5128 but all he showed for a proof point was the same IEM measurements he has been showing and how an IEM they tested was bright and GRAS measurement didn't show it but BK5128 did (or the reverse, can't recall exactly which).
Sean was also asked about his favorite headphones and he mentioned a couple that were in under $200 range.
When he covered the difference bass preferences, he was asked what they target. He said that the JBL line is boosting the bass a couple of dBs with the assumption that younger crowd were buying so were more inclined to want more bass. AKG line was staying true to the target bass response.
One sour note was Samsung Legal censoring one of his slides. And the fact that the company's policy has become to make sure competitors are NOT taught how to compete with Harman as they disclose their research. This means we will not hear as much as we have heard in the past about their work. I can see why Samsung has taken this position from business point of view, but for the rest of us, it is not a good development.
I hope Samsung management/Legal understand that by sharing their research, their work has become a review criteria which can be very beneficial to them.
That's all I remember to post. I have asked HBK if they will provide on-demand versions of these presentations but I have not heard back.
But if you look at previous version of the HE1 you wonder why it took so long to implement.
