• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

ASR Headphone Testing and BK 5128 Hats Measurement System

JohnYang1997

Master Contributor
Technical Expert
Audio Company
Joined
Dec 28, 2018
Messages
7,175
Likes
18,292
Location
China
And something like subjective EQ curve to flat/taste is probably going to be more valuable than measurement results. IF we have consistent listener/s.
 

antdroid

Active Member
Reviewer
Joined
Apr 1, 2018
Messages
111
Likes
250
Location
Seattle
I would say, current measurement equipment can help creating "less bad" headphones but not enough for "better" headphones. For evaluation purposes, it depends on where you draw the line. But according to current speakers measurements and reviews I can only see less correlation between measurements and subjective results. And to be scientific we will see something like: 'your ear canals will likely be different so you may or may not have the peak' much more often.

I agree and disagree a bit.

Yes, the current rigs and measurements have limitations and assumptions built-in. I use a IEC60718-4 coupler for IEMs. It has limitations and a well-known resonance inherently in the system that can be shifted around with insertion depth. That can also be the same with a real live human, and how deep you insert can change the treble response of the IEM -- that I definitely agree with.

I do still think measurements, even the limitations we have today are beneficial and useful to have and one can learn a lot from them. But of course, measurements are only useful as long as you know the assumptions and how to actually read them.

There's an entire side of psychoacoustics and human anatomy that isn't touched upon when it comes to all of this sound stuff enough, and it does definitely play a big factor and sometimes can be forgotten when people look at graphs.

And something like subjective EQ curve to flat/taste is probably going to be more valuable than measurement results. IF we have consistent listener/s.

Shouldn't having a baseline measurement be useful to have any consistency for EQing to determine a preference target? (that's essentially what Olive et al did in their studies, as well as others in the industry who haven't published data).
 

JohnYang1997

Master Contributor
Technical Expert
Audio Company
Joined
Dec 28, 2018
Messages
7,175
Likes
18,292
Location
China
I agree and disagree a bit.

Yes, the current rigs and measurements have limitations and assumptions built-in. I use a IEC60718-4 coupler for IEMs. It has limitations and a well-known resonance inherently in the system that can be shifted around with insertion depth. That can also be the same with a real live human, and how deep you insert can change the treble response of the IEM -- that I definitely agree with.

I do still think measurements, even the limitations we have today are beneficial and useful to have and one can learn a lot from them. But of course, measurements are only useful as long as you know the assumptions and how to actually read them.

There's an entire side of psychoacoustics and human anatomy that isn't touched upon when it comes to all of this sound stuff enough, and it does definitely play a big factor and sometimes can be forgotten when people look at graphs.



Shouldn't having a baseline measurement be useful to have any consistency for EQing to determine a preference target? (that's essentially what Olive et al did in their studies, as well as others in the industry who haven't published data).
I agree what you are saying.
Now let me point out some very important points you were saying.
Measurement procedure itself becomes an art(just like Tyll said before, and was demonstrated by Crinacle). You can alter the measurement results to fit your perception. That's a very important part of measurements for headphones/in ears. (however the peaks sometimes won't at all appear in the correct frequency and you often get a wide dip after the peak)
Having a baseline measurement is definitely useful to bring some objectiveness and consistency. (but don't you think spending 50 grand for just a baseline measurement that's not really necessary is a bit too much?)
Harman doesn't publish the detailed data(well we know why they can't but still), so that doesn't replace the job for the reviewers. I definitely agreed with the approach the thought process of Harman Target. Just the conduction of the researches was not satisfactory. Little things here and there can lead to large variation in the end result. And I also know that Harman has done well for their being. But don't you think we need many more repeat experiments from others to confirm the results for it to be any meaningful? That's the issue.
 

solderdude

Grand Contributor
Joined
Jul 21, 2018
Messages
15,891
Likes
35,912
Location
The Neitherlands
In the end, what is interesting in Amirs short term experiments is that some well known and measured headphones on various sites can be compared to other rigs (standard or not) compared to this particular combo.

I expect to see little to no meaningful difference below 1kHz, some differences between 1kHz and 5kHz and substantial differences above 5kHz.
Alll of this depending on applied correction and smoothing of the plots, averaging of measurement positions (especially on HATS) etc.

Given the research done on ear canals and pinnae and how HATS manufacturers come up with their own 'solution' to correct and apply a certain target based on perception/preference research (also averaged) is what is particular of interest.
All HATS have somewhat different compensations due to the choices they made. This spread between test rigs and actual ears is IMO the reason why some feel oratory provides the 'best' EQ and others prefer Sonarworks or EQ based on Tyll's or other plots (Rtings etc.)
My suspicion is that particularly the higher frequency EQ, which seems to be derived so after EQ it becomes 'flat' on THAT particular test rig only, may have a comensation closer to that of their own hearing.
For this reason I am rather convinced that 'exact EQ opposite to a specific measurement' is not correct.
The reason is one can never be sure if the generated EQ is accurate or not. A sharp peak or null (above 5kHz) could be incorrect to reality because the measurement and applied correction may not be compatible with that headphone. A measured dip that isn't really there would thus be 'corrected' the other way.
Sure it will be an improvement in general compared to no EQ. (the latter is why we want to measure anyway).
It also explains why after such 'specific to a test rig only' EQ is applied to different headphones they still don't sound exactly the same but do have a similar tonal balance.

Also curious to see if Amir plans to review subjectively as well and which data will be selected to show as a review and if there will be a certain build-up of the review. I mean for speakers there seems to be a suite of data. For amps and DACs there is no common 'layout' except for the dashboard and certain plots.
 
Last edited:

Robbo99999

Master Contributor
Forum Donor
Joined
Jan 23, 2020
Messages
6,878
Likes
6,674
Location
UK
There's one game that we can play to prove how inconsistent the harman target/the current measurement equipment is.

First, get a flat speaker sounding flat(to in room target) in the room without any dips or peaks. (perfect sounding speakers basically)

Second, apply EQ of harman target compensated frequency response of various headphones to the speakers.

Thrid, play the preference game and score all the different headphone EQs including the original speakers without EQ.

Then you will know the problem.
What, that doesn't make sense to me, are you suggesting some kind of an experiment where you take the EQ filters from headphones that have been designed to meet the Headphone Harman Curve and take those same filters and apply them to a flat speaker.....that's crazy because the Headphone Harman Curve is definitely not supposed to be flat. Have I missed a step in my understanding of what you're proposing, and what is it?
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,524
Likes
37,057
So we are saying science has no way of helping create a better headphone?
NO, I am not saying that. I am saying to be a bit provocative, that current efforts are more like looking for your lost keys by the lamp post because that is where there is light in the dark. But you lost your keys somewhere else.

The reason measurement rigs and headphone designs don't translate when you put phone on head/ears is the variability in torso, head, outer ear, and ear canal. Not to mention an unnatural effect from turning your head, but the sound stays the same.

The work to jump the gap needs to be in customizing for the difference in users, or in figuring out a way to custom tune a headphone which is close until it works correctly for the end user.

If I get the unit you are considering purchase of, and used it to make a great headphone from measuring on that rig, much of the greatness is gone if my pinna is different and/or my ear canal has different peaks due to different shapes.

The binaural recording world is instructive. Some binaural dummy head recordings work really well for some people, and not for many others. I've not done it, but reports are if you get some mics to record at the entrance to your own ears, binaural recordings can be very lifelike, but only for you. Give your recordings to someone else and they'll vary in how well they work for others.
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,524
Likes
37,057
Maybe serious headphone listeners need to standardize their ears. We'll need an industry agreed upon larger than life pinna shape everyone uses. It will fit over your ears, and connect at the opening to the ear canal. That reduces differences. Then a simpler EQ for ear canal differences might be possible as I don't see how to create an industry standard ear canal for end users.
 

solderdude

Grand Contributor
Joined
Jul 21, 2018
Messages
15,891
Likes
35,912
Location
The Neitherlands
What, that doesn't make sense to me, are you suggesting some kind of an experiment where you take the EQ filters from headphones that have been designed to meet the Headphone Harman Curve and take those same filters and apply them to a flat speaker.....that's crazy because the Headphone Harman Curve is definitely not supposed to be flat. Have I missed a step in my understanding of what you're proposing, and what is it?

I don't think that is what he means.
His thought experiment is to compare a flat measuring speaker (I assume in a 'conditioned' listening room) and compare its sound impression (thus including body sensation addition to perception) and compare that complete perceived sound to the 'sound signature/experience' of an EQ'ed headphone.
What John questions is that these may differ depending on the used headphone, applied EQ based on specific rigs and when different 'speaker in room' corrections are applied.

I would add that this experiment would yield different results at different average SPL (say 70dB and 80dB).

For this reason I would like to see how Amir's measurements of well documented headphones (or rather the used HATS) compares to other measurements. Specifically above 5kHz where I found most problems in headphones are and when EQ'ed just a few dB 'wrongly' can make a difference.

Amir's experience is very welcome and useful in this way. The question is once the differences appear to be smaller than I expect is it worth investing 40-50k$ for the pleasure of a few people that might use generated EQ from Amirs rig compared to that of the already established Oratory or Rtings plots.
Also what data and listening impressions are posted and what other value is added compared to Rtings and the rather 'basic' info Oratory provides.

IMO it will be the added measurements/info/data that could make this work. I can tell you the measuring part is NOT the time consuming part.
The information gathering, plots generation, listening and evaluation and writing part are the time consuming things.

When Amir wants to reach > 200 headphones as to have a meaningfull database that increases ASR fame and traffic it will be years later.

In such case it might be handy if Amir measures and listens and puts that in words and perhaps let the data crunching, fomatting etc. be handled by someone else. Thereby maximizing the costs of hiring or buying such expensive gear.
 
Last edited:

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,524
Likes
37,057
I don't think that is what he means.
His thought experiment is to compare a flat meausuring speaker (I assume in a 'conditioned' listening room) and compare its sound impression (thus including body sensation addition to perception) and compare that complete perceived sound to the 'sound signature/experience' of an EQ'ed headphone.
What John questions is that these may differ depending on the used headphone, applied EQ based on specific rigs and when different 'speaker in room' corrections are applied.

I would add that this experiment would yield different results at different average SPL (say 70dB and 80dB).

For this reason I would like to see how Amir's measurements of well documented headphones (or rather the used HATS) compares to other measurements. Specifically above 5kHz where I found most problems in headphones are and when EQ'ed just a few dB 'wrongly' can make a difference.

Amir's experience is very welcome and useful in this way. The question is once the differences appear to be smaller than I expect is it worth investing 40-50k$ for the pleasure of a few people that might use generated EQ from Amirs rig compared to that of the already established Oratory or Rtings plots.
Also what data and listening impressions are posted and what other value is added compared to Rtings and the rather 'basic' info Oratory provides.

IMO it will be the added measurements/info/data that could make this work. I can tell you the measuring part is NOT the time consuming part.
The information gathering, plots generation, listening and evaluation and writing part are the time consuming things.

When Amir wants to reach > 200 headphones as to have a meaningfull database that increases ASR fame and traffic it will be years later.

In such case it might be handy if Amir measures and listens and puts that in words and perhaps let the data crunching, fomatting etc. be handled by someone else. Thereby maximizing the costs of hiring or buying such expensive gear.
This is why I thought partnering with someone on the measuring end, and maybe Amir just does the listening made some sense. Or perhaps this is a case sense every person is different where a volunteer panel who agrees to do listening evaluations after Amir measures would work. Maybe a panel of 20 or 30 people who agree to do this. Develop a standard form of reporting. Something like MUSHRA and cookie cutter outlines of personal evals.
 

Frank Dernie

Master Contributor
Forum Donor
Joined
Mar 24, 2016
Messages
6,445
Likes
15,781
Location
Oxfordshire
It seems to me that headphone measurement has many people doing it and no real consensus on what is either a good measurement method or what the result should look like, so @amirm just risks being one of several people with a load detractors and supporters. He has plenty of these already, so I am not sure testing headphones would add anything to this site at all, apart perhaps more subscribers and definitely more argument.

Personally I find the speaker tests by far the most interesting and informative of all the tests done here so far and would be disappointed if adding measuring headphones to the job list reduced the number of speakers which can be measured.
 

solderdude

Grand Contributor
Joined
Jul 21, 2018
Messages
15,891
Likes
35,912
Location
The Neitherlands
It seems to me that headphone measurement has many people doing it and no real consensus on what is either a good measurement method or what the result should look like.

Every advantage has its own disadvantage. Claiming any particular test setup is the only trustworthy one is nonsense.
In the end the goal is to find correlation between measurements and perceived sound.

Too many variables on the perception/preference side and too big deviations between measurements of an 'average' measurement and reality will prevent perfect correlation. Now and in the future. Everyone is free to choose what they prefer based on whatever knowledge they (think) they possess.
Adoration usually blinds. It's blind adoration that causes heated discussions so Amir will likely get to endure more flak also from the headphone community. Learning to ignore it is essential as a reviewer I reckon.
 
Last edited:

Absolute

Major Contributor
Forum Donor
Joined
Feb 5, 2017
Messages
1,084
Likes
2,125
I would love headphone measurements from ASR, but not until there's an established scientific consensus about validity to the measurements and the correlation to subjective preference.

When there's a standard like the spinorama for speakers established, I'm all for it.
As long as the speaker measurements don't suffer for it because no one else is doing that.

I don't know how ASR can add value to the headphone-world as of now, so why waste the time/money better spent where ASR makes a difference?
 

ezra_s

Active Member
Forum Donor
Joined
Mar 23, 2020
Messages
293
Likes
327
Location
Spain
What is a known bad headphone that is cheap?

I once bought "TIN Audio T2 Pro", those are in ear, it sounded like garbage.
 

zenki

Active Member
Joined
Apr 3, 2018
Messages
102
Likes
34
You need T2+.

From the new measuring system, the FR probably wouldn't improve much. The other things though such as IR, THD etc2x could be something
 

antdroid

Active Member
Reviewer
Joined
Apr 1, 2018
Messages
111
Likes
250
Location
Seattle
One thing to consider with grado or other on-ears is that it's harder to measure due to the fact it's on ear and placement varies even more than a standard over ear which already can be affected by placement despite getting seal.

For in-ears, tips and insertion depth will affect treble peaks/response.
 

Robbo99999

Master Contributor
Forum Donor
Joined
Jan 23, 2020
Messages
6,878
Likes
6,674
Location
UK
I don't think that is what he means.
His thought experiment is to compare a flat measuring speaker (I assume in a 'conditioned' listening room) and compare its sound impression (thus including body sensation addition to perception) and compare that complete perceived sound to the 'sound signature/experience' of an EQ'ed headphone.
What John questions is that these may differ depending on the used headphone, applied EQ based on specific rigs and when different 'speaker in room' corrections are applied.

I would add that this experiment would yield different results at different average SPL (say 70dB and 80dB).

For this reason I would like to see how Amir's measurements of well documented headphones (or rather the used HATS) compares to other measurements. Specifically above 5kHz where I found most problems in headphones are and when EQ'ed just a few dB 'wrongly' can make a difference.

Amir's experience is very welcome and useful in this way. The question is once the differences appear to be smaller than I expect is it worth investing 40-50k$ for the pleasure of a few people that might use generated EQ from Amirs rig compared to that of the already established Oratory or Rtings plots.
Also what data and listening impressions are posted and what other value is added compared to Rtings and the rather 'basic' info Oratory provides.

IMO it will be the added measurements/info/data that could make this work. I can tell you the measuring part is NOT the time consuming part.
The information gathering, plots generation, listening and evaluation and writing part are the time consuming things.

When Amir wants to reach > 200 headphones as to have a meaningfull database that increases ASR fame and traffic it will be years later.

In such case it might be handy if Amir measures and listens and puts that in words and perhaps let the data crunching, fomatting etc. be handled by someone else. Thereby maximizing the costs of hiring or buying such expensive gear.
Ah, is that what he's getting at (JohnYang).....well for me the Harman Headphone Oratory1990 EQ'd HD600's I have, for me they sound very close to my Harman Curve EQ'd JBL 308's, whereas my NAD HP50's that also have a Harman Headphone Oratory1990 EQ don't sound quite as accurate to my speakers, they miss the treble, treble is a bit dull in the NAD HP50's. So yes, there's variation from headphone to headphone, and even if they are EQ'd both to the Headphone Harman Curve they won't always sound the same, but the HD600 I have with Oratory EQ is a very good match for my speakers.

Well I welcome the potential edition of headphone testing on this site....there are of course some grey areas of what is to be achieved and how valid is B&K and what can be done with the data.....but I welcome the venture, and I suppose decision time will be once the trial period of the B&K is up, and I suppose there's the idea that it could be rented for longer if more investigation time was required before committing. (Or maybe the measurement gear can be sold on for hardly any loss, I don't know).
 

JohnYang1997

Master Contributor
Technical Expert
Audio Company
Joined
Dec 28, 2018
Messages
7,175
Likes
18,292
Location
China
I don't think that is what he means.
His thought experiment is to compare a flat measuring speaker (I assume in a 'conditioned' listening room) and compare its sound impression (thus including body sensation addition to perception) and compare that complete perceived sound to the 'sound signature/experience' of an EQ'ed headphone.
What John questions is that these may differ depending on the used headphone, applied EQ based on specific rigs and when different 'speaker in room' corrections are applied.

I would add that this experiment would yield different results at different average SPL (say 70dB and 80dB).

For this reason I would like to see how Amir's measurements of well documented headphones (or rather the used HATS) compares to other measurements. Specifically above 5kHz where I found most problems in headphones are and when EQ'ed just a few dB 'wrongly' can make a difference.

Amir's experience is very welcome and useful in this way. The question is once the differences appear to be smaller than I expect is it worth investing 40-50k$ for the pleasure of a few people that might use generated EQ from Amirs rig compared to that of the already established Oratory or Rtings plots.
Also what data and listening impressions are posted and what other value is added compared to Rtings and the rather 'basic' info Oratory provides.

IMO it will be the added measurements/info/data that could make this work. I can tell you the measuring part is NOT the time consuming part.
The information gathering, plots generation, listening and evaluation and writing part are the time consuming things.

When Amir wants to reach > 200 headphones as to have a meaningfull database that increases ASR fame and traffic it will be years later.

In such case it might be handy if Amir measures and listens and puts that in words and perhaps let the data crunching, fomatting etc. be handled by someone else. Thereby maximizing the costs of hiring or buying such expensive gear.
Yeah this is better.
 
Top Bottom