Master Complaint Thread About Headphone Measurements

[amirm]

That is very surprising. I thought they showed what people prefer, not what a headphones sounds like to a human.

Unbelievable.....

 

[Chagall]

I also think most of 'detail retrieval' is just people's imagination. They see this complex shaped audio signal representation, and think some headphones can create all those nook and crannies better than others. Check out posts here or on head-fi and you will see a lot of people making such claims.

Fair point. If it turns out to be imagination fine by me, I'll learn something about my brain and cognitive bias. I did try searching for posts here and found a lot of confusion, misrepresentation, and no agreement.

When we figure out what it is, I think it will most likely be boosting of certain frequencies without distorting them.

Don't think so, a boosted bass will mask higher frequencies making the headphones less detailed, and so will a lack of bass. As stated before the assumption is that "detailed" doesn't apply to any one region of the frequency response.

And I agree, it is a term usually abused to explain what an expensive headphone has to offer compared to cheaper one, and can simply be ignored without any side effect most of the time.

Sometimes sure! Sometimes maybe the expensive headphones are just better, sometimes they aren't. Bottom line we don't know for sure.

 

[amirm]

You do not identify the headphones with the best tonality if the best is to mean what most people prefer. You identify the ones that comply to a target, and present them as best to drive an agenda of standardization.

You sure have a way of making these conversations distasteful. I have already answered this half a dozen times yet you repeat this claim again???

Salnotes Zero was very popular long before you reviewed them, your review did not make them popular.

I said nothing about popularity. I have no idea how many it sold before or after my review and neither do you. The facts we have is person after person trying them after my review, posting how they are amazed, absolutely amazed at what fidelity they are getting out of them. IEM users didn't know you could go down to $20 and get this level of performance. And non-IEM users adopted them.

Nevertheless, your site, your review, your call.

Yet you chose to give me advice. Either that was a bad call or this one is.

 

[IAtaman]

How can you say that the research shows what people prefer, and at the same time say:


...which is to say that the research do not show what people prefer.

I'm not following the logic here.

I hope posts above would be able to clarify what I mean better. But in summary, maybe I can say preference is not as black as white as compliance to target. Research shows quite a few headphones that do deviate from the target might still be preferred by people, especially untrained ones.

 

[solderdude]

I have an opinion about differences in 'detail retrieval' of headphones.
It all has to do (again IMO) about the quality and elevation of the treble range above 8kHz.
What I found to be the case with headphones that (again to me) have an excellent detail retrieval all seem to measure (on my non any standard cheap) fixture treble response that is (often) slightly elevated but above all 'smooth' as in very little varying response between 8kHz and 15kHz (the last octave as it were).
As long as that is on the same level as the mids, the bass response is not too elevated and 'smooth' and it can even deviate in level in the 2kHz range those headphones seem to sound 'smooth' and 'grain-free' and cymbals etc sound 'nice'.

It should be noted that a lot of people feel headphones that simply have elevated treble (even peaky ones, most are b.t.w.) are 'highly detailed'.
When these peaks happen to be in the same range as their ear resonances then I can even believe these folks do not recognize that peak as 'sibilance, or 'sharpness'.

There is a very fine line between 'sharpness' and 'highly detailed' that also seems to be recording dependent. What is 'highly detailed' is someone else's sharpness. What is someone's 'bassy' is another one' s 'bloated' and what is someone's 'tight bass' is another one's 'bass-shy'. What's someone's 'forward/clear' is another one's 'shouty'.

This all has to do with ears and brains differing from standard(s).

Can we measure 'detail retrieval' and 'imaging' as well as 'sound stage' AND relate that to how ALL people perceive this ....no but there are indications in some measurements that do point at some of the aspects.

A popular 'case' is Hifiman headphones. Nearly all of them have a (nasty to me) peak in the 8-10kHz range often +6 to +10dB which you simply cannot see on certain industry standard fixtures for obvious reasons. That does not mean they are not there. It also does not mean all people are bothered by it.
One of the Hifiman headphones that (again acc. to my non-standard) measurements does not have 'peaky' highs but is elevated a little but 'smooth' in the treble is the well hated/liked Susvara which sets it apart a little from the other 'higher end' Hifiman.

A similar thing is the case with OLLO S5X and (once a peak at 6kHz is removed) in the HD800(S) as well and even a (modified) SRH-940/1540 has this as well as a DT1990 with the treble peak removed b.t.w.
All these headphones sound 'detailed' and have no 'harshness' nor 'grain' nor sibilance' even though they sound different in tonality (no EQ other than the treble for some).
A lucky shot it seems rather than engineering. Materials that happen to play 'nice' when combined.

I have to say (not confirmed this for myself though) that it looks like the 3 DCA top models have excellent 'flat' treble response. I need to get my ears/fixture on them one of these days. Dan may actually be onto something with his material in front of the driver.

So while the Harman research is primarily based on preference tonality the response above 8kHz is not included in that research (is not possible either) yet the response above 8kHz does matter even though some claim it doesn't (mostly because you can not measure it accurately for several reasons) yet those frequencies determine 'grain', 'sharpness' and possibly 'imaging' and who knows even 'soundstage' for some.

Just my non-scientific findings and (amateur) opinion.

 

[solderdude]

Research shows quite a few headphones that do deviate from the target might still be preferred by people, especially untrained ones.

That does not invalidate the research though. In fact it is even clear from the research and what Dr. Olive states himself... about a preference for the majority of people.

This means you cannot please everyone but you can please the majority. It means a minority has a different preference. When you do you can still fully utilize Amir's (and other people's measurements) that adhere to Harman by realizing YOU want more or less in a certain frequency range.

Amir measures acc. to a known standard (nothing wrong with that) and prefers the Harman target (just like the majority of people) so uses it to measure it against.
It is a choice Amir made based on a large body of research. How can you fault this ?

I look at Amirs measurements and get info from it.
With some recordings and once gotten used to a presentation from a headphone can enjoy Harman sound as well as one that is deviating from it. I expect the same is true for all people.

You want Amir to admit that 'THE target' is not preferred by all people. This is already clear from the Harman research but science has to adhere to standards for several reasons, people don't. Accept that. You are totally free to like something else than the Harman target. Nothing wrong with that but Amir wants to adhere to a standard which makes perfect sense from a science p.o.v.

 

[IAtaman]

That does not invalidate the research though. In fact it is even clear from the research and what Dr. Olive states himself... about a preference for the majority of people.

This means you cannot please everyone but you can please the majority. It means a minority has a different preference. When you do you can still fully utilize Amir's (and other people's measurements) that adhere to Harman by realizing YOU want more or less in a certain frequency range.

Amir measures acc. to a known standard (nothing wrong with that) and prefers the Harman target (just like the majority of people) so uses it to measure it too.
It is a choice based on a large body of research. How can you fault this ?

I look at Amirs measurements and get info from it.
With some recordings and once gotten used to a presentation from a headphone can enjoy Harman sound as well as one that is deviating from it. I expect the same is true for all people.

You want Amir to admit that 'THE target' is not preferred by all people. This is already clear from the Harman research but science has to adhere to standards for several reasons, people don't. Accept that. You are totally free to like something else than the Harman target. Nothing wrong with that but Amir wants to adhere to a standard which makes perfect sense from a science p.o.v.

I never suggested invalidation of research. I have not done any additional research so I can not invalidate anything. All my opinions are based on my understanding of the research conducted by Dr. Olive and his fellow researchers.

I do prefer the Harman target for OE headphones as well. I am not trying to argue for a different tuning or a standard. I actually find it irritating when people put their own made up targets on FR responses that has no research to support its correlation with preference.

I have no issues with the measurements, standards used in the measurements or their interpretations. My disagreement is on the conclusions.

Minority in the context of research might not be such a minority depending on the product by the way. What percentage of the people who are in the market for $5000 headphones are 50+ year old, untrained males you reckon? Or how prevalent are <25 year olds in the IEM market? Where is that nuance of the research in the conclusions of reviews?

I perceive Amir to be presenting his views and opinions on headphones as "the objective truths of scientific research". I disagree with that.
I think Amir treats every tuning that is not compliant to a target as incorrect. I disagree with that.
Amir says his conclusions are objective, independant of the mind as he quoted from Wiki, as in every objective mind would come to the same conclusion he did. I disagree with that.

Overall, I think we can get more people interested in the science of headphones with the "carrot" that is variation and nuance of preference rather than the "stick" of the compliance to a target in my opinion. That is my point.

 

[amirm]

Amir says his conclusions are objective, independant of the mind as he quoted from Wiki, as in every objective mind would come to the same conclusion he did. I disagree with that.

You have serious communication problems stating things like this. I made that statement in the context of measurements. Not my "conclusions." My conclusions include many factors, often going beyond pure performance.

 

[amirm]

Overall, I think we can get more people interested in the science of headphones with the "carrot" that is variation and nuance of preference rather than the "stick" of the compliance to a target in my opinion. That is my point.

Which is wrong and based a supposition. Such carrot would encourage headphone companies to continue to build to random frequency response that is less preferred by majority of people. There is no carrot here for buyers other than confusing them with what the story of the headphone engineering is.

 

[Chagall]

I have an opinion about differences in 'detail retrieval' of headphones.
It all has to do (again IMO) about the quality and elevation of the treble range above 8kHz.
What I found to be the case with headphones that (again to me) have an excellent detail retrieval all seem to measure (on my non any standard cheap) fixture treble response that is (often) slightly elevated but above all 'smooth' as in very little varying response between 8kHz and 15kHz (the last octave as it were).
As long as that is on the same level as the mids, the bass response is not too elevated and 'smooth' and it can even deviate in level in the 2kHz range those headphones seem to sound 'smooth' and 'grain-free' and cymbals etc sound 'nice'.

It should be noted that a lot of people feel headphones that simply have elevated treble (even peaky ones, most are b.t.w.) are 'highly detailed'.
When these peaks happen to be in the same range as their ear resonances then I can even believe these folks do not recognize that peak as 'sibilance, or 'sharpness'.

There is a very fine line between 'sharpness' and 'highly detailed' that also seems to be recording dependent. What is 'highly detailed' is someone else's sharpness. What is someone's 'bassy' is another one' s 'bloated' and what is someone's 'tight bass' is another one's 'bass-shy'. What's someone's 'forward/clear' is another one's 'shouty'.

This all has to do with ears and brains differing from standard(s).

Can we measure 'detail retrieval' and 'imaging' as well as 'sound stage' AND relate that to how ALL people perceive this ....no but there are indications in some measurements that do point at some of the aspects.

A popular 'case' is Hifiman headphones. Nearly all of them have a (nasty to me) peak in the 8-10kHz range often +6 to +10dB which you simply cannot see on certain industry standard fixtures for obvious reasons. That does not mean they are not there. It also does not mean all people are bothered by it.
One of the Hifiman headphones that (again acc. to my non-standard) measurements does not have 'peaky' highs but is elevated a little but 'smooth' in the treble is the well hated/liked Susvara which sets it apart a little from the other 'higher end' Hifiman.

A similar thing is the case with OLLO S5X and (once a peak at 6kHz is removed) in the HD800(S) as well and even a (modified) SRH-940/1540 has this as well as a DT1990 with the treble peak removed b.t.w.
All these headphones sound 'detailed' and have no 'harshness' nor 'grain' nor sibilance' even though they sound different in tonality (no EQ other than the treble for some).
A lucky shot it seems rather than engineering. Materials that happen to play 'nice' when combined.

I have to say (not confirmed this for myself though) that it looks like the 3 DCA top models have excellent 'flat' treble response. I need to get my ears/fixture on them one of these days. Dan may actually be onto something with his material in front of the driver.

So while the Harman research is primarily based on preference tonality the response above 8kHz is not included in that research (is not possible either) yet the response above 8kHz does matter even though some claim it doesn't (mostly because you can not measure it accurately for several reasons) yet those frequencies determine 'grain', 'sharpness' and possibly 'imaging' and who knows even 'soundstage' for some.

Just my non-scientific findings and (amateur) opinion.

Thank you @solderdude for taking the time and replaying. Was hoping to hear your opinion on this.

In my very humble opinion, DCA models have all the requirements to be highly detailed. Tonally accurate, low distortion, and the AMTS for standing waves.
The only thing I don't know is how E3 measures at the ear drum, but will find out soon enough.

 

[IAtaman]

You have serious communication problems stating things like this. I made that statement in the context of measurements. Not my "conclusions." My conclusions include many factors, often going beyond pure performance.

My statement is based on an interaction we had recently. I asked you do you claim your conclusions are objective and you said of course.

Here is a screen shot and here is the link to your post.

 

[amirm]

My statement is based on an interaction we had recently. I asked you do you claim your conclusions are objective and you said of course.

Yeh, conclusions about the measurements. Not the full review. Clearly the review has subjective parts that include listening tests, etc. This is the exchange that led to the response you quoted from me:

See how you clearly claimed headphone measurements do not lend any objective conclusions? And talk about "graphs?' That is what I was answering. They absolutely do as repeatedly explained to you.

 

[IAtaman]

Yeh, conclusions about the measurements. Not the full review. Clearly the review has subjective parts that include listening tests, etc. This is the exchange that led to the response you quoted from me:

View attachment 339224

Well, it might indeed be there is some confusion or misunderstanding somewhere because I was talking about the Conclusions part of your review as I mentioned in the original post you partially quoted.

See how you clearly claimed headphone measurements do not lend any objective conclusions? And talk about "graphs?' That is what I was answering. They absolutely do as repeatedly explained to you.

You did keep indeed saying graphs and measurements are objective and I kept repeating that I am not talking about graphs and measurements, I am talking about your conclusions. I suppose you thought I meant your conclusions of the measurements? I was referring to the conclusions of your reviews.

For the conclusions of measurements, the only conclusion that can be had objectively is whether headphone has resonances and high distortion, and then maybe a high level view of tonal balance I think. Is there any other objective conclusions one can reach from the measurements you do and share?

 

[IAtaman]

Which is wrong and based a supposition. Such carrot would encourage headphone companies to continue to build to random frequency response that is less preferred by majority of people. There is no carrot here for buyers other than confusing them with what the story of the headphone engineering is.

It is an opinion and it can be wrong. I am not suggesting to sugarcoat it if there is a high distortion or uncontrolled resonances. I am suggesting allowing for more leeway in tonality preference (as the research shows) will get more people on board with the research, and manufacturers will do whatever people are willing to pay for.

 

[solderdude]

Minority in the context of research might not be such a minority depending on the product by the way. What percentage of the people who are in the market for $5000 headphones are 50+ year old, untrained males you reckon? Or how prevalent are <25 year olds in the IEM market? Where is that nuance of the research in the conclusions of reviews?

You simply can't review for every 'group' or people or every preference.
The measurements show things.
Amir has a listen and has an opinion. He writes that down, everyone has an opinion. Opinions can vary.
Even with research the outcome can vary depending on the goal of that research.

It's just a matter of accepting that measurements are what they are. They are made by one person, on one fixture, in one configuration, in certain conditions and with one copy of a headphone in a certain condition and seating.

I am suggesting allowing for more leeway in tonality preference (as the research shows) will get more people on board with the research, and manufacturers will do whatever people are willing to pay for.

There is no leeway when it comes to measurements on a standard fixture while using a target.
Amir is not doing research. He just reports. All we can do is hope he is impartial on his opinion and looking at the facts that are important to him and fits in his viewpoint.
It is O.K. to disagree with Amir's conclusions or question the validity of the measurements.

Headphone measurements and relating them to measurements is an area with lots of variables and uncertainty. The leeway will have to come from the reader when it comes to assessing the measurements and Amir's opinion.

It would be weird if Amir would end (or start) with all reviews and state something like:
You may have a different preference than me or the Harman target and ti is totally fine to enjoy this device even when I find it poor value/sounding/quality and it this particular copy did not measure that well on my fixture.
That does not come across as a 'technical review' even when it would be accurate.

manufacturers will do whatever people are willing to pay for.

And there is the thing ... the market. Some are willing to spend up to $ 50.- others may be willing to fork out $ 5k. These people may or may not have the same goal for their purchase or the same reason. It is not something ASR nor Amir should worry about nor other people egos or preferences.

 

[ObjectiveSubjectivist]

I have to say (not confirmed this for myself though) that it looks like the 3 DCA top models have excellent 'flat' treble response. I need to get my ears/fixture on them one of these days. Dan may actually be onto something with his material in front of the driver.

Yes you have to.
Just remember about good very good seal with them.
It's their achilles heel.

 

[solderdude]

I want to, I even have access to them if I want to.
Could only do some crude on-site measurements though but is enough to form an opinion.
Time and opportunity are the things that have prevented me so far.
For a proper evaluation I need to have them a few days... at home.
For this to happen I would have to buy them and misuse return obligations of sellers or sell them at a loss and € 2k+ is not something I am comfortable with.

 

[IAtaman]

You simply can't review for every 'group' or people or every preference.
The measurements show things.
Amir has a listen and has an opinion. He writes that down, everyone has an opinion. Opinions can vary.
Even with research the outcome can vary depending on the goal of that research.

I believe you can. Instead of saying it comes way short in tonality for HD800S for example, you can say it has a balanced tonality that is lacking some energy in bass as well as ear gain area. This kind of tuning can be preferable to some who don't mind lack of bass, but if you EQ it with these filters, then it becomes delightful. For younger audience, this might not be a great match as it would require a lot of amplification to bring the bass levels to what is generally preferred.

Did you see my previous post by the way by any chance? Did you know in 3 of the 5 tests conducted to find a statistical model to estimate preference, untrained listeners found various headphones as preferable as the target itself? As Amir pointed out, program material was limited and lacking in sub bass but that is the case nonetheless. I was trying to find out which headphones they are, and I was told one of them might be HD800S. If that is indeed the case then that means on average untrained people, under controlled test conditions, find HD800S's out of the box tuning as preferable as the target itself while listening to music lacking in sub bass. If you told these people they were wrong, and they should actually be liking this other tuning better according to science, what do you think their reaction would be?

It would be weird if Amir would end (or start) with all reviews and state something like:
You may have a different preference than me or the Harman target and ti is totally fine to enjoy this device even when I find it poor value/sounding/quality and it this particular copy did not measure that well on my fixture.
That does not come across as a 'technical review' even when it would be accurate.

That would be weird indeed. But as we discussed in another thread I believe, his original disclaimer was very good and maybe can be reused.

And there is the thing ... the market. Some are willing to spend up to $ 50.- others may be willing to fork out $ 5k. These people may or may not have the same goal for their purchase or the same reason. It is not something ASR nor Amir should worry about nor other people egos or preferences.

This is where we disagree. If you do scientific reviews, and are claiming to be evaluating headphones' tonality objectively based on research on preference, you need to accommodate the variations in preference in my opinion.

 

[IAtaman]

I am sorry, I did not mean to ignore your comments but was carried away by other discussions

Amir checks for the compliance against the "preference reference target".

From the HE400SE review:

Let's check the FR:

View attachment 338784

This graph is similar to one of your examples which is HP14.

Yes, it does look indeed similar to HE400SE. HE400S was one of the headphones studied, maybe it might be that one, I am not sure. I heard some people say it might be the HD800S as well.

It is yours to to decide how you take the objective measurements when you are at the eve of buying a headphone. Looking for good compliance? Looking for best compliance, good compliance but good price, you may use as it is or you may apply EQ? Best compliance but money doesn't matter then go for a Dan Clark.

If that is all there is to the reviews then why all the listening tests and the written comments and such, why not share PDF files like Oratory1990 does and let everyone decide for themselves?

I am sorry but you seem like chasing a ghost and completely forget the logic behind a review is.

I have a background of reviewing computer hardware and you absolutely can't put any subjectivity in it. When you review a product you always check against some target (it may include user preference, industry standards etc..), how does it place among other products, check price/performance etc.. and come to some conclusion like if the product really worth the price or not.

Of course you can. Do all your benchmarking of graphics cards using Blender Cuda and see how the results look. Or take any Apple presentation on the performance of their hardware vs that of competition. Do you think those are objective? For the record, that is not what Amir is doing for sure. I do fully trust the reliability of his measurements. My point is you can manipulate measurements to look a certain way, even in computer hardware world.

For example, Dan Clark Stealth is a recommended headphone by Amir, but I would never pay that price. Another example is 1More Sonoflow NC Headphone, for the price this is a really good Wireless ANC headphone if you are looking for one, this also is recommended by Amir but I would never buy this headphone either.

Yet another example: 7Hz x Crinacle Zero:2, this has a top notch compliance with the target (the target I also personally favour , if you don't then it sucks to be you what can I say? Make your own target and check against it or completely forget the target no body cares ), it's dirt cheap and if I don't buy it Amir will kill me. So, I take the recommendation. See my point?

I think I do. You read the recommendations and if it makes sense to you, you go for it. What about the products you did not go for, because the conclusion was not very positive. Did you make any attempt to listen to them? If you get a chance to listen to them, do you think you'd be able to evaluate whether you like them or not without any bias?

 

[Pe8er]

If you do scientific reviews, and are claiming to be evaluating headphones' tonality objectively based on research on preference, you need to accommodate the variations in preference in my opinion.

Why though? It's clear you would like to see it, but how would it cater to ASR mission?

As far as I understand it, science (in general) demands adherence to a repeatable methodology and the simpler the result of research, the more applicable / useful it is. Especially in the context of our extremely convoluted industry. Accommodating variations (like you suggest) would make the results more complicated, diluted and less accessible to the public.

I think your prerogative is to make ASR research cater to a wider audience (which is a noble goal in itself!) but what you propose could actually end up perpetuating the current status quo. What do you think?