Strong with the knowledge of all the literature, you should then logically conclude that a statement like this is incorrect :
Absolutely not. I am confident if you ask Sean he will tell you the same thing. You are setting a standard of 100% accuracy and such a thing simply doesn't exist. It doesn't exist for speakers and certainly doesn't exist for headphones where our measurement abilities are more limited. I have said repeatedly that objective headphone assessments are 60 to 70% prescriptive. That is heck of a lot more than zero and at the risk of stating the obvious, it is not 100%.
Dan Clark headphones have self adjusting cup pressure. So clearly you can come up with scenarios that show large variations in response. If someone's face is very narrow for example, there may be leakage effects. Or you may have inability to mount it on their head in the optimal position. This is a given compromise in the whole category. That can't be determinative or normative relative to my work.
Speaking of my work, I measure and then apply and test the measurements on a sample of one: me. If both the measurements and I agree, then that is a high confidence assessment. Not remotely a guarantee, but high confidence. When someone then asks me if they should buy a headphone that I recommend, I can say with confidence that they can but should have the option to return. Likewise, when both measurements and I say something sounds bad, then you have high confidence that it does. Again, not a guarantee.
You have been linking to Sean's powerpoint. I have trouble accepting all that there is there without a paper to read with all the detail. Personally, I have always disliked testing that shows the range of responses and then averaging them. It is trivial to misposition a headphone on a fixture, or on your head. It makes no sense to show that, nor does it make sense to average that in some kind of final response. Averaging is a lousy low pass filter anyway. As you probably know, there is no exact science on how you put a headphone on someone's head. There is variability in that very thing. But sure, I can design a headphone with adjustable clamping pressure and soft enough pads to get lower variance. Dan Clark headphones don't have that but what they do have is comfort for many that fit in its automatic tension range.
I'm glad that ASR exists and I think that it's overall providing a great service to the community, and you'll do whatever you want anyway, but don't be surprised whenever someone writes in a thread like the Stealth's review "I don't like how the Harman target sounds" or "my experience doesn't seem to match the measurements", as there are tangible, measurable factors that could lead to these impressions, other than mere psychoacoustics.
I don't know how in the context of going over literature, we all of a sudden want to rationalize such statements made by someone in totally uncontrolled and anecdotal experience. No way can you assign cause and effect there unless you test the person and even then there are pitfalls as I mentioned above. I hear those comments day in and day out from owners of products. It is not something I can act on or value in my testing. Can they be right? Sure. But since we can't put any weight behind such subjective remarks, that is that.
The proper way to respond to that is not to believe them then. But to help them conduct a controlled test with equalization.
Getting back to what I said:
"When the response resembles the dashed line, you can have high confidence that you will like the sound. If not, you can apply a bit of EQ but the response should be close."
That is absolutely correct. It even addresses your concern by saying you may have to adjust things. But that you start with a very good starting point that should be close to optimal as opposed to getting a headphone with wild frequency response. Surely that headphone doesn't magically get better with moving a few mm here and there on your head.
