Master Complaint Thread About Headphone Measurements

[Chagall]

This makes no sense to me. A blurry window will have cut off the high frequency spectrum of the signal (i.e. it is a low pass filter). It is called reduced MTF (modulation transfer function) due to spreading function. So if that is the analogy, we have it exactly in measurements of headphones.

But let's say it is true. How is it quantified? Just some vague words that could mean a headphone is 40% good or just as well 80%? How do we know the reviewer knows to even evaluate such a thing? Do they have special training? Content that can provably show such issues/performance?

What if we went and surveyed all the people who test a headphone on this criteria. You think they all agree with each other? Something tells me they don't. And that is the problem with not having objective standards.

One of the main problems with such characteristics is that it gives liberty to a reviewer to speak from both sides of their mouth. Headphone doesn't measure well but oh, "it has great detail retrieval." Or that it measures well but not liked because it is not good at detail retrieval. So we are to throw out what we know is reliable and trust the nebulous?

Mind you, nothing wrong with reviewers coming up with new criteria. I have done that with sub-bass performance. Ask me about it though I both show the frequency response and hand you on a platter a single track that will easily qualify the performance of a headphone/speaker.

So if that is the analogy, we have it exactly in measurements of headphones.
Not exactly, but the assumption is that the headphones' frequency response, distortion, and lack of other masking elements like resonances should show a correlation to more detailed vs less detailed headphones.

Will people agree on these criteria?
Of course, they won't. Even when having objective standards, different people can have different conclusions.

Headphone doesn't measure well but oh, "it has great detail retrieval."
Reviewers being biased is a problem, but again, this can't happen or should not happen because of what determines "detail retrieval" or "transparency" or "resolution" or whatever term is used. If the assumption is correct.

All you are asking me now I can ask you about spatial qualities.
How do we objectively measure the spatial qualities of a headphone? How is it quantified? Do you think all will agree? Is the spatial quality of headphone A 80% better than that of headphone B?
We know some headphones have this effect and others don't and we can't just put a number on it...yet.

Or as you said in the Stealth review (in the video review), that the Stealth is transparent.
How is it quantified? Do you think all will agree? Is the transparency of headphone A 80% better than that of headphone B?
Honestly, I think "transparency" in your case and "detail retrieval" are the same thing.

Amir, I'm sure I don't have all the answers I'm just trying to learn and it is very difficult.

 

[Resolve]

So I post an interview where you claim detail retrieval is a function of high-end headphones.

Nonsense - I didn't at all claim it was a function of high end headphones at all, that's literally you reading your own narrative into my words. I asked in his mind where this 'detail' comes from - where some of the higher end ones are clearly better received than some lower end ones. Did you even listen to the dialogue? We're literally asking him the question because we want to attach it to measurements! You're yet again trying to create a false narrative.

 

[NTK]

Thanks, Amir.

I'm sure some YouTube reviewers use the term as you said, but for example, the explanation/analogy that Resolve gives in his reviews holds:

Detail Retrieval​

When considering detail retrieval, I like to borrow the 'image clarity' analogy expressed by a friend of mine. If you're looking through a window at a scene, how clear is the window? Even in a semi-opaque or translucent window, the images are identifiable, so it's not that all the various pieces of the music don't come through on lesser headphones, it's just that all the details about the musical elements are that much more clear and well-defined on a headphone like the XXXXXX. In other words, the more clear that image, the better detail retrieval the headphone has. This also redounds to improved representation of textural nuances and image structure in the music.

If that is a metric for window quality, does that mean that a window with a polarizer is automatically good? Wondering what is the equivalent of polarizer for audio?

 

[Resolve]

If that is a metric for window quality, does that mean that a window with a polarizer is automatically good? Wondering what is the equivalent of polarizer for audio?

View attachment 339187

It's not a metric for anything - it's an analogy. The metric would be frequency response at the ear drum... most likely.

 

[Chagall]

If that is a metric for window quality, does that mean that a window with a polarizer is automatically good? Wondering what is the equivalent of polarizer for audio?

View attachment 339187

Again this was just an analogy to describe detail or @Resolve said image clarity through a window. We can all watch the video.

I mean it's similar to @amirm analogy:
In some ways the Stealth reminded me of OLED TVs. The first time you watch them and see the inky blacks and wonderful contrast, you have regret that all that time you didn't get to see what your content really looked like. Stealth does that for sound reproduction.

 

[majingotan]

All you are asking me now I can ask you about spatial qualities.
How do we objectively measure the spatial qualities of a headphone? How is it quantified? Do you think all will agree? Is the spatial quality of headphone A 80% better than that of headphone B?
We know some headphones have this effect and others don't and we can't just put a number on it...yet.

Or as you said in the Stealth review (in the video review), that the Stealth is transparent.
How is it quantified? Do you think all will agree? Is the transparency of headphone A 80% better than that of headphone B?
Honestly, I think "transparency" in your case and "detail retrieval" are the same thing.

Ask this question on a subjective forum and they’ll easily tell you these subjective terms (detail, spatial or soundstage/imaging) are system dependent and not fully intrinsic with the headphone. The context of “% better” is subjective and NOT objective in this case. Strictly speaking “% better” as objective then Amir’s review is your guide for that

 

[majingotan]

If that is a metric for window quality, does that mean that a window with a polarizer is automatically good? Wondering what is the equivalent of polarizer for audio?

View attachment 339187

The first thing that came to my mind is a tube amp

 

[Resolve]

Ask this question on a subjective forum and they’ll easily tell you these subjective terms (detail, spatial or soundstage/imaging) are system dependent and not fully intrinsic with the headphone. The context of “% better” is subjective and NOT objective. Strictly speaking “% better” as objective then Amir’s review is your guide for that

Or Dr. Olive's statistical model I imagine. But yeah, that tends to be the way things go for this question.

 

[amirm]

All you are asking me now I can ask you about spatial qualities.
How do we objectively measure the spatial qualities of a headphone? How is it quantified? Do you think all will agree? Is the spatial quality of headphone A 80% better than that of headphone B?
We know some headphones have this effect and others don't and we can't just put a number on it...yet.

You can ask me but it is not the same thing. Spatial qualities is something that is recognized to exist. From paper:
Validation of a Virtual In-ear Headphone Listening Test Method
Todd Welti, Sean E. Olive, and Omid Khonsaripour

"Listeners were
instructed to rate the headphones based on the
overall perceived sound quality considering
attributes related to timbre, spatial and distortion
attributes."

There is no such thing in the literature as "detail retrieval."

I do have some ideas of how to quantify it and hope to find the time to perform some research on it. Until then, we are talking about an accepted performance feature of a headphone in literature vs made up stuff in review land.

 

[Chagall]

Ask this question on a subjective forum and they’ll easily tell you these subjective terms (detail, spatial or soundstage/imaging) are system dependent and not fully intrinsic with the headphone. The context of “% better” is subjective and NOT objective in this case. Strictly speaking “% better” as objective then Amir’s review is your guide for that

That's why I'm a proud member of this forum. And I'm sure if some not entirely objective question arises here we can discuss it, right?

 

[Resolve]

There is no such thing in the literature as "detail retrieval."

I do have some ideas of how to quantify it and hope to find the time to perform some research on it. Until then, we are talking about an accepted performance feature of a headphone in literature vs made up stuff in review land.

This is something we agree on haha. Not sure how to quantify stuff that runs afoul of the private language problem, but some indication of preference may be useful.

 

[amirm]

Or as you said in the Stealth review (in the video review), that the Stealth is transparent.
How is it quantified? Do you think all will agree? Is the transparency of headphone A 80% better than that of headphone B?

Such subjective commentary is provided in passing. No one is expected much less demanded to agree. Importantly, it is not at all a review category/criteria. The strength of performance of Stealth is in its objective performance, not the few words I said subjectively.

Honestly, I think "transparency" in your case and "detail retrieval" are the same thing.

Not remotely so. My definition of transparency is lack of coloration and distortion. Detail retrieval makes people think a headphone is reaching into the recording and finding things.

As I said, a few words of subjectivity is fine. I wish we could advance the research to the point where this is not necessary but it is currently. Question is whether subjectivity plays equal or even stronger role than objective measurements. In my reviews, they don't. In others, they do.

 

[majingotan]

You can ask me but it is not the same thing. Spatial qualities is something that is recognized to exist. From paper:
Validation of a Virtual In-ear Headphone Listening Test Method
Todd Welti, Sean E. Olive, and Omid Khonsaripour

"Listeners were
instructed to rate the headphones based on the
overall perceived sound quality considering
attributes related to timbre, spatial and distortion
attributes."

There is no such thing in the literature as "detail retrieval."

I do have some ideas of how to quantify it and hope to find the time to perform some research on it. Until then, we are talking about an accepted performance feature of a headphone in literature vs made up stuff in review land.

Spatial in this context is still system dependent. An Airpods 3 with Spatial Audio turned OFF would have less preferable spatial presentation compared to when Spatial Audio is turned to ON. DSP is what caused the spatial preference towards Airpods 3 to increase and not Airpods 3 itself in this case.

 

[amirm]

Spatial in this context is still system dependent. An Airpods 3 with Spatial Audio turned OFF would have less preferable spatial presentation compared to when Spatial Audio is turned to ON. DSP is what caused the spatial preference towards Airpods 3 to increase and not Airpods 3 itself in this case.

I am not talking about 3-D virtualization/fold down to 2 channel if that is what you are speaking of. That is an entirely different thing and can indeed, create true phantom images in 3-D space.

 

[Chagall]

Not remotely so. My definition of transparency is lack of coloration and distortion. Detail retrieval makes people think a headphone is reaching into the recording and finding things.

OK, then I misunderstood your post from yesterday.

@Resolve or you or anyone else can't be accountable for what some phrase makes people think. But I guess that can happen - people can think what they want, so to avoid any confusion let's discuss it, define it, and see what if anything determines it in measurements.

As I said, a few words of subjectivity is fine. I wish we could advance the research to the point where this is not necessary but it is currently. Question is whether subjectivity plays equal or even stronger role than objective measurements. In my reviews, they don't. In others, they do.

Think we all agree with that.

 

[Chagall]

You can ask me but it is not the same thing. Spatial qualities is something that is recognized to exist. From paper:
Validation of a Virtual In-ear Headphone Listening Test Method
Todd Welti, Sean E. Olive, and Omid Khonsaripour

"Listeners were
instructed to rate the headphones based on the
overall perceived sound quality considering
attributes related to timbre, spatial and distortion
attributes."

There is no such thing in the literature as "detail retrieval."

I do have some ideas of how to quantify it and hope to find the time to perform some research on it. Until then, we are talking about an accepted performance feature of a headphone in literature vs made up stuff in review land.

Point taken. But is there a middle ground? I mean we all heard more detailed headphones than another. And of course, source gear has nothing to do with this, I'm only talking about headphones.

 

[IAtaman]

I don't know how doing what you suggest will cast a wider net. Surely if I identify headphones that have best tonality, that will do that a lot better than sugarcoating the performance of a lesser headphone. My recent review of $20 IEMs have gotten a ton of people interested in using them precisely because they match the target and have superbly low distortion, almost assuring a great experience. This is the power of the way I have been doing reviews. It is working quite effectively and is in no need of modification as you suggest.

You do not identify the headphones with the best tonality if the best is to mean what most people prefer. You identify the ones that comply to a target, and present them as best to drive an agenda of standardization. Salnotes Zero was very popular long before you reviewed them, your review did not make them popular. Nevertheless, your site, your review, your call.

 

[IAtaman]

Measurements absolutely show what a headphone sounds like to a human. What do you think Dr. Olive and crew were doing for five years? Pontificating?

That is very surprising. I thought they showed what people prefer, not what a headphones sounds like to a human. Could you please point out where they show what a headphones sounds like to a human?

 

[IAtaman]

Point taken. But is there a middle ground? I mean we all heard more detailed headphones than another. And of course, source gear has nothing to do with this, I'm only talking about headphones.

I also think most of 'detail retrieval' is just people's imagination. They see this complex shaped audio signal representation, and think some headphones can create all those nook and crannies better than others. Check out posts here or on head-fi and you will see a lot of people making such claims.

When we figure out what it is, I think it will most likely be boosting of certain frequencies without distorting them.

And I agree, it is a term usually abused to explain what an expensive headphone has to offer compared to cheaper one, and can simply be ignored without any side effect most of the time.

For other qualities of headphones that are not strongly correlated with specific measurements such as "imaging" and "sound stage", rtings.com has an interesting test suite they do for every headphone in an effort to quantify those characteristics people are interested in. There is no experiment that demonstrates if what they measure does indeed explain what people perceive as far as I am aware, but I think they give a good impression on what people think might be going on in the background.

 

[splitting_ears]

That is very surprising. I thought they showed what people prefer, not what a headphones sounds like to a human.

How can you say that the research shows what people prefer, and at the same time say:

You do not identify the headphones with the best tonality if the best is to mean what most people prefer. You identify the ones that comply to a target

...which is to say that the research do not show what people prefer.

I'm not following the logic here.