Master Complaint Thread About Headphone Measurements

[seyl3r]

I wonder if passive soundstage will be a testing parameter in the future, as far as I know Rting is the only one doing this with big collections at the moment.

Are there any plans related to this: https://www.rtings.com/headphones/tests/sound-quality/passive-soundstage? Soundstage and Imaging differs a lot on different headphones and is not visible in the current data presented, right?

I used SonarWorks True-Fi for a long time and with frequency correction, many headphone sound very similar tonally. However, even after correction an HD 800S e. g. is not comparable to a K371?

 

[maverickronin]

Are there any plans related to this: https://www.rtings.com/headphones/tests/sound-quality/passive-soundstage? Soundstage and Imaging differs a lot on different headphones and is not visible in the current data presented, right?

Amir can't do that with his current testing rig. The GRAS 45CA is basically just a box on pedestal with some ears on the side. He'd need at least a partial HATS like Rtings has in order to figure out the pinna transfer function.

 

[MayaTlab]

I used SonarWorks True-Fi for a long time and with frequency correction, many headphone sound very similar tonally. However, even after correction an HD 800S e. g. is not comparable to a K371?

It's highly doubtful that even after correction both headphones reach the same FR at your eardrum. There seems to be quite a bit of variation at both extremes of the FR spectrum across listeners. At higher frequencies, because of anatomical variations.
https://www.aes.org/e-lib/browse.cfm?elib=16877
You can possibly confirm that running sweeps, particularly above 5000hz, and for example note where the remaining peaks are located. There's a pretty good chance it's different, which would already give a clue that the correction didn't bring them to the exact same curve at your own eardrum.
Personally anything above 5000hz I prefer to EQ by ear, but I'm just not competent enough to even remotely reach parity across various headphones, current PEQ options are difficult to use at higher frequencies, and even changing the position of the headphones over your head can throw your EQ off.
It's probably better to start with a pair of headphones that sounds reasonably "right" to you past 5000hz already, it's easier that way. Personally that wasn't the HD800 .

 

[Racheski]

Weigh the headphones and compare them to manufacturer's spec. My HD800S weighs 370g without cables vs Sennheiser spec that claims 330g.

 

[pseudorndnbr]

Weigh the headphones and compare them to manufacturer's spec. My HD800S weighs 370g without cables vs Sennheiser spec that claims 330g.

Maybe Sennheiser doesn't include the weight of the pads?

 

[Racheski]

Maybe Sennheiser doesn't include the weight of the pads?

Could be...Not going to test because the pads are such a pain to take off and on They do mention that 370g is the weight without cables online, so it would be strange to not mention that the pad weight is excluded.

 

[RHO]

I don't know if this has been answered already: Why no IEM reviews/measurements?

 

[Cahudson42]

Hopefully not a complaint, but a 'suggestion'?

With what seems to me great ASR headphone test interest, and the large number of HP yet to be tested, why not start with an 'initial' test, followed when time permits, the full test?

Start with only 2 items: fr graph/test, and Distortion graph/test (%). That's it. No subjective/other - no PEQ tweaking - that's to be with the full test.

Just these two items for, example:
Ananda
Sundara
Deva/HE5XX
HE560s
He400se International
He400se China

Would be enough for many of us 'sitting on the fence' about 'which HIFIman' to be able to make an intelligent objective purchase decision.

Second, a periodic list of HP Amir 'has/has coming' would let us know which HP of interest still need to be provided..

 

[Racheski]

I don't know if this has been answered already: Why no IEM reviews/measurements?

They require a different test rig.

 

[Inazzab]

Suggestion:
Perhaps you could do some preliminary listening tests and EQing before looking at the HP measurements to eliminate bias?

If you expect there to be a frequency response error, then your expectations may lead you to perceive it regardless of its actual existence, or you may perceive the frequency response error as negative just because it has already been labeled as an "error". Similarly, EQ filters that you expect to be helpful may seem beneficial only because of your expectations (I know you individually test each filter on/off with eyes closed, but even so you can audibly tell when the filter is on/off, which may color your expectation of whether the effect on the sound is positive or negative). This expectation bias would be eliminated if you don't know the HP measurements beforehand.

I think this would help make the reviews more objective, which is something we can all agree is desirable!

 

[amirm]

They require a different test rig.

In theory it does not. In practice people use much simpler jigs to get better coupling. I will be trying this soon. Hopefully I don't have to spend $5K for a jig just for IEMs!

 

[amirm]

Suggestion:
Perhaps you could do some preliminary listening tests and EQing before looking at the HP measurements to eliminate bias?

How the heck you eliminate bias in a sighted test like this? I say be afraid, be very afraid of anyone who gives you subject assessments of speakers/headphones without seeing measurements, claiming that is better. They are giving you random opinion of some speaker's sound and who knows if that is accurate at all.

At least I have a compass to guide me as to what impairments I need to pay attention to. With them, you have to trust them blind that they have exceptional hearing and auditory memory to remember speakers from days and weeks past relative to the one tested!

I will do a video on this but it is a serious misconception to think I am giving you pure subjective listening tests results of headphones/speakers. I am not at all doing that. My listening tests are there to analyze audibility of individual tonality deviations as shown in frequency response measurements. As such, seeing the measurements is absolutely essential to what I do: give you reliable objective analysis of a headphone/speaker.

If you just want some online guy's subjective opinion of a speaker/headphone without any objective guides, the entire Internet/youtube is there for you. But let me caution you: a lot of these people, especially with headphone tests, peak at frequency response. No way would they know about the "200 Hz bump." And for the rest, I don't trust any of them.

 

[amirm]

Hopefully not a complaint, but a 'suggestion'?

With what seems to me great ASR headphone test interest, and the large number of HP yet to be tested, why not start with an 'initial' test, followed when time permits, the full test?

I find it hard to know how much emphasis to put on the measurements without listening tests to confirm. Headphone measurements are inexact. We need confirmation that what we have produced is directionally correct. I don't know how to do this without listening tests and development of EQ. The latter lets me turn individual filters on and off and determine the effect. That way I can tell if it is a major deviation or even a plus.

 

[paolomo]

No way would they know about the "200 Hz bump." And for the rest, I don't trust any of them.

Admittedly, that’s the region where I struggle the most to tell any difference (full disclosure, my ears are beyond untrained).

 

[Inazzab]

How the heck you eliminate bias in a sighted test like this? I say be afraid, be very afraid of anyone who gives you subject assessments of speakers/headphones without seeing measurements, claiming that is better. They are giving you random opinion of some speaker's sound and who knows if that is accurate at all.

I wholeheartedly agree that post-measurement listening and EQing are essential to having (mostly) objective and consistent reviews. However, also adding pre-measurement listening could eliminate (or significantly reduce) any measurement bias (even if it can't eliminate all other biases).

For example, you mentioned in your Focal Elex review that you often noticed that notes seemed very clean, and that this may have been because of the distortion measurements. If you had tested the Elex pre-measurement as well we would have been able to compare the pre-measurement and post-measurement impressions to see whether that cleanliness was actually audible.

 

[Robbo99999]

I wholeheartedly agree that post-measurement listening and EQing are essential to having (mostly) objective and consistent reviews. However, also adding pre-measurement listening could eliminate any measurement bias (even if it can't eliminate all other biases).

For example, you mentioned in your Focal Elex review that you often noticed that notes seemed very clean, and that this may have been because of the distortion measurements. If you had tested the Elex pre-measurement as well we would have been able to compare the pre-measurement and post-measurement impressions to see whether that cleanliness was actually audible.

The problem is that you get used to a new headphone's sound so quickly that you can't remember if it's accurate unless it has some strange frequency anomalies or is massively removed from your best target curve. I think one way of getting round that would be to do say a 5min listening session with your benchmark EQ'd headphone, say the HD650, then switch quickly to the new headphone to be tested and listen to it on the same track or two to create a quick impression....then do the measurements, then the main listening tests & EQ.....but that's just my idea.

 

[Dealux]

Will @amirm review his own Stax headphones? I'm really interested in that.

 

[John_M]

This has probably been said before but I think the focus on the frequency response target is too much about personal preference. One review states this:

"Dan says he doesn't think complying with the target generates what he hears live. I agree with him on that, but disagree on desirability of it in the arsenal of the audiophile. I like that energy there!"

It would be better in my opinion just to focus on the ability of headphones to replicate what is heard live, not to generate "energy" which the reviewer subjectively prefers.

 

[amirm]

Will @amirm review his own Stax headphones? I'm really interested in that.

I have gotten so berried in the last few months with loaned gear that I don't know when I will get to test my own bits like Stax....

 

[amirm]

First IEM test has been done: https://www.audiosciencereview.com/...anasonic-rp-tcm125-a-review-budget-iem.22797/

So far the results are very encouraging. If this continues, they are actually easier to test than headphones!