Compensation curves and non standard couplers. Don't take squig results at face value, as most don't show the raw result or don't own genuine 711s or GRAAs 43AGs. You can trust Oratory (43AG), Crinacle (BK 5128 and 43AG) not the old "clone"), Amirm (43AG), Headphones.com (when they are using the 43AG or BK 5128, that is, mostly on
@Resolve s reviews) and Sai's from sai.squig.com (43AG). With others, you'll have to filter the compensation's used and the disclosured equipment.
View attachment 446796
EDIT: Given that we're talking about Over-Ear headphones, these precautions are even more valid as there will be another variable: the ear used itself (literally, the earpiece that simulates a human ear). GRAAS and BK couple (heh) their measuring rigs with ears of their own, but for the "711" couplers reviewers have to default to known solutions like the Kemar KB0066 (which is the one SuperReview uses), but there are other ones, and even clones of the former that aren't 100% compliant to the standard.
All good info, and I'd like to add a few things:
1.) Headphone Position: For headphones, position in relation to the pinna can result in different measurements. Typically, there is room for it to be slid forward, back, up, or down and and that can all affect the frequency response. That, coupled with the fact that there is no one standard position, can result in fairly different measurements.
2.) Pad Issues. There are obvious things such as not getting a seal. There's also the fact that some reviewers don't mention which pads they are using, and that can vary the measurements. The less obvious thing is the age of the pad. With age, the foam in some pads may be less resilient which could result in the headphone driver being closer to the head, resulting in a change in frequency response. I would assume that most reviews have newer pads, but it can be a variable, particularly if the headphone sample is an older one. Also, variability in the pad construction can factor into it as well.
3.) Test Rigs. This adds onto the above post, but there is variability even between known, qualified test fixtures. I have two GRAS 45CAs (one with RA0045 and one with RA0402 High Frequency ear simulators), a B&K 4128 and a B&K 5128. The B&K 5128 obviously measures quite different than the other three, but there are differences between the 45CAs/4128 as well that is a reminder that one does need to be a bit cautious when analyzing frequency response measurements of a single headphone on a single rig. And for every test rig that I have, there are other pinna options as well as other choices that can result in a different result.
Measurements are really important (I couldn't do design work without them), but it is important to understand that there are a LOT of variables at play here. Amongst them, are the individual headphone unit variability (drivers, pads, etc), test system variability (ear simulator, head and torso used, pinna, etc), positioning differences, calibration of the test system, and the skill of the person doing the test. There's also the variability in ear canal resonance, the HRTF and how a headphone physically fits on an individual. So, by all means review the measurements (I certainly do), but it's also best to listen if that's a possibility. There's no guarantee that something that works for one person will work for another, and vice versa. Which is why you may see a positive review in one place and a negative review in another. There's no getting around individual preference.