1) Comparing several measurements from different sources
Which circles back to the question of sample variation. How do you make sure that the set of measurements you pick will correspond to your sample ? Thankfully some sources will occasionally measure several copies, but it isn't, to my dismay, systematic.
This is three samples of the Moondrop Chu, six earbuds in total, after normalisation :
Perhaps I've been lucky (I mean, all of them had their filter in place), but that's quite reassuring and, if continued with additional samples, is definitely the sort of thing where I'd be confident to dismiss sample variation as a significant concern when using third party measurements.
But it's not quite as pretty for some of the other IEMs for which I've been able to measure several samples. Or even between the left and right channels to be honest.
2) EQing IEMs to a specific FR. For example comparing IEM 'A', which was EQed to 'B' with the original 'B'.
With the same tips as the ones used during the measurements, or the same tips used for both of your own samples, when listening to them ?
I think that the choice of tips and insertion depth / position can go from a rather benign problem to a tricky one. Here's a rather interesting bit that I stumbled upon recently. The Etymotic ER2SE is now sold with clear transparent triple flange tips that can easily change the FR when inserted in a straight 711 coupler clone in rather surprising ways, even when a) no leakage is present, b) the tip's flange is always positioned in the same way inside the coupler, c) the response produces a resonant peak at a similar frequency, and d) for most of them that response could be replicated at will with several reseats (these are averages).
For some of these traces the only thing I did was either pulling or compressing the tip :
4) Theoretical reasoning, besides the ear canal resonance, there is simply no other interaction from the sound source to the ear drum, so no room for mysteries happening along the way.
Sam ex-from Rtings shared the comparison he made between the 5128 and Rtings' HMS. This is the data for in-ears :
You basically get a trend with lots of noise around it. That upwards trend (with a hump around 800Hz and / or a low shelf below 500Hz or so) is similar to the one you'd get from a database like
Hypethesonics (5128 vs GRAS). It isn't exactly clear what could have caused all that noise around the trend (could be leakage, or something called "acoustic impedance" of which I'm absolutely not qualified to talk about, effective insertion depth, etc.), but I'll give you one that neither relies on eartip nor on leakage.
I've started dabbling in measuring in-ears with a DIY probe microphone inserted through a single eartip (that happens to work for several IEMs), and compared the measurements with ones made inside a 711 clone coupler (from Aliexpress), still using the probe's mic - all the while also measuring the IEMs from the coupler's own mic at DRP :
On the plus side it's reassured me that for the few passive dynamic IEMs that I've compared in this way, not only can the probe inside the coupler measure the same relative difference (difference in FR between headphones A and B) as the coupler's own microphone at DRP, at least up a few kHz, the in-ear measurements also match these relative differences very well -
at least as long as the eartip remains a constant. So if I like how one of these passive IEMs sound, I can EQ the others to it per the coupler's measurements, and get the same FR (if I use the same eartip, I don't know yet otherwise). Above several kHz, more difficult to say. And in regards to the absolute values... well let's just say that I believe that in the end, when plotting the in-ear measurements over the in-coupler measurements, I'll obtain something not too dissimilar to Rtings' trends above, on average (not surprising given what we already know of the 5128 vs 711 couplers and the few in-ear IEM measurements that have appeared).
This all deserves a much longer post to be honest, but here's a rather interesting bit : measuring the difference between the probe inside my ears and the probe inside the coupler with two ANC IEMs (Anker A40, JBL Live Pro 2)
with a feedback mechanism, both with ANC on and off :
"711P" above means : measured from the probe inside the coupler, while "EAR" means : measured from the probe inside my ears. I made sure that the difference between ANC on and off was measured similarly by the probe inside the coupler as the coupler's own mic at DRP. The IEMs were inserted in the coupler to produce a resonant peak around 8kHz, but these trends are quite resistant to varying insertion depths (the SPL when ANC is turned off does not vary as much with insertion depth as the difference you see above). Of course in my ears the IEMs are just shoved as far as they'll go with that eartip (that won't produce a resonant peak at the same frequency, but that's exactly the sort of problem you're facing anyway when EQing IEMs based on ear simulator measurements).
These traces were not normalised at all, and all measurements were taken at the exact same volume, whether in my ears or the coupler.
The perfectly flat lines aligned at 0 up to a few hundred Hz when ANC is enabled should give a hint as to what is happening I think : regardless of how the IEMs behave in my ears vs. the coupler, the feedback mechanism clamps down on any difference in the range where it operates (up to a few hundred Hz).
I'm glad that EQing your IEMs per other measurements made with different samples have worked out fine for you, but unfortunately for me that hasn't been the case, at least at higher frequencies for passive IEMs, and throughout the spectrum for active ones with a feedback mechanism. Let's just say that I'm quite happy to have my own coupler and in-ear measurements as useful assists to EQ headphones to satisfaction.
Not that I want to stomp on measurements' usefulness, far from it, but neither am I in the camp "yeah they're good enough to EQ one IEM to sound like the other".
