So, what does make sense to me?
Free-field (anechoic) measurement.
This would at least indicate how the device performs prior to convolution with the ear.
Show me frequency response, distortion measurements, usable range, etc. garnered from such testing.
With the knowledge of how only the headphone or iem performs, I can at least pretend to make an informed decision about quality.
If I have to mentally deconvolve frequency response from compensation curve, I'm just guessing. At best.
That is basically what I am doing. Measuring 'free-field' (but sealed and partially sealed) and somewhat anechoic from the side/distance where the actual sounds come from not hindered by Pinnae, just the speaker itself.
The bass correction I apply has little to do with HRTF but has other causes. HP50 is an example as when it is measured without bass compensation it has huge amounts of bass (+10dB) yet some people perceive it as 'realistic'.
The above picture is from
HERE and know no 'bass correction' has been applied for certain.
They measured
quite a few headphones but IMO their compensation around 3kHz is a bit 'off' and too much smoothing is applied to my taste.
As I don't have an artificial ear canal, which obviously has to be corrected perfectly afterwards anyway without success on the HATS measurements I have seen ?, the only 'corrections' I have to make is that of the mic on an infinite baffle.
This gives me plots that do not 'correlate' with (by me) perceived tonal balance.
I would agree that this is the 'raw' signal and would show equal 'loudness' over the entire audible range (and beyond) and makes sense as it is easier to compare if others also do not apply this.
However, those plots would only be interesting for those knowing what the plot means and would have to visually 'substract' their personal favorite 'compensation'.
I want a plot to show how it is perceived (primarily by me as at first I only measured for myself) and this simply does not jive with a flat measurement.
So.... I added the correction which works for me. A flat line gives me realistic sound with a full bodied but not bassy sound.
measuring IEM's is a mess. Insertion depth, seal and artificial ear canal shape and size would need to be standardized (there seems to be a standard for this) but then comes the correction part to 'translate' to perceived sound. That is a mess because all people insert and seal different.
I don't like shoving things in my ear so that's why I don't measure them.