I have measured drivers in free field as well - it's an interesting opportunity to look at their behavior "ceteris paribus", and IMO quite worthwhile in the process of driver selection and design. The congruence of free field measurements and a coupled measurement omitting a pinna, however, will depend in part on the construction of the front volume of the headphone (inclusive of the earpads and any space between the driver's front output and the pads) - I would imagine that if you measured in free field with the earpads, you would get something closer to what an earless flat plate shows, but this of course would have the same issues.
Variation in response above 1khz or so has a few main components:
* positional/placement variation with the headphone itself, which generally rises as a function of frequency from the midrange, but is more extreme for some models than others
* when different units of a given headphone are used, the variation between the units in question - this isn't just a question of manufacturing tolerances, even small differences in pad wear can be quite substantial in frequency response impacts
* variation in the compensation applied (ex. all HRTFs will differ in the >3khz band, so if one plot is DF compensated and another is free field 0-0, you will see very large differences from this alone), including the way that the compensation data was derived (a 1/3oct smoothed population average free field 0-0 HRTF isn't going to spit out the same output as an individualized 1/12oct smoothed one)
* variation in the HRTF of the measurement systems, although in premise barring "individual hrtf-headphone interactions" this should be cancelled by the use of a standardized compensation that is specific to the measurement system in question.
Human HRTFs have a very broad spread in the 4+khz band, so looking at raw responses, it is unsurprising when variation occurs in this area; indeed, the mannequins we use have lower variation due to their intention as population averages. Consider the spread observable in 40 human DF-HRTFs from
Hammershøi & Møller's Design Criteria for Headphones (1995):
View attachment 77964
Human head/ear anatomy is simply rather variable, yielding differing HRTFs for different people in the same circumstances... including, as
their other neat paper from 1995 shows, headphones:
View attachment 77968
Somewhere up-thread I'd written about my complaints regarding subjective matching between different apparent acoustic sources, so I'll leave that aside for the moment, but I have to ask, what are we to compare to with "what we hear" even at the best of times? Ignore the SLD effect and all that, my speakers - or Amir's speakers, or what have you - are the definitive reference for headphones? But what's the reference for my speakers? I'm rehashing
Olive, as always, here, but it has a lot of pertinence when you're thinking about what we hear, what we measure, and how they "should" relate.