I feel this acutely since I frequently find myself in this predicament. It doesn't matter though. The burden of proof doesn't change just because it's hard to prove. If the prosecution lacks evidence the defendant walks.
A report of a subjective experience is valid with respect to that subjective experience. But when we know that cognitive bias is a huge confound, and the possible difference is very small, a sighted subjective experience tells us nothing about whether there would be audible differences under blind conditions.
Absolutely, the difference is that we don't know the confounding factors, and have no reason to think those confounding factors are large. It's possible that people think that they like more bass, and think that they like more treble, and maybe would have a different result if they were presented with different sound signatures to prefer rather than a dial that they could turn up or down. But that's rank speculation.
We're also talking about huge differences, with popular headphones often deviating over 10dB from each other at certain frequencies. So yes, I have no problem with Harman's preference research.
That's just not how it works. That opens up the door to all manner of specious claims, from fancy cables, to cable risers, to ethernet switches, to fuses, to repurposed radio shack clocks, to magical stones. These have all been marketed to improve audio performance, and they have all been purchased by satisfied customers who enjoyed the higher sonic fidelity they subjectively perceived. Pick the level at which you become incredulous.
Pragmatically speaking - yes I have measurement microphones, and I have done some measurements to check for setup issues etc...
But I did subjective listening tests of my various amp options, including bridged vs standard, and bi-amped vs standard, as well as comparing several different AVR's and power amps.
It was a purely subjective comparison (although at some point I will possibly redo some the bi-amp tests and measure response to see whether I am extracting a few more low end Hz by doing that...) - and my conclusion was, that most differences were sufficiently slight that I could not discount bias / placebo/nocebo effect in my conclusions.
But in a couple of cases, the subjective response was sufficiently pronounced to convince me that there is definitely something going wrong with that configuration. Specifically with the Integra DRX3.4 running solo driving my 4ohm nominal / 1.6ohm minimum speakers.... when the same setup had one of several power amp options driving the mains... the problems went away... and the problems weren't there when using my older SR876 AVR or DTR70.4 (both of which are very substantial beasts, with much more power, and massive power supplies).
So I make my conclusions based on my experiences as anyone else does... and provide advice to others based on my experiences.
The trouble with measurement centricity - is the standard sets of measurements don't provide us with a mapped measurement to perception relationship.... there is no measurement that will tell you how a speaker will Image, or how realistic the reproduction of female voice will be with a given speaker.... we can measure distortion of various kinds, dispersion patterns... but ultimately we just don't have a really good set of measures that tell us how a speaker will sound ... that remains quite subjective... (I pick on speakers as the most imperfect of the standard components! - hence providing the most obvious example)
So I can provide these data points
Running into my current speakers (Gallo Nucleus Ref 3.2)
Onkyo SR876 - sounded good
Quad 606 - possibly slightly smoother than the 876 (maybe a very slightly rolled off high end causing that impression?)
Crown XLS2500 - not as smooth as the 606, perhaps a touch more bass weight
Integra DTR70.4 - sounded the same as SR876
All the above I would rank as much of a muchness, and the differences could be caused by bias
Integra DRX3.4
Midrange muddled, imaging lost, soundstage collapse - something definitely wrong!
Integra DRX3.4 with external power amp (either Quad 606 or Crown XLS2500)
System is back to sounding as it should
Do I know exactly what is going wrong with the Integra 3.4 - not for sure... but it is an AVR with a relatively small power supply, driving a speaker with a minimum impedance of 1.6ohm.... sounds to me like the AVR is misbehaving into the low impedance load... (same test done with a more benign speaker load, and all the above examples sounded pretty much the same as each other, including the Integra 3.4)
So yes - subjective evaluation does have validity.... and I feel certain that I identified a case where an amp (AVR in this case) was running well outside of its designed performance envelope ... and sounded like it.
But I would very much like to get some more science into that conclusion - just don't have the tools and perhaps the knowhow to do that.
, I recommend you learn what is actually going on.
