If one is not careful, there is a conundrum produced if you derive a too-extreme stance based on the blinded research. (And I'm specifically
concerned in this case with speakers - where we know the differences are audible).
By "not careful" I mean believing things like "
sighted/uncontrolled impressions of speakers are wholly useless, uninformative, worthless" etc.
(Here I'll be using "sighted" impressions to mean non-blinded/not under scientific-level controls).
So here's the conundrum:
1. If sighted bias so reliably distorts our perception of the sound of speakers, then the blind tests seem to have little relevance to the real world conditions in choosing and listening to speakers, which will be done sighted. Our biases will distort the sound so who cares what it sounds like under blinded conditions?
BUT...
2. IF the blind tests for sonic quality are to have any relevance to our normal listening situation, you'd have to make the case that THE SOUND somehow predicts some level of listening satisfaction, that it carries over to sighted listening. So, somehow the specific pleasing sonic characteristics perceived under blind conditions will also be recognized under sighted conditions!
But to make the case for the relevance of blind tests in #2, you've just accepted *some* level of accuracy for sighted listening!
In order to hue to an extreme view against the worth of sighted listening, it seems you either throw out the relevance of blind testing (for audibly different devices like speakers!) to our actual listening conditions. OR you say the blind tests allow us to discern qualities identifiable under sighted conditions too, in which case portraying sighted conditions as utterly unreliable or worthless is incoherent.
These problems don't attend a viewpoint that isn't extreme, where one isn't throwing up a false dichotomy of "Reliable" (scientific controls) or "Totally Unreliable/worthless" (no scientific controls), but rather take the more nuanced stance More Reliable (scientific controls) vs Less Reliable (uncontrolled listening).
It's totally rational for anyone to want "More Reliable" data, and if they want to look to blind testing for such info. What's not rational is to think that "less reliable" conditions equate to "useless/of no worth/completely unreliable." Don't go extreme, and no conundrums arise.