It is hard to accept as reliable this un-blinded comparison given the visual influence on the listener. Also, even if one accepts the described audible dichotomy between the two speakers, they afford no relationship to accuracy, just personal preferences as revealed by the chosen adjectives.
Yes, which is why I qualified my observations. And I also wasn't speaking to accuracy, only the apparent subjective effects.
Though this brings to mind the problem we enter when any reports, or even one's own experience listening to speakers, are dismissed if not done double-blind.
It seems to be one thing when talking about areas where the very technical claims are dubious - e.g. those behind any number of audiophile tweakdom like power cables. I would not trust anyone's anecdote in areas where the very audibility of a phenomenon is in dispute by people educated in the subject (e.g. EEs etc).
But it becomes a question of how far to take this. We don't normally reject one another's, or our own, impressions where there are good empirical reasons to think differences are discernible - e.g.
"are you SURE your orange juice actually tasted different than your milk? I'm not going to accept this unless you've done double-blind experiments." Yes, bias is always there, but it's also the case that there are objective reasons for why orange juice would taste discernably different from milk. Surely we would end up being crippled by such stringent demands on our experience and can provisionally accept claims of differences in every day life, where differences have some plausible basis.
So where do speakers fall in to this equation? Speakers are acknowledged to have objective differences in performance that correlate to detectable sonic differences. At least on the surface, it seems reasonable to relax the stringent demands somewhat on personal experience. In other words, if someone thinks X speaker sounded different from Y speaker....that's entirely plausible that they sounded different...unlike technically controversial claims for cables etc.
And then that person can reasonably say "I preferred X over Y."
But then it has to be acknowledge when we are getting really careful about our conclusions, that bias operates even at the level of deciding between items that do, in fact, sound sonically different! Which is the point you make, and the point made by research like Floyd Toole and HK's facilities.
So we don't escape bias effects even when comparing audio components like speakers that do have detectable sonic differences.
But, again, how far do we reasonably take the demands for blind testing? If we want to be sure of a claim to a scientifically acceptable degree....blind testing is a great tool. But how reasonable would it be to therefore dismiss any worth of an individual's experience comparing two speakers outside the lab? If we dismiss it all, it seems to me going too far. It's true that when I compare, say, a smaller Harbeth box speaker with big Avante Guarde horns, that some level of visual/conceptual bias may be playing part of my impressions. But it is also plausibly the case that the speakers do sound quite different, and in the ways I may detect and describe.
If we want to be so rigorous in assessing speakers that we simply can not rely on our own reactions and impressions to a speaker, that seems to be rather severe. It would seem we would be stuck only considering those speakers from companies - quite rare at the moment - providing spinorama-like measured documentation of their products. And so HK can tell us "trust us...we've done the measurements of our products vs competitors...you'll prefer our products, no need to bother listening to the competitor as we've done the technical work."
Personally, I don't like narrowing the field that much. I've encountered so many different types of speakers that I've really enjoyed, that would not necessarily come out on top in HK's blind testing facilities. And when I've auditioned the Revel speakers, I've found them very well designed and competent, but did not love the sound as I have any number of other speakers. So do I simply dismiss my own experience and say "even though I've listened to the Revel speakers and found myself not moved by the sound, I should just buy Revel anyway because they've done tests suggesting most people would prefer them under blind conditions?"
The blind tests give us statistical probabilities. On HK's work, I'm statistically likely to prefer the HK speakers over, say, the Devore speakers I *think* I enjoyed more. But then, I may fall within the statistics of those who wouldn't pick the Revels. I can't know unless I take a trip to Revel and do a blind shoot out between Revel and any number of other speakers I may be interested in. But that's frankly not a workable idea for the vast majority of us.
So my point being: I absolutely agree that if we want a scientific level of confidence about the type of impressions I've described...or simply in terms of comparing speakers in the consumer context, then we have to acknowledge the factor of bias and control for it as HK does.
But in terms of every day practicality, it seems reasonable to be less stringent and be able to talk with one another about what he heard from one speaker to another. Otherwise it would seem all intersubjective references and discussions about how speakers sound is to be dismissed or banned as inconsequential and disallowed.
"So you bought new speakers? How do they sound compared to your old ones?" "Sorry, can't have that conversation. I don't have access to a testing facility where I can do double blind comparisons of speakers." Which sounds like a world that really sucks to my mind.
Yes, I'm sure very few would take things to the extreme I'm envisioning...which is my point. Objective data is obviously very important. But there can start to be a sort of tyranny-of-measurements to some degree, where one's own input, or even technical claims, is never good enough as it can always be challenged on technical grounds. I believe you started to experience a bit of that on the AVSforum for your still enjoying and lauding of B&W speakers even though you are aware of the technical/scientific case made for Revel speakers.
"How can you say those speakers sound good - they don't measure great and excell in double blind tests like Revel!"