We know that bias is hovering over virtually everything we perceive. (Folks on other audiophile forums get annoyed by my bringing up the problem of sighted bias all the time!). The issue is a need to keep perspective: how far is it reasonable to push skepticism on what can be ascertained under uncontrolled conditions?
At some point along the continuum between "totally unreliable" and "very reliable" we will find some middle ground of "reliable enough" to come to pragmatic (even if tentative) conclusions. We do this all day long. As I keep pointing out, the only reason my efforts "work" in my job is that there is some level of predictability in how other human beings will perceive the sound I am creating. The general success of the work would be inexplicable on the hypothesis that sighted listening conditions so derange perception that it becomes too unreliable to predict. (Much of my job is manipulating perception...in a reliable manner).
Similarly, if you took two neutral speakers and equ'd a lift between 2 and 4K,
at some point as you keep increasing the difference, it is highly likely it will be reliably identified in both sighted and blind conditions. One being reliably identified (by audiophiles who can still hear those frequencies!) as brighter than the other. Again, if this weren't the case, all the sonic manipulation mixers and engineers do - including EQ and mastering - would be irrational and uncesseful.
Now, when it comes to the specific study of speakers, in which subjects preferences changed from sighted to blind conditions, the one I keep seeing cited is this one, discussed in Sean Olive's blog:
An ongoing controversy within the high-end audio community is the efficacy of blind versus sighted audio product listening tests. In a blind...
seanolive.blogspot.com
It's the original 1994 test set up by Olive et al at Harmon. Admittedly, while I have seen a lot of Toole's writing, watched his talks etc, I haven't read his book yet, so I don't know what else he has in there in terms of specific blind vs sighted tests like the one described above. I note in his terrific, much watched talk on youtube, he cites the same study as Olive above when talking about the problem of sighted listening.
So looking at the results:
The point Olive and Toole will make from this test is how you can see some distinct changes in ratings between sighted and blinded listening, as well as
how the sighted listening impressions seem to also overwhelm the sonic changes even in changing the speaker position. All well and good. But is that the ONLY thing the data seem to show? While, yes it shows variation from sighted to blind ratings - anyone with any familiarity with psychology would predict that! - does it completely trash sighted perception? For speaker position 1, there is real variability. But for loudspeaker position 2...much less so. For speaker position 2, some general trends hold from sighted to blind listening:
For Speaker Position 2, in both sighted and blinded conditions:
1. G and D are rated higher than S and T
2. D rated very best
3. S rating is pretty consistent
4. T shows the most variability, though still rated below G and D
So, loudspeaker position 2 seems to have some effect on the reliability of the ratings - general trends similar in sighted to blind listening are there. I'm curious: What would a bunch more studies with speaker in position 2 show? Would we see more of these sighted listening trends carry over to blinded conditions? (If that was followed up, I'm all ears).
Another graph is, I presume (and would like to be corrected if wrong) a summary of the trends found in the test (presumably including both speaker positions?)
If I'm interpreting the graph correctly, it tells us that, despite some differences, everything taken together the trend is that G and D were consistently rated best, and with a similar difference in rating, S suffered the least correlation, T a relatively strong correlation. (Again, please correct my understanding if wrong).
If so, even this study, often cited to trash sighted listening still suggests some correlation between sighted impressions and "sound only" impressions.
Which wouldn't be a total surprise: our hearing evolved in sighted conditions, so there has to be SOME level of reliability in sighted conditions.
I have also on this forum cited numerous times where the sonic impressions of audiophiles and audio writers tracked well with the measurements of a speaker.
It's always *possible* that we can be imagining things. Even Big Things. As I've often said on other forums: People have imagined they are probed by aliens. You don't think an audiophile can imagine a reduction of "midrange glare?" when he tries a new AC cable?
But hyper-skepticism would be paralyzing and untenable. We could set up countless tests that show our perception of sound to be very reliable. If I test whether you can tell the difference between a dog bark and a cat meow, or between the voices of two people you know well, you'd be likely, like most people, to reliably score %100 correct. Your blinded test will track perfectly with your sighted tests. Similarly, when a pair of speakers diverge *enough* in sound - say in the piercing quality of the high frequencies, or the depth/pounch of the low frequencies, then in all likelihood people would be just as accurate identifying the differences under blind conditions, as they heard in sighted conditions. The question becomes where do we approach this middle ground were it is reasonable to say "Yeah, I probably am identifying sonic characteristics between A and B, even in sighted conditions."
You beat me to it!
Yes, lots of gray area. That's why at some point we have to acknowledge when it's reasonable to come to conclusions even though we haven't scientifically controlled for biases.
"Yeah, it's possible what I'm perceiving is skewed via a bias effect, but it's also plausible enough for me to proceed on the conclusion the differences are real."