Just stumbled on this thread from a couple months ago and thought I'd chime in.
There's a bit of snark from a few people here that seems... unnecessary to me. The OP seemed to be asking a reasonable question in good faith. Engaging with that question and "steelmanning" a possible argument against blind testing seems to me to be a worthwhile exercise from time to time.
In this case, I understand the OP's possible argument against blind testing to be, "in blind tests where people were asked to identify foods only by taste, without other indicators that we usually use to identify food, such as sight or touch, most people did extraordinarily badly at identifying even foods with very strong taste. Does that mean that isolating individual senses and removing other senses from the equation actually makes us poorer, not better, at using the isolated sense?"
Totally fair question. As a couple of other people have said in the thread, I think the important point where this doesn't analogize to blind testing in audio is the difference between identification and distinction. Ask people in a blind test to differentiate between a pickled herring and a strawberry, and I'll put $1,000 on 10 contestants scoring 10/10. Likewise, if you just walk me into a room blindfolded and play some music and ask me to identify the speakers or the amplifier, other than just throwing out random guesses, I'll never get it in a million years, even if you're playing music on speakers I'm intimately familiar with.
It is true that our other senses are extremely important to identifying things; however, those other senses can often override and fool us when trying to differentiate between very similar things. There's a famous example involving wine, where wine experts were served white wine with food colouring to make it appear red, and they all used words associated with red wine to describe the flavour. Would these wine experts have been able to differentiate coloured white wine from red wine in a blind test? Almost certainly. But the power of expectation, the power of their other senses, overrode or interfered with what their taste buds were telling them.
Give people in a blind test a glass of red wine, and a glass of white wine with food colouring, and most will be able to differentiate between them. Serve them a single glass in isolation, blindfolded, and they may do poorly in identifying whether they're drinking red or white. Serve people in a blind test two glasses of Merlot, one from a $15 bottle and one from a $30 bottle, and the overwhelming majority, even trained sommeliers, will be unable to tell the difference. What conclusion should we draw from that? I'd say the most reasonable conclusion is that there are meaningful differences between different types of wine, but the differences between different "qualities" of the same type of wine are largely attributable to presentation and expectation, and not to the wine itself.
Likewise for audio. Most of the distinctions people claim to hear in DACs, amplifiers, preamps, CD players, or other things that measure very uniformly, are due to expectation and presentation. In order to control for those other senses overriding our hearing, we need to test those devices in a blind/controlled manner.
