I conducted two "unsighted evaluations" in the past; here is a short story on what went down on one of them;
I am part of another audio-site called CAM. Perhaps ten years ago, there were a few RCA patch cable "manufacturers" on the site offering to freely pass around cables for folks to audition. The prerequisite to participate was that one needed to have been a member of the forum for a year or more, as I was.
A set of "pure silver / super-shielded" cables, retailing for about $350 CAD (don't recall the exact amount, but $350 CAD or thereabouts) was up for loan, so I put my name on the waiting-list, eventually receiving the cables and using them for an unsighted audition.
I put them up against what I can only describe as "The FrankenCable From Hell". I picked up a flimsy parallel-bonded RCA patch cable that shipped with an old VCR player back in the day, and cut it with a pair of scissors into three separate lengths. I crudely stripped the wire back on all ends and twisted them together, then scotch-taped the exposed wire. That was about as crude and ugly of a cable-competitor as I could come up with for the pure silver / super-shielded RCA patch cables - which many audiophiles on the CAM site had auditioned prior, and were lavishing with praise.
The FrankenCable was so thin and crudely reassembled that even I was thinking that it might negatively affect the signals passing through it…
The listeners: I elected to invite four audiophiles, plus myself to audition the cables, in two separate listening sessions. We collaborated on what music they wanted to use for the audition, and picked a selection of the most familiar recordings; all on CD and SACD, some using analog masters, some pure digital.
One of the audiophiles was part of the local symphony choir, the other three listeners were local audiophiles with stereo system worth more than my car, and were confident that they could definitely hear a difference in RCA patch cords. Especially when they saw the FrankenCable!
There were two sessions of listeners; early afternoon and late afternoon, with two groups of two, plus myself. Each of us had notepads, each had excellent seating, and each was able to choose the music that they wanted to use during the audition.
The Listening Room: I lightly treated the listening room for early reflections and bass-emphasis. Listeners were able to audition music at leisure prior to the listening session. We shared a glass of wine, but not enough to inebriate, and chatted about all-things-audio.
The Reproduction Hardware: I set up a pair of Magneplanar 1.6QR loudspeakers in the listening room, per mfg. recommended positioning specifications, and ran 12-gauge copper speaker wiring (mfg. by AR) from the speakers, under a door, and into a separate room with a fully closable door. The listening room also held a pair of Paradigm PDR-12 powered subwoofers, run at a barely perceptible output level and only producing output below 50Hz.
The separate room / behind the closed door, held the stereo system consisted of a Roskan Kandy integrated amplifier and a Pioneer Elite SACD player. We agreed upon a position on the volume control that was to be used during every playback. The patch cords under evaluation were run between the CD player and the amplifier.
The Infrastructure Hardware: I ran a separate 120 volt line from the power-panel to the room which held the stereo gear. I replaced the standard / generic 110 volt wall-plugs with "audiophile-grade" / "hospital-grade" receptacles. I used the stock power-cable that shipped with the Elite SACD deck, but employed a massive, 'arc-welder-thick' "audiophile" power cable for the integrated amplifier (that thick “audiophile” power-cable became part of a separate blind evaluation at a later point).
The Process: We all talked over how the process was to run, and conducted several dry-runs; in a nutshell - two listeners would choose a song, and the third listener would become the stereo operator behind closed doors. After a full song, played twice, we'd meet in the main room and chat, take notes and then swap who would be the next stereo operator. The job of the stereo operator was to change cables and music. One would play a track, then swap cable, and play the same track again, noting which cable was in place and when. Sometimes the cable was not swapped at all, which was also noted.
At the end of the session we compared notes.
As you might imagine, the results were completely random. The only person that came even vaguely statistically close was me. I guessed correctly somewhere in the neighborhood of 65% of the time. But, in all honesty, I was simply guessing, as I had absolutely no idea which cable I was listening through at any point.
At the end of the second session the two listeners asked me to go back into the room and install the pure silver / super-shielded RCA patch cables. I dutifully went back into the separate room and inserted... the FrankenCable. I couldn't resist.
Sure enough, when the beautiful voice of Norah Jones swooned out of the Maggies, via the FrankenCables, my two listener friends began to wax poetically with statements like "ah, yes, that's it, listen, I knew that these silver cables sounded better, the sweetness of the sound, the roundness of the notes, it’s just fuller sounding..."
These were two audiophiles (one was the symphony orchestra player, the other a fairly well-to-do audiophile) who had just found that they could not tell any audible difference between absolute junk RCA patch cables and audiophile overkill pure silver / super-shielded RCA patch cables moments prior - when they could not see which cable was in play.
But once they were sure that they knew what cable was in place - because they had requested it - they were suddenly blessed with golden ears ~ despite having actually just gushed over the FrankenCables.
This, my friends, is what we face in trying to educate audiophiles about snake-oil.
I conducted a blind evaluation with 120v power cables a few months later, with a different process, but the results were just more of the same.
The results of these unsighted evaluations made me somewhat of a Persona non grata on the CAM site for quite some time
Andrew