• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

ASR sponsored double blind test event

Punter

Active Member
Joined
Jul 19, 2022
Messages
285
Likes
1,329
Outside the Amazing Randi, I'm struggling to find any sort of definitive attempt to run a substantial double-blind test event for audio equipment. By that I mean an event that is set up specifically to test disputed audible effects of various devices, accessories and components.

Of course the prime target here would be cables. The thing about cables now is that there's several types to choose from as opposed to earlier times when it was only speaker cables. The next in line were interconnects but now we have mains cables, ethernet cables, USB cables and even optical interconnects.

What does the forum think of maybe a GoFundMe to set up a properly controlled double blind event? Based on Amir's access to many snake-oil products in the past, it shouldn't be too difficult to assemble a proper A-B or A-B-X setup.

The most important element however, will be the attendees. I wonder if we could attract some "names" from the high-end audiophile reviewer ranks? There's more than enough tech knowledge in the membership to design a suitable test rig, hopefully with some hifi equipment that would satisfy the reviewer set as being "resolving" enough.

Thoughts?


hq720.jpg
 
Cables CAN make a system sound different and is easy to prove with measurements why that would be the case.
Cable resistance and a varying load impedance are the cause.

So ... if I use 2 extremely different cables on say... a Magnepan with flat ohmic impedance across the audible band there won't be any differences except for overall level due to voltage division.
If I use the same cables on a speaker with a wildly varying impedance and one of the cables is high resistance they can be picked out in a blind test.

If I use 2 different cables where one has an unusual high capacitance and the other does not and use an amplifier that becomes unstable than possibly there is an audible difference.

There is the crux. There is an infinite combination from amps, cables and speakers and some of these combinations can cause audible differences.
It is not the cable that sounds different though its the circumstances that can cause that.

Such a proposed test would only be valid for the used amp, cables and speakers and thus cannot end the debate. It could at best settle it for that combo with the chosen recordings in that room with the available ears.

Doomed to fail experiment.
 
The problem is, that at an event the listeners are not experienced in doing testing for Just Noticeable Differences. They don't know what to listen for and they stat with products that really sound and measure the same.
Meanwhile in the pro area, trained / experienced listeners can hear extremely small differences.
 
hi end Cables, DAC's, streamers and ethernet network devices... these seem to be the areas with the most fud.

There still is a place for expensive class A and A/B amps (up to some reasonable $ value) and of course speakers (they kinda go hand in hand... big speakers need big amps).

Lots of issues, the least of which is procuring the high end gear... which would have to be on loan from dealers or some magic ASR benefactor.

Which dealer would lend you this stuff if they knew the purpose was to debunk that the stuff they sell is snake oil.

Same with establishment reviewers... they would lose their "street cred" if again it was found the stuff they had been touting for years was snake oil.

Finally, who are we trying to convince?. The believers in the hi end would just say "Look its those weirdos from ASR... nothing to see here"



Peter
 
What does the forum think of maybe a GoFundMe to set up a properly controlled double blind event?
I can think of a long list of causes I'd rather put money towards instead of the "Save people from buying fancy cables" fund.
1734646243063.png
 
It would be interesting for some of us. Those who are already certain they are hearing differences would not be dissuaded by their inability to tell the difference in a double blind test. They would insist that the equipment wasn't set up well enough, or that the test itself was too stressful to be able to really relax and hear the subtleties.
 
ethernet network devices... these seem to be the areas with the most fud.
As someone with more than 20yrs in the networking industry including working most of that time for the largest 2 manufacturers, this one drives me up the wall…. :banghead: now :facepalm:.
 
It would be an enormous effort for most likely an outcome that many can predict. After it is shown that no one can reliably tell if there is any difference between various tests, it just gets tossed into the trash bin of history, as another test showing that people can't hear what they think they can hear. Now saying that, I don't want to be a wet blanket on the idea. If you can pull it off, it would at least be a new test. The logistics is formidable. Most testing that I was involved with was paid for by very large corporations and done by engineering firms that specialized in sound systems. So, they had near unlimited funds to do all the testing, get all the equipment and set it up etc. Then you have to get people to show up for listening. But, if you can do it the results should be very interesting! Go for it! :)
 
As someone with more than 20yrs in the networking industry including working most of that time for the largest 2 manufacturers, this one drives me up the wall…. :banghead: now :facepalm:.
I have learnt to shut my mouth, smile, and move away.

So happen yesterday I had the opportunity to listen to a million dollar horn system. The phrase “audiophile switch” came up several times. There was also mention about $100k grounding box in the mix. I smiled and kept quiet.
 
The problem is, that at an event the listeners are not experienced in doing testing for Just Noticeable Differences. They don't know what to listen for and they stat with products that really sound and measure the same.
Meanwhile in the pro area, trained / experienced listeners can hear extremely small differences.
My proposal includes some individuals from the "golden ear" reviewer brigade (if they can be tempted to participate...). Surely they would qualify as trained/experienced listeners? After all, they're the ones claiming to hear "differences" and promoting the products.
 
As far as convincing people is concerned, I'm not concerned with that. I'm more concerned with getting something on the record that's been done using sound engineering principles. I don't believe this needs to be incredibly expensive or complex either. Probably the hardest thing to achieve would be finding an appropriate, sound treated room to run the test in. The test groups would be necessarily small, maybe an individual at a time. It would be interesting to see the reaction of the golden ear types to an offer like this too :)
 
Thoughts?

A controlled test?

The desired attendees probably already know what the statistical outcome would be, and that wouldn't look good on their resumé, so, why take the risk?

What's the reward?
 
Our problem as objectivists is that we can not prove a negative. The reason we know we can't hear a difference between cables is because (1) it is theoretically implausible, (2) repeated measurements show the difference to be way below the limits of audibility. These two alone should be enough to dismiss the idea that cables make an audible difference but some of us go as far as (3) conducting blind tests which never show a difference.

You can conduct as many blind tests as you want, but people will always find a way to dismiss your blind test if they want to cling to their beliefs.

I am much more interested in quantitative blind tests that asks the question "what level of X is audible". For example, how much distortion / group delay / MP3 compression / noise / etc can be added before listeners can reliably pick a difference. Do diffusers make an audible difference? Or do they improve the appearance of the ETC without actually improving the sound (as asserted by Toole). Or another type of blind test, whether X or Y is preferable because I suspect that people like a bit of distortion, or that some nonlinear frequency response might sound better to some people. Or maybe settle once and for all the "less reflections vs. more reflections" debate. Lots of people on ASR say that the goal of hi-fi is "high fidelity" which is music reproduction without adding "character" of its own. Is it really? Let's blind test that and find out.

There are plenty of things to blind test, but cables are not one of them. Cables are a settled debate. Anybody who still thinks that cables make an audible difference is on the shrooms and can be safely dismissed.
 
A controlled test?

The desired attendees probably already know what the statistical outcome would be, and that wouldn't look good on their resumé, so, why take the risk?

What's the reward?
Instant 1000 Likes at ASR :)
 
I’m much more interested in blind speaker testing, but I know that’s a lot harder to pull off.
 
I conducted two "unsighted evaluations" in the past; here is a short story on what went down on one of them;


I am part of another audio-site called CAM. Perhaps ten years ago, there were a few RCA patch cable "manufacturers" on the site offering to freely pass around cables for folks to audition. The prerequisite to participate was that one needed to have been a member of the forum for a year or more, as I was.

A set of "pure silver / super-shielded" cables, retailing for about $350 CAD (don't recall the exact amount, but $350 CAD or thereabouts) was up for loan, so I put my name on the waiting-list, eventually receiving the cables and using them for an unsighted audition.

I put them up against what I can only describe as "The FrankenCable From Hell". I picked up a flimsy parallel-bonded RCA patch cable that shipped with an old VCR player back in the day, and cut it with a pair of scissors into three separate lengths. I crudely stripped the wire back on all ends and twisted them together, then scotch-taped the exposed wire. That was about as crude and ugly of a cable-competitor as I could come up with for the pure silver / super-shielded RCA patch cables - which many audiophiles on the CAM site had auditioned prior, and were lavishing with praise.


The FrankenCable was so thin and crudely reassembled that even I was thinking that it might negatively affect the signals passing through it…


The listeners: I elected to invite four audiophiles, plus myself to audition the cables, in two separate listening sessions. We collaborated on what music they wanted to use for the audition, and picked a selection of the most familiar recordings; all on CD and SACD, some using analog masters, some pure digital.

One of the audiophiles was part of the local symphony choir, the other three listeners were local audiophiles with stereo system worth more than my car, and were confident that they could definitely hear a difference in RCA patch cords. Especially when they saw the FrankenCable!

There were two sessions of listeners; early afternoon and late afternoon, with two groups of two, plus myself. Each of us had notepads, each had excellent seating, and each was able to choose the music that they wanted to use during the audition.

The Listening Room: I lightly treated the listening room for early reflections and bass-emphasis. Listeners were able to audition music at leisure prior to the listening session. We shared a glass of wine, but not enough to inebriate, and chatted about all-things-audio.

The Reproduction Hardware: I set up a pair of Magneplanar 1.6QR loudspeakers in the listening room, per mfg. recommended positioning specifications, and ran 12-gauge copper speaker wiring (mfg. by AR) from the speakers, under a door, and into a separate room with a fully closable door. The listening room also held a pair of Paradigm PDR-12 powered subwoofers, run at a barely perceptible output level and only producing output below 50Hz.


The separate room / behind the closed door, held the stereo system consisted of a Roskan Kandy integrated amplifier and a Pioneer Elite SACD player. We agreed upon a position on the volume control that was to be used during every playback. The patch cords under evaluation were run between the CD player and the amplifier.


The Infrastructure Hardware: I ran a separate 120 volt line from the power-panel to the room which held the stereo gear. I replaced the standard / generic 110 volt wall-plugs with "audiophile-grade" / "hospital-grade" receptacles. I used the stock power-cable that shipped with the Elite SACD deck, but employed a massive, 'arc-welder-thick' "audiophile" power cable for the integrated amplifier (that thick “audiophile” power-cable became part of a separate blind evaluation at a later point).

The Process: We all talked over how the process was to run, and conducted several dry-runs; in a nutshell - two listeners would choose a song, and the third listener would become the stereo operator behind closed doors. After a full song, played twice, we'd meet in the main room and chat, take notes and then swap who would be the next stereo operator. The job of the stereo operator was to change cables and music. One would play a track, then swap cable, and play the same track again, noting which cable was in place and when. Sometimes the cable was not swapped at all, which was also noted.


At the end of the session we compared notes.

As you might imagine, the results were completely random. The only person that came even vaguely statistically close was me. I guessed correctly somewhere in the neighborhood of 65% of the time. But, in all honesty, I was simply guessing, as I had absolutely no idea which cable I was listening through at any point.

At the end of the second session the two listeners asked me to go back into the room and install the pure silver / super-shielded RCA patch cables. I dutifully went back into the separate room and inserted... the FrankenCable. I couldn't resist.

Sure enough, when the beautiful voice of Norah Jones swooned out of the Maggies, via the FrankenCables, my two listener friends began to wax poetically with statements like "ah, yes, that's it, listen, I knew that these silver cables sounded better, the sweetness of the sound, the roundness of the notes, it’s just fuller sounding..."

These were two audiophiles (one was the symphony orchestra player, the other a fairly well-to-do audiophile) who had just found that they could not tell any audible difference between absolute junk RCA patch cables and audiophile overkill pure silver / super-shielded RCA patch cables moments prior - when they could not see which cable was in play.

But once they were sure that they knew what cable was in place - because they had requested it - they were suddenly blessed with golden ears ~ despite having actually just gushed over the FrankenCables.

This, my friends, is what we face in trying to educate audiophiles about snake-oil.

I conducted a blind evaluation with 120v power cables a few months later, with a different process, but the results were just more of the same.

The results of these unsighted evaluations made me somewhat of a Persona non grata on the CAM site for quite some time :)

Andrew
 
Back
Top Bottom