A USB Cable Test Designed to Satisfy Skeptics AND Believers—Feedback Welcome

[SPOautos]

I agree there is a subset of people who won't be convinced. I actually deal with some anti-vax people in my day job, so that is partly why I think transparency and input at the test design stage are so important. If we use a public comment period to allow people to have input on the test before we conduct it, some portion of the non-believers will be more open to the results.

I wouldn't relate this to 'anti-vax' at all....more like flat earthers. Anti-vax actually has some merit too it, it's already been proven to create health issues in some people as well as the untrusted quick process and lack of manufacturer liability.

What you're speaking of is more like flat earthers where the only reason to believe such a thing is just because it's what they desire to believe.

 

[Roland68]

The question is who needs the research. As you can imagine, our camp doesn't need it.

The camp that needs it doesn't believe in any controlled testing. You mention any protocol and the game is over.

Let me give you some evidence of how hard it is to convince the diehards in the other camp. A local audiophile has a system that by now is clocking close to a million dollars. He believes in every and all audio tweaks, no matter how remote. He once said not only did new outlets improve the sound of his systems, but actually changing the covers on the outlets helped remove another veil!

About 10 to 15 years ago, the above audiophile agreed to participate in a test of his MIT cables against another cheap cable, in his room and his system. The objectivist who conducted test performed it blind. After a few trials, the above audiophile agreed that he was not able to detect his expensive cables in blind tests even though he was so sure he could. This was all published in another forum and said audiophile said he had some reflections to do.

Well, he reflected alright. After a short while, he simply went back to his old ways, throwing everything at his system, including said outlet upgrades. If you as much as mention the word "measurement," he will throw you out of the room. "His ears tell the truth" and that is all there is to it.

Now, the above used to be limited to uber audiophiles. But in the last decade, it spread to new and everyday audiophiles. They read the forums and traditional rags and started to believe that you must spend hundreds of not thousands of dollars on audio tweaks. That is the bad news. The good news is that our work, through explanation of technology and measurements to back it, has made an incredible dent in this. Many were skeptical of audiophile claims but had read nothing to the contrary. Now they have.

You could argue that your testing will help this new audience and it probably can. But whatever the test is, it needs to be dead simple like what the above objectivist did with the cable swap. Any story more complicated than that will get lost. To wit, Ethan Winer did null hardware test of cables. Problem is that even in a good null, there is still noise. That was enough for the test to be dismissed.

So while I very much applaud your efforts, as a practical matter, I don't think it is worth the trouble.

I can fully confirm your experience.
In our blind tests over the last 25 years, we've always had so-called audiophiles participating, although the majority only take part once or twice at most. It was no different among my friends and acquaintances, which also affected friendships.

In my experience, there are four basic groups, listed in descending order of size:
- Shocked that they couldn't hear the supposedly huge differences, invent all sorts of absurd excuses as to why this is the case, completely ignore and deny reality, and never come back.
- Shocked the first time (see above), disillusioned the second time, but eventually fall back into old patterns of behavior, often motivated/supported by dealers and "friends."
- The third group is also shocked/disillusioned, much more critical and arrogant towards audiophile equipment and snake oil, but they can't completely escape the whole thing; slight relapses are not out of the question.
- The fourth group completely changes course within days/weeks, accepts the results, and from that point on absolutely questions audiophile equipment and accessories.

 

[SanojS]

What would it take to end the cable debate? This is the question I keep thinking about as I read the cable discussions that inevitably devolve into religious arguments. Although individual hobbyists have conducted their own personal experiments and audio scientists have conducted peer-reviewed research, the current evidence still leaves the debate open. So, what kind of test would it take to satisfy both skeptics and believers?

In my day job, I've worked in medical research for 25+ years, supporting randomized controlled clinical trials to evaluate new treatments for diseases such as cancer, Alzheimer's, and diabetes. Randomized controlled trials are considered the "gold standard" by doctors and regulatory agencies worldwide, and the level of rigor applied to human clinical trials is among the highest in any industry. When you want to put a new drug in a human for the first time, you don't have room for errors. This is the level of control that comes to my mind when designing a debate-ending USB cable test.

So if we were to design an audio ABX test that is conducted to the same (or near) control level as a randomized controlled trial for clinical research, AND we want that ABX test to avoid the listener fatigue and stress-induced performance anxiety of previous audio research, I'm proposing a modified ABX test that puts an end to the debate. Over the past day or so, I used some of my clinical trial design tools to draft a modified ABX protocol for testing USB cables, and this is what I came up with. [full protocol link at the bottom]

Plain Language Summary

What is this study about?

This experiment checks if pricey audiophile USB cables really sound better than regular, cheaper cables when playing digital music.

Why is it tricky to test?

People often get nervous when they know they're being tested, which can make it harder to notice small changes in sound quality. Even a slight difference in volume can fool you into thinking one cable sounds 'better' just because it's louder.

How does this test work?

The study uses two identical digital-to-analog converters (DACs), each connected to a different cable. Both play at the same time, and an electronic switch lets you instantly choose between them. This way, there's no need to unplug or reconnect any cables. Listeners hear the same 90-second music clip before and after a short break. During the break, someone might secretly switch the DACs, or might leave them as they are. The listener just rates how 'easy' and enjoyable the music feels.

The test is done twice: first with basic DACs, then with high-end isolated DACs. This helps show if cable differences only matter when using cheaper digital-to-analog converters.

What makes this test fair?

The volume is matched exactly for each test. The person collecting the ratings doesn't know which cable is being used. Some tests include 'fake' switches to check for bias. Finally, the cables are swapped between the DACs and tested again to make sure the results are accurate.

Providing Feedback

If you are interested in poking holes in the methodology, feel free to read it and post a comment or message me directly. If you have a question or a specific issue you think needs to be addressed, please reference the section number in your comment.

Next Steps

Right now, I'm just focused on drafting a gold-standard protocol and haven't thought through all the next steps of conducting the test. If you're interested in facilitating the test or using this protocol in a publication, please DM me.

Complete Protocol

Full protocol published on Google Docs: https://docs.google.com/document/d/...ltGERprVcUAj83xrGoJh9oLU3h8752Iy5BY2lb7-0/pub

Not a good test.

You should test the cable characteristics since after the DAC has done its job, you have an analogue signal, not a digital.

Test the impedance, shielding, jitter and quality of the transmitted digital signal to see what quality it gives. As long as the digital signal arrives without loss of bits, and with low jitter (if USB receiver doesn't have jitter control), then sound won't be impacted at all.

 

[SanojS]

I’m puzzled by all of the expressions of futility and pointlessness here in response to the idea of rigorous and well-designed tests with the potential to expose and embarrass the audiophile claims of people currently hunkered down in the fog bank of unchallenged subjectivity.

Apathy and bare-ass cynicism are not scientific values.

Because testing a digital cable, on the signal after conversion to analog signal, isn't a rigorous and well-designed test. You have to test the digital cable characteristics and the actual signal it transmits (I've been working in IT for 30 years and have led digital signal conversion training as an educator in the 90s in telecom, so I have some experience and understanding on this subject...). Modern USB conversion products are very capable at reproducing the digital signal despite low quality signal coming in.

 

[solderdude]

The problems with any of these tests is:
Not all EMC and common mode leakage will result in the same 'issues' (if there are any) in different DACs and different setups.
Not all DACs have equally good performance in 'unwanted crap rejection'
Not all DACs have equally good 'common mode current rejection'.
What levels of 'crap' and types of 'crap' injection is done and where (around the cable, coupled into the cable) and how does one control/measure this ? One would require an EMC lab with EMI receivers for that.

So even when cables or combinations of cable + DAC is tested the results may still be different in another setup even using the same DAC and cable but connected to different gear.

During tests it would also be important to monitor the analog out (headphone out or speaker out) to check if there is a signal difference or not.
If it is going to be definitive it would have to be witnessed and peer reviewed or repeated by an independent party.

In other words ... an impossible task if only because the test would also depend on human perception.

 

[SanojS]

Only thing that could impact the analog signal, is if the USB cable carries "noise" over the ground loop to the target. The best way is to have a competently built USB target. A cable can help reduce it but not remove it.
It doesn't cost a ton to make a properly shielded USB cable...

 

[Roland68]

Not a good test.

You should test the cable characteristics since after the DAC has done its job, you have an analogue signal, not a digital.

Test the impedance, shielding, jitter and quality of the transmitted digital signal to see what quality it gives. As long as the digital signal arrives without loss of bits, and with low jitter (if USB receiver doesn't have jitter control), then sound won't be impacted at all.

Because testing a digital cable, on the signal after conversion to analog signal, isn't a rigorous and well-designed test. You have to test the digital cable characteristics and the actual signal it transmits (I've been working in IT for 30 years and have led digital signal conversion training as an educator in the 90s in telecom, so I have some experience and understanding on this subject...). Modern USB conversion products are very capable at reproducing the digital signal despite low quality signal coming in.

We're talking about asynchronous USB music transfer here; the receiver (DAC/DDC) doesn't know if data loss or corruption has occurred.

Does anyone have information on what percentage of the data needs to be affected before dropouts or clicking noises become noticeable?

 

[SanojS]

We're talking about asynchronous USB music transfer here; the receiver (DAC/DDC) doesn't know if data loss or corruption has occurred.

Does anyone have information on what percentage of the data needs to be affected before dropouts or clicking noises become noticeable?

I know.

If a USB audio stream loses bits, you hear glitches, pops, clicks, digital artifacts, or brief silences (dropouts), sounding like a skipping CD, because the digital data packets are incomplete, causing the DAC to miss some signal bits, resulting in audio interruptions rather than quality loss, unlike lossy compression.

I think even a tiny amount of data loss is instantly audible as a "pop" or "click"

Still, the signal is analog (ones and zeros) and the competently made USB converter will be able to determine, what is a one and what is a zero (binary data). There's still CRC happening on the signal since it adheres to the USB protocol.

The USB audio signal is isochronous, only clocking can be asynchronous handled, meaning the receiver handles the clocking to improve jitter (I have been a trainer on PCM signals for audio transfers using voice over digital signals).

 

[Roland68]

I know.

If a USB audio stream loses bits, you hear glitches, pops, clicks, digital artifacts, or brief silences (dropouts), sounding like a skipping CD, because the digital data packets are incomplete, causing the DAC to miss some signal bits, resulting in audio interruptions rather than quality loss, unlike lossy compression.

I think even a tiny amount of data loss is instantly audible as a "pop" or "click"

Still, the signal is analog (ones and zeros) and the competently made USB converter will be able to determine, what is a one and what is a zero (binary data). There's still CRC happening on the signal since it adheres to the USB protocol.

The USB audio signal is isochronous, only clocking can be asynchronous handled, meaning the receiver handles the clocking to improve jitter (I have been a trainer on PCM signals for audio transfers using voice over digital signals).

However, this doesn't reflect real-world experience.
A few years ago, we deliberately and intentionally contaminated the data stream from USB transmissions to DACs, as well as the SPDIF data stream from CD players. It was astonishing how late the dropouts started.
We later repeated the same test with a USB DDC and recorded the data stream via SPDIF. With increased contamination, the recorded data became more inconsistent with the original file. Again, it took quite some time before dropouts occurred on the connected DAC.

This also aligns with @NTTY's measurements with CD players that could compensate for gaps up to 4 mm in size. This corresponds to a data loss of up to 3% at a time, and even more when spread out.
I observed the same thing a year ago in a test with a CD player with a USB host connection. The USB-connected DAC played back even heavily damaged CDs without any dropouts. The data loss was likely around 4-6%.
This CD drive has a built-in ripping function. Interestingly, the same damaged CDs exhibited dropouts and clicking noises in the WAV files during the rip process.

Ripping these damaged CDs fails on every PC drive, or results in significant deviations.

 

[danadam]

This also aligns with @NTTY's measurements with CD players that could compensate for gaps up to 4 mm in size.

Isn't that just Reed-Solomon error correction of CD-Audio in action? What does it have to do with errors in USB transmission?

 

[audio_tony]

You would need special software to play two USB DACs at the same time and at least attempt to keep them in sync. Roon does this but many do not.

MPD (Music Player Daemon) can output to multiple DACs in sync. I have 3 DACs connected to my system and there is no noticeable delay when switching between them.

 

[sigbergaudio]

Why USB cables, specifically?

 

[solderdude]

Only thing that could impact the analog signal, is if the USB cable carries "noise" over the ground loop to the target. The best way is to have a competently built USB target. A cable can help reduce it but not remove it.
It doesn't cost a ton to make a properly shielded USB cable...

Shielding simply conducts the common mode signals.
When shielding is done properly so only screen connected to both enclosures and ground/common/return path is separate then that can be of help.
One has to increase the common mode resistance (ferrites and a couple of turns through one) or break it (data transformer/optical).

It all depends on how and where those unwanted signals enter the gear and how grounding and signal handling is done.

Why USB cables, specifically?

I suppose because people mostly use USB and are 'hearing' things... and people might be plagued with common mode gremlins in USB audio from PC/laptop etc ?

 

[Geert]

Why USB cables, specifically?

Because the debate on analog cables has come to a conclusion, didn't you receive the note

 

[SanojS]

Shielding simply conducts the common mode signals.
When shielding is done properly so only screen connected to both enclosures and ground/common/return path is separate then that can be of help.
One has to increase the common mode resistance (ferrites and a couple of turns through one) or break it (data transformer/optical).

It all depends on how and where those unwanted signals enter the gear and how grounding and signal handling is done.


I suppose because people mostly use USB and are 'hearing' things... and people might be plagued with common mode gremlins in USB audio from PC/laptop etc ?

100% agree with what you state.

 

[EERecordist]

Your medical experience is valuable, but I do not think double blind studies are applicable to cables.

The reason we use double blind studies in medicine is first, the immune system is always active and works in tandem with therapies. The immune system is influenced by the nervous system. So we don't want the nervous system to know which treatment it is getting. And we run as large a number of subjects as can be afforded. Second, the body is very complex. So we use statistics to separate the treatment effect from subject variation.

I'm an electrical engineer and have worked in many areas including digital communication.

Digital communications are ridiculously simple compared to the body. Simplified, the transmitter generates ones and zeros, they travel down the cable or fiber, and the detector senses the one or zero. One way to characterize it is an eye diagram. Home audio cables and the transmitter-receiver are hugely over engineered for what they do.

I think someone with a medical background might enjoy our psychoacoustics sub forum and reading up on that. The ear is a mechanical transducer, the cochlea a mechanical to nerve sensor with all kinds of nonlinearities, and the auditory cortex is very complex and varies person to person. Then aging causes deterioration. One of our forum members JJ is an expert and has many talks online. The book by another ASR member https://www.routledge.com/Sound-Rep...phones/Toole-Olive-Welti/p/book/9781032761930 is also a good read.

I always suggest ear training, listening to as many acoustical instrument performances as practical and then listening instrument by instrument to live verses recordings. With that, people may develop preferences on how they want instruments to sound in recordings. It can't be measured but it is a fun use of time.

 

[SanojS]

Your medical experience is valuable, but I do not think double blind studies are applicable to cables.

The reason we use double blind studies in medicine is first, the immune system is always active and works in tandem with therapies. The immune system is influenced by the nervous system. So we don't want the nervous system to know which treatment it is getting. And we run as large a number of subjects as can be afforded. Second, the body is very complex. So we use statistics to separate the treatment effect from subject variation.

I'm an electrical engineer and have worked in many areas including digital communication.

Digital communications are ridiculously simple compared to the body. Simplified, the transmitter generates ones and zeros, they travel down the cable or fiber, and the detector senses the one or zero. One way to characterize it is an eye diagram. Home audio cables and the transmitter-receiver are hugely over engineered for what they do.

I think someone with a medical background might enjoy our psychoacoustics sub forum and reading up on that. The ear is a mechanical transducer, the cochlea a mechanical to nerve sensor with all kinds of nonlinearities, and the auditory cortex is very complex and varies person to person. Then aging causes deterioration. One of our forum members JJ is an expert and has many talks online. The book by another ASR member https://www.routledge.com/Sound-Rep...phones/Toole-Olive-Welti/p/book/9781032761930 is also a good read.

I always suggest ear training, listening to as many acoustical instrument performances as practical and then listening instrument by instrument to live verses recordings. With that, people may develop preferences on how they want instruments to sound in recordings. It can't be measured but it is a fun use of time.

I think the idea of double blind test in speaker testing is to avoid any chance of unintended bias.
It means you have a 3rd party who switch between the cables who is not part of the test, or you have a randomized way of selecting which cable is playing at a certain time.

 

[antcollinet]

and with low jitter

Jitter is not a thing with modern USB audio interfaces using synchronous protocols. The data is requested out of the source according to the DAC's own clock, then buffered and clocked into the DAC with the same stable clock. Jitter is completely irrelevant for USB audio.

 

[antcollinet]

If a USB audio stream loses bits, you hear glitches, pops, clicks, digital artifacts, or brief silences (dropouts)

I thought the same, but a single sample error seems not to be audible. I edited a song, found a sample that was close to full positive, and changed it to full negative. I thought it would manifest as a click or tick sound.

It was not audible.

So it would be interesting to find out just what error rate becomes audible.

 

[ahofer]

The debate is only "open "because someone has their big clown shoe in the door.