Blind-testing four DACs

[solderdude]

Not so difficult if you do the test using the helper.

Just try it first with 1 DAC using 2 (very different) filter settings but same procedure.

 

[solderdude]

At this point, probably just going to try solderdude’s idea of having a helper. Go to another part of the house, wait a given time while the switching occurs, enter the room blindfolded, sit facing away from the equipment, listen, repeat.

It's fun to do, you have some interaction with another human which you will be sharing the experience with.
Make it an relaxed evening, take plenty of 'rest' between the listening.

 

[Chagall]

One more option if you have a PC or laptop, you can connect all four DACs via USB and start a 14-day Roon trial. In Roon, you can group all four DACs into a single zone so they play simultaneously. Then just ask your friend to flick the switch.

 

[TheEman]

And just to be clear…I do not have a Gustard R30 or Chord DAVE. But I know people who do.

 

[ahofer]

but now you’ve got multiple sources,

Multiple *digital* sources. Do you honestly think that would make a difference?

The thing you have to think about is volume equalization across the DAC outputs, the rest will make even less difference than the likely inaudible DAC differences. Given how small the differences are going to be here, any volume difference will absolutely dominate the test.

 

[Multicore]

For fun…? Yeah, I’m weird, but doesn’t everyone on this forum fit that to some extent?

To be clear, I don’t believe that competently-designed DACs will sound different, but having the blind testing is a good exercise to demonstrate this to myself.

Yes, that could be fun. And you will learn about the complexities of doing these tests.

 

[DVDdoug]

Given that audio memory is only around 7 seconds, I need to be able to switch quickly.

You are correct that quick (or instant switching) is "better" (a more-sensitive test), BUT as a PRACTICAL MATTER, I'd argue that if you can't hear a difference from one day to the next there's no point in choosing a different DAC or replacing the one you're using. And it makes no economic sense to spend more money for a "better one" if you can't hear the difference the next day.

I'd agree it's probably is better to compare 2 devices at a time with an
ABX test. Maybe choose one as the the "standard" (for example "A") and compare all of the others to "A". If you can't statistically-reliably hear a difference between "A" and any of the others, you're done. If you can hear a difference, things are getting interesting and you can switch-out "A" and repeat and or do more advanced quality/preference tests.

The protocol for ABX tests is well defined, and it's OK to "cheat" a bit and do it single-blind as long as you're not publishing a formal study. (A double-blind test can be very difficult to do.) Or with certain experiments (like when comparing MP3 to WAV) the computer can do the switching so it's not truly double-blind (the computer knows if X is "A" or "B") but it can still be a as good as a double-blind test.

An ABX test doesn't require you to decide which one sounds better. It's simply a test to confirm that you can actually hear a difference by identifying if you can reliably hear a difference. So again, if you can't hear a difference you're done! If you can hear a difference you may want to do further listening tests or measurements.

Or the difference may be something obvious like noise (hum, hiss, or whine in the background) and in that case you don't really need a blind test! The problem is, in many cases people just THINK they are hearing a "night and day" difference. See my signature below.

BTW - An audible filter difference is a valid difference so that's something worth investigating if you're reliably hearing a difference.

A level difference is also a valid difference but it's not a very interesting difference and (again assuming an ABX test) if it allows you to identify "X" every time, people often think they are hearing other differences too, but then those other differences have to validated by eliminating the level difference and repeating the test...


Also see Controlled Audio Blind Listening Tests (video).

 

[solawind]

how i did such a test:

1. you can connect multiple dacs to the same pc.
2. you can run multiple audio player instances (for example, foobar), each with a different dac selected.
3. you can set foobar play on keyboard shortcut, so with single key press all dacs will be playing exactly the same allows you to A/B instantly.

the rest is trivial. on the first dac, select a volume level using real music that you are comfortable with, and then level-match with voltage meter in RMS mode all the other dacs to this one using a 1 khz sine wave test file in foobar.

your test rig is ready. But you need someone to help, who will plug headphone/speaker amp in this DACs behind you.

i don’t have one, so i used a simple usb relay switch (5$ on aliexpress) with a python script that can

1. switch inputs randomly and allow me to guess which device is connected, and
2. perform a proper ABX test in which i can listen to A, B, and a random X and then decide what X is

basically, i never needed second setup, because even in simple A/B tests i can’t distinguish between any of my dacs over a long run about 20 trials, which should be enough to get something statistically relevant.

 

[Sokel]

It's far easier to get something like an pocket miniDSP or an Adept, digitize the outputs and then Deltawave them to the point of no return.
Far better than ears.

 

[solawind]

It's far easier to get something like an pocket miniDSP or an Adept, digitize the outputs and then Deltawave them to the point of no return.

i agree, but strictly speaking, it’s not the same. also, it’s always fun to investigate your own hearing limits with something like r2r dacs that have a lot of distortion.

 

[Sokel]

i agree, but strictly speaking, it’s not the same. also, it’s always fun to investigate your own hearing limits with something like r2r dacs that have a lot of distortion.

It does not need to be an R2R, many new DACs have what they call "sound colors" and such, some SINAD champions as well.
The SINAD of those "colors" with their "tube-like" or transistor -like" descriptions rides up to -60dBr or -50dBr with the added distortion and you can have it on the fly.

(consensus through forums is that no difference )

 

[techsamurai]

What did you find out?