Blind testing two DACs - Proper Process

[davmol]

May I ask which pieces of music you used for the comparison?

I listened mainly to these playlists on Tidal and it Spotify. I tried to listen to a reasonably wide range of genres. The philharmonic does pop/rock was the closest I could get to classical without falling asleep, so I'll have to try harder in future

ASR Critical Music Tracks for Speaker & Room Testing

Listen to ASR Critical Music Tracks for Speaker & Room Testing on TIDAL

tidal.com

Hi-Res FLAC - 24bit/192kHz

Playlist - Hi-Res FLAC - 24bit/192kHz - 74 items

tidal.com

Electronic for Beginners

Playlist - Electronic for Beginners - 20 items

tidal.com

 

[HeebieGenie]

The amount of anecdotal support for more expensive DACs and amps being noticeably better than cheaper ones is so overwhelmingly large that I have always thought in the past "Surely this HAS to be true, right?"

I have never heard a true difference in sound quality, only volume. But, people just tell me "You don't have the ears to hear the difference, I would pass blind tests"

 

[krabapple]

The amount of anecdotal support for more expensive DACs and amps being noticeably better than cheaper ones is so overwhelmingly large that I have always thought "Surely this HAS to be true, right?"

And now you know that line of reasoning is mistaken generally, right?

 

[antcollinet]

You don't have the ears to hear the difference, I would pass blind tests"

And yet they never try.

 

[HeebieGenie]

And now you know that line of reasoning is mistaken generally, right?

Yes, I tend to believe that nowadays. I really think it's placebo of some kind, or psychoacoustics.

 

[NTTY]

"bad" though the mini is - it still achieves 85+dB Sinad (0.006% THD+N). Still pretty unlikely you'll be able to hear any difference between that and a "better" DAC.

And an interesting investigation because of that.

And that's only at full scale. Things can get messy when far from 0dBFS.

This is the worst (high-end) DAC I’ve seen, at full scale:

Significant distortion as per today's standard, but not something we can hear.

Let's try @-40dBFS:

Yep, that's 2% distortion... And it was 14% @-60dBFS, with a non linearity reaching a whooping 3dB! (signal played too loud). Really nasty...

In a blind test (record of music played by two DACs including this one, with high precision interface (input gain adjusted to less than 0.02dB), then played in foobar with the ABX plugin), compared to another well behaved DAC, it was not possible to identify this ugly one (with Tracy Chapman - Fast Car (don't shoot) and Diana Krall - Peel me a grape (don't shoot again)).

That's when I usually tell my friends to give a try to the Klippel audio test, it's fun to know what level of distortion we can hear.

Cheers

 

[Old_School_Brad]

And that's only at full scale. Things can get messy when far from 0dBFS.

This is my worst (high-end) DAC at full scale:

View attachment 418060

Significant distortion as per today's standard, but not something we can hear.

Let's try @-40dBFS:

View attachment 418061

Yep, that's 2% distortion... And it was 14% @-60dBFS, with a non linearity reaching a whooping 3dB! (signal played too loud). Really nasty...

In a blind test (record of music played by two DACs including this one, with high precision interface (input gain adjusted to less than 0.02dB), then played in foobar with the ABX plugin), compared to another well behaved DAC, it was not possible to identify this ugly one (with Tracy Chapman - Fast Car (don't shoot) and Diana Krall - Peel me a grape (don't shoot again)).

That's when I usually tell my friends to give a try to the Klippel audio test, it's fun to know what level of distortion we can hear.

Cheers

Yep, it’s quite an eye-opener when you realize how bad things have to get before it becomes audible. It kind of makes the whole SINAD race feel pointless -well, that's not entirely fair of me to say, but let's call it more of a showcase of engineering excellence. In practical terms though...

 

[NTTY]

SINAD is indeed a showcase, especially when used to rank items, it’s inevitable. And it could be done at different output scales. It is, by the way, since Amir, by his own convention, looks at it at 4Vrms output (balanced) and would reduce digitally the sine tone to match this target, if a DAC would output one or two dB more, and even if it wouldn’t be what the end user would get from the device in real life (listening to a master that reaches 0dFBS).

That being said, and because it mixes THD and Noise into one measure, we know that when SINAD is good (let’s say above 90dB) then the device does not suffer from significant noise or harmonic distortion. At a lower SINAD, there’s indeed a need to understand what is the source of the problem. Harmonic distortion would remain hidden into music until very high levels, not so much about noise.

 

[antcollinet]

And that's only at full scale. Things can get messy when far from 0dBFS.

This is the worst (high-end) DAC I’ve seen, at full scale:

View attachment 418060

Significant distortion as per today's standard, but not something we can hear.

Let's try @-40dBFS:

View attachment 418061

Yep, that's 2% distortion... And it was 14% @-60dBFS, with a non linearity reaching a whooping 3dB! (signal played too loud). Really nasty...

In a blind test (record of music played by two DACs including this one, with high precision interface (input gain adjusted to less than 0.02dB), then played in foobar with the ABX plugin), compared to another well behaved DAC, it was not possible to identify this ugly one (with Tracy Chapman - Fast Car (don't shoot) and Diana Krall - Peel me a grape (don't shoot again)).

That's when I usually tell my friends to give a try to the Klippel audio test, it's fun to know what level of distortion we can hear.

Cheers

Thats odd - wouldn't we normally expect non linear distortion products to drop in proportion to the test signal? WTH is that DAC doing?

 

[NTTY]

Thats odd - wouldn't we normally expect non linear distortion products to drop in proportion to the test signal? WTH is that DAC doing?

With 1bit DACs, yes, with older ones (R2R here), less so. But this one was quite a joke indeed, high-end of the time, at least by its price.