• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

What Audio Cues to look for in a blind DAC test?

Joined
Jul 7, 2022
Messages
8
Likes
2
Hi ASR,
This is my first post, although I have been scrolling and enjoying the content on this site for several years now. Please let me know if there's anything wrong with the way I've posted etc and I will modify it.
So here goes. My goal is to blind test a few DACs I have, to see if I can tell the difference between the 2 (not preference, just difference).

Work done from my side -
  • Went through threads related to "DAC Blind Test" and the likes to get a feel of general methodology recommended and if this topic has already been discussed (1 , 2 , etc.) but I couldn't find a similar question. If this is a duplicate Q, I'll delete it.
  • I have previously done a somewhat blind DAC test, and this time will improve upon it. This thread is not about the methodology though, I will post a separate thread for that if needed. (Volume matched with ADC, switching done by another person etc).
  • I watched @amirm 's excellent video related to Audio Blind Tests , in which he describes the critical listening skills required for ABX tests.
What I'm looking for -
  • If I am testing say, a poorly measured dac like Schiit Bifrost MB, and a good one, like SMSL Sanskrit Mk 2, what audible differences should I look for? Amir has done a great job in explaining some of the differences between lower and higher bitrate MP3 for example (where freq spectrum as well fidelity suffers), but I am hesitant to apply to the same to DACs.
  • If I record, say a very short clip of a guitar string being plucked (or a bass note, or whatever) will that be one of the good ways? Again, this is in the video, hence asking.
I will of course use some tracks that I'm very familiar with as well, however it seems that might not be useful if I'm not listening for the right things, hence this thread.
I understand what I'm asking for is critical listening skills, which I guess I need to patiently train for through some separate course, but I was wondering if someone has a good "cheat sheet" on what to typically look for (for DACs).

Thanks you in advance, cheers.
 
Last edited:
OP
I
Joined
Jul 7, 2022
Messages
8
Likes
2
Unless a DAC is very broken or not level matched with the others, you're not likely to hear any differences. Really.
One of the DACs is going to be Schiit Bifrost Multibit, so yes the test will satisfy the required criterion of a broken dac.
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,521
Likes
37,049
Why not something simpler. Use pkanes Distort software to compare a clean signal vs a distorted one using Foobar ABX. See how much distortion is audible whether using music or things like guitar plucks.

It would be valuable informative experience prior to moving on to DAC testing.
 
OP
I
Joined
Jul 7, 2022
Messages
8
Likes
2
Why not something simpler. Use pkanes Distort software to compare a clean signal vs a distorted one using Foobar ABX. See how much distortion is audible whether using music or things like guitar plucks.

It would be valuable informative experience prior to moving on to DAC testing.
How will I replicate the exact distortion delta introduced by say, SMSL Sanskrit and Bifrost Multibit? Perhaps one of the parameters I can, but to replicate the characteristics of 2 dacs seems infeasible.
I think it can be a good experiment to observe the threshold of distortion detection for different individuals, I'll check out the software you have mentioned, but I would prefer to stick to DACs for this experiment.
 

Jimbob54

Grand Contributor
Forum Donor
Joined
Oct 25, 2019
Messages
11,063
Likes
14,694
Do a search for posts by ex member Pdxwayne . He was very keen on determining cues/tells in blind tests.

But for my money, if you have to look for miniscule artifacts or small discrepancies, what is the point?
 

DVDdoug

Major Contributor
Joined
May 27, 2021
Messages
2,916
Likes
3,831
If there aren't "glitches" (obvious defects like clicks or losing the audio for a moment) there are only 3 characteristics that define sound quality - Noise, distortion, and frequency response. (See Audiophoolery ) As you probably know, Amir combines noise and distortion as SINAD).

Distortion and frequency response are almost always better than human hearing but sometimes there is audible noise or one device may have more noise than the other. Noise is somewhat common from a soundcard and sometimes noise gets into a USB-powered DAC through the USB power. It's a more common problem on the microphone input on an audio interface, since the noise gets amplified along with the microphone signal. Computer power supplies are notoriously noisy and usually it's a high frequency whine from the switching power supply and the other high-speed data/switching inside the computer. Your "odds" of low noise are better if the DAC has it's own separate power supply.

So the easiest way to hear a difference is to listen for noise with no signal... After level matching because it's actually the signal-to-noise ratio that's important... You can have lower output so you can't hear the noise but if the signal is also quieter that "doesn't prove anything".

This is true with most electronics. If there's an issue it's probably noise.

With speakers (and headphones) it's usually frequency response and almost all speakers sound different-enough that you can hear a difference without careful listening or training.

explaining some of the differences between lower and higher bitrate MP3 for example (where freq spectrum as well fidelity suffers),
MP3 is a little more tricky. Because of the complex processing some defects don't show up in traditional noise, distortion, and frequency response tests. With most MP3 settings there is a loss of high frequencies and that can be easily measured, but with higher bitrates it's NOT usually what's heard. If a high bitrate MP3 sounds different from the uncompressed original it's usually a time-related effect described as "pre echo" that happens with transient sounds like castanets.

In the context of music, the highest frequencies that you might be able to hear when played loud in a hearing test are weak and masked (drown out) by slightly-lower frequency sounds so you can roll-of the highs (near 20kHz) and you probably won't notice any difference. And if you are an adult (especially an older adult like me) you may not be able to hear 20kHz anyway. The main way that MP3 (and other lossy encoding) is able to make a good sounding file while throwing away 80% of the "data" is by analyzing the spectral content and throwing away details that are masked. (It sort-of takes a shortcut with the highest frequencies and just assumes they are masked without much analysis.)

P.S.
I don't like hearing defects so the LAST THING I'd want is to train myself to better hear them! ;) I grew up with viny and the "snap", "crackle", and "pop" always annoyed me, although most people didn't seem bothered by it. And I didn't have to train myself or listen carefully...
 
Last edited:
OP
I
Joined
Jul 7, 2022
Messages
8
Likes
2
Do a search for posts by ex member Pdxwayne . He was very keen on determining cues/tells in blind tests.

But for my money, if you have to look for miniscule artifacts or small discrepancies, what is the point?
Thanks a lot, I searched and there are indeed very interesting expts he has done related to audio cues, I will go through those.
As for your second point, I believe that is similar to the difference between reasonably lower and higher bitrate audio files? So that part is subjective, if something is worth the money or not, I'm more interested at this point in the results :)
 
OP
I
Joined
Jul 7, 2022
Messages
8
Likes
2
If there aren't "glitches" (obvious defects like clicks or losing the audio for a moment) there are only 3 characteristics that define sound quality - Noise, distortion, and frequency response. (See Audiophoolery ) As you probably know, Amir combines noise and distortion as SINAD).

Distortion and frequency response are almost always better than human hearing but sometimes there is audible noise or one device may have more noise than the other. Noise is somewhat common from a soundcard and sometimes noise gets into a USB-powered DAC through the USB power. It's a more common problem on the microphone input on an audio interface, since the noise gets amplified along with the microphone signal. Computer power supplies are notoriously noisy and usually it's a high frequency whine from the switching power supply and the other high-speed data/switching inside the computer. Your "odds" of low noise are better if the DAC has it's own separate power supply.

So the easiest way to hear a difference is to listen for noise with no signal... After level matching because it's actually the signal-to-noise ratio that's important... You can have lower output so you can't hear the noise but if the signal is also quieter that "doesn't prove anything".

This is true with most electronics. If there's an issue it's probably noise.

With speakers (and headphones) it's usually frequency response and almost all speakers sound different-enough that you can hear a difference without careful listening or training.


MP3 is a little more tricky. Because of the complex processing some defects don't show up in traditional noise, distortion, and frequency response tests. With most MP3 settings there is a loss of high frequencies and that can be easily measured, but with higher bitrates it's NOT usually what's heard. If a high bitrate MP3 sounds different from the uncompressed original it's usually a time-related effect described as "pre echo" that happens with transient sounds like castanets.

In the context of music, the highest frequencies that you might be able to hear when played loud in a hearing test are weak and masked (drown out) by slightly-lower frequency sounds so you can roll-of the highs (near 20kHz) and you probably won't notice any difference. And if you are an adult (especially an older adult like me) you may not be able to hear 20kHz anyway. The main way that MP3 (and other lossy encoding) is able to make a good sounding file while throwing away 80% of the "data" is by analyzing the spectral content and throwing away details that are masked. (It sort-of takes a shortcut with the highest frequencies and just assumes they are masked without much analysis.)

P.S.
I don't like hearing defects so the LAST THING I'd want is to train myself to better hear them! ;) I grew up with viny and the "snap", "crackle", and "pop" always annoyed me, although most people didn't seem bothered by it. And I didn't have to train myself or listen carefully...
Thank you for the detailed response, @DVDdoug.
I think apart from noise as you suggested, I would be interested in distortion artefacts as well. Say, the audio cues difference between second order harmonics, and third order harmonics (and/or the lack thereof). This would help me evaluate the DACs at a later point in a subjective/preferential manner as well (for my own tastes of course).
Hence, my first step is to just check if I can even notice a difference between DACs at hand - if I can't, that's fine (and even pocket friendly). If yes, then I would go the route of subjective preferences.
 

Beershaun

Major Contributor
Forum Donor
Joined
Oct 3, 2019
Messages
1,864
Likes
1,910
I wouldn't worry too much about picking out miniscule differences. I would just see if you can reliably and repeatably tell the difference between them. Like if 7 out of 10 times you pick the same DAC in a random blind test. It tells you that you can discern a difference, whatever it is. Blind tests should give you time to relax and enjoy the exercise and take the time you need to determine if you hear anything. Then the results tell you if it was consistent or random.

In my case it's been, I reliably hear better/louder bass response at the same listening level.
 
OP
I
Joined
Jul 7, 2022
Messages
8
Likes
2
I wouldn't worry too much about picking out miniscule differences. I would just see if you can reliably and repeatably tell the difference between them. Like if 7 out of 10 times you pick the same DAC in a random blind test. It tells you that you can discern a difference, whatever it is. Blind tests should give you time to relax and enjoy the exercise and take the time you need to determine if you hear anything. Then the results tell you if it was consistent or random.

In my case it's been, I reliably hear better/louder bass response at the same listening level.
Perhaps I misunderstood what you said, but I am indeed trying to see if I can reliably and repeatably tell 2 DACs apart via ABX tests (8/10 for ex). For this, I would need to pick up miniscule differences between the two during the test.
 

Jimbob54

Grand Contributor
Forum Donor
Joined
Oct 25, 2019
Messages
11,063
Likes
14,694
Perhaps I misunderstood what you said, but I am indeed trying to see if I can reliably and repeatably tell 2 DACs apart via ABX tests (8/10 for ex). For this, I would need to pick up miniscule differences between the two during the test.
Do you have a DAC that has selectable reconstruction filters? Can you tell the difference between (for eg) a really slow filter and brickwall?
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,521
Likes
37,049
How will I replicate the exact distortion delta introduced by say, SMSL Sanskrit and Bifrost Multibit? Perhaps one of the parameters I can, but to replicate the characteristics of 2 dacs seems infeasible.
I think it can be a good experiment to observe the threshold of distortion detection for different individuals, I'll check out the software you have mentioned, but I would prefer to stick to DACs for this experiment.
Firstly, the Distort software is more advanced than you think. You could in fact use measurements to build a distortion profile very much like the DACs. You are able to set distortion for each harmonic and however many harmonics you like to include. However, I wasn't suggesting you simulate those DACs instead of testing with them directly.

My suggestion is if you've no experience doing blind testing, like most things, practice and experience improve your results and make you more comfortable with it. Also you don't want to start with something extremely difficult to perceive. Better to start with something relatively easy and then work toward finer and finer differences. Distort lets you set distortion any way you wish. So you start with something easy and see what it is like to hear such a difference. Then reduce distortion and repeat. You can build very real world types of distortion with that software.

In addition, you may find distortion not nearly so easy to hear as you imagined. If you aren't able to hear less than 1% (-40 db) on music, then it makes little sense to waste very much time testing yourself with a "bad" DAC that has .01% distortion (-80 db). You could still go ahead and test the DACs themselves in case something not distortion related is audible. Even so the experience practicing with Distort will be very helpful to you. And I'll tell you ahead of time, once you precisely match levels, and get the same frequency response there is rarely anything else in modern DACs you will be able to hear. Reconstruction filters are sometimes audible, but because they alter the upper octave or so of frequency response. For this reason you might wish to use 96 khz playback for testing. It pushes the FR differences between DACs due to filtering upward into the ultrasonic region where they don't interfere with results of such testing.
 
Last edited:

Beershaun

Major Contributor
Forum Donor
Joined
Oct 3, 2019
Messages
1,864
Likes
1,910
Perhaps I misunderstood what you said, but I am indeed trying to see if I can reliably and repeatably tell 2 DACs apart via ABX tests (8/10 for ex). For this, I would need to pick up miniscule differences between the two during the test.
I'm saying they best way may be to take your time during the exercise and feel comfortable listening to your music over and over again in a relaxed manner until you settle on what differences you are hearing. I also agree with what others have said about listening to a quiet or 'blank' passage with headphones to see if you can hear noise from the DAC.

Also the differences may not be miniscule. Particularly when one DAC can deliver at a higher nominal voltage than another at peak SINAD. In my case that manifested in much better bass response at the same listening level.
 
OP
I
Joined
Jul 7, 2022
Messages
8
Likes
2
Do you have a DAC that has selectable reconstruction filters? Can you tell the difference between (for eg) a really slow filter and brickwall?
I don't think my DACs have a selectable filter. I might play around with different distortions as @Blumlein 88 suggests, perhaps that software will also let me play around with effects of different filters.
 
Top Bottom