• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required as is 20 years of participation in forums (not all true). Come here to have fun, be ready to be teased and not take online life too seriously. We now measure and review equipment for free! Click here for details.

What measurements really matter?

barrows

Active Member
Joined
Jan 24, 2020
Messages
116
Likes
138
#1
The question is meant to be open ended. The thread is going to be premised on listening to music, rather than measuring components, but is meant to make attempts at sorting out what measurements do matter, and at what level they matter. I will start with some simple observations, and suggest that those who might be intersted, please make some observations of your own before commenting.

Here at ASR there is published a SINAD chart, where DACs are ranked by this stat alone, how relevant is that to sound quality, meaning, in room playback performance over loudspeakers? At this point I would suggest that anyone interested, get a Stereophile test disc (I think mine is number 2), and find the -90 dB, 1 kHz tone. Set your system to a normal, critical listening level with some music, best to make this the loudest level you would actually use for listening. Now, play the -90 dB tone from the Stereophile Test Disc... can you hear it from your normal listening position? Here I can definitely hear it, but it is also very, very low in level. Note that 1 kHz is smack dab in the area where human hearing is very sensitive, so it is fairly safe to assume that a single tone of other frequencies at the same level would be no more audible than this 1 kHz tone. I do not have test tones at lower levels, but just roughly considering this, I very much doubt anything below -110 dB could be audible here in my room. My takeaway from this is that I am not too concerned about distortion products below -110 dB, and neither would I be likely to rank DACs against each other based on what goes on below -110 dB, as it is inaudible in a real world situation. If we apply this thinking to the SINAD chart, we can see that even DACs in the second tier (green) mostly exhibit levels of distortion plus noise which are inaudible. So then, we can see, that there is no audible advantage going from say, a Topping D-90, to Chord Mojo, in terms of distortion and noise (of a single tone). So now the question becomes, do these two DACs sound identical? And if not, clearly the difference is not expressed by the SINAD numbers. Could another measurement actually more readily express any sonic differences? I would like to hear some opinions, but it does appear to me from looking at a few different measurements here, that the multitone test excites more distortions at higher levels, than what appears in a simple SINAD test of a single tone, perhaps we should pay more attention to distortion spikes higher than -110 dB in the multitone test? What do you guys think?

Or would you just conclude that all DACs from the green zone and up sound identical to each other and cannot be distinguished by listening?
 

RayDunzl

Major Contributor
Central Scrutinizer
Joined
Mar 9, 2016
Messages
10,148
Likes
8,727
Location
Riverview FL
#2
Could another measurement actually more readily express any sonic differences?
Maybe the problem is in expecting (or desiring?) to hear differences.

Shouldn't all DACs reproduce the input digits equally? (They don't, quite)

If not, the goal remains undefined.

Audible flavor is best applied elsewhere.
 
Last edited:

Voo

Member
Joined
Jan 7, 2019
Messages
89
Likes
50
#3
my ears. I have a few items this site has recommended and measured well. also have some things that dont measure well or havent been measured yet. yet I like all my stuff. sometimes for different reasons. I enjoy stuff that can play loud and clear. my maggie lrs cant play that loud, but they sure are clear.
 
OP
B

barrows

Active Member
Joined
Jan 24, 2020
Messages
116
Likes
138
Thread Starter #4
Maybe the problem is in expecting (or desiring?) to hear differences.

Shouldn't all DACs reproduce the input digits equally? (They don't, quite)

If not, the goal remains undefined.

Audible flavor is best applied elsewhere.
I would suggest that you are misinterpreting the point of the thread. I would ask that you re-read my first post.

I am not talking about flavors, I am asking a legitimate scientific question: to be entirely clear, succintly: What measurements are actually most relevant to DAC sonic performance? Or, if it is your belief that all the DACs which are in the green zone and above actually do sound the same, then you could answer that way as well. I would be fine with that answer.
 

andreasmaaan

Major Contributor
Forum Donor
Joined
Jun 19, 2018
Messages
4,846
Likes
4,949
#5
So now the question becomes, do these two DACs sound identical? And if not, clearly the difference is not expressed by the SINAD numbers. Could another measurement actually more readily express any sonic differences? I would like to hear some opinions, but it does appear to me from looking at a few different measurements here, that the multitone test excites more distortions at higher levels, than what appears in a simple SINAD test of a single tone, perhaps we should pay more attention to distortion spikes higher than -110 dB in the multitone test? What do you guys think?
I think you're correct in believing that the multitone test will tend to give better data than a harmonic distortion test at a single frequency.

However, although you may be able to hear -90dB in your room when a 1kHz tone is played in isolation, this is not the same as being able to hear a distortion product that is 90dB below a fundamental. And this is even more true when the "fundamental" is a complex signal like music.

If you want to demonstrate this for yourself, you could try the Sheffield A2TB Test Disc. Tracks 71-76 give you a single tone with specified percentages levels, and then tracks 77-82 give a piece of music with the same.

The music tracks can be referenced to Track 2 of the same disc, which contains no added distortion.

I believe you'll quickly find that the effects of auditory masking that occur when listening to a music signal are significant here.
 
OP
B

barrows

Active Member
Joined
Jan 24, 2020
Messages
116
Likes
138
Thread Starter #6
Andreas, I do not disagree, necessarily. Your analysis supports my hypothesis even more, would you suggest then that all DACs for the green zone and above sound the same and are indistinguishable from each other?
Or, would you offer that, perhaps, something more is at work here, and what measurements might we use to describe this other thing (If it exists)?
 

andreasmaaan

Major Contributor
Forum Donor
Joined
Jun 19, 2018
Messages
4,846
Likes
4,949
#7
Andreas, I do not disagree, necessarily. Your analysis supports my hypothesis even more, would you suggest then that all DACs for the green zone and above sound the same and are indistinguishable from each other?
Or, would you offer that, perhaps, something more is at work here, and what measurements might we use to describe this other thing (If it exists)?
I'm inclined to believe that, except in pathological cases (e.g. where distortion rises rapidly above 1kHz, or where a strange reconstruction filter has been applied that audibly modifies the frequency response below 20kHz), all the DACs in the yellow zone and above sound the same with music.

As to the measurement question, a multitone test may be more revealing than the single harmonic distortion test at 1kHz, definitely. It's a way of verifying that nothing unacceptable is happening at frequencies other than the 1kHz single-tone test frequency. Another alternative would be IMD vs frequency (or even THD vs frequency). If IMD/THD stay within acceptable limits across the whole audible spectrum, there's nothing to worry about IMO.

As to distortion audibility generally, I wrote a long post about it here. FWIW, my views haven't changed much since that time, except that these days I might emphasise a bit more that (1) audibility is heavily dependent on type/spectrum of distortion, and (2) very low levels of audible distortion (say 0.03-0.1%, if such levels are even audible in a given case) are unlikely to interfere with listening enjoyment.

Of course, when it comes to DACs it's very easy to avoid even these levels of distortion, which is why it's my personal approach to ensure that no distortion harmonic/product rises above -90dB across the whole audible spectrum, even if this is probably overkill for most purposes.
 

RayDunzl

Major Contributor
Central Scrutinizer
Joined
Mar 9, 2016
Messages
10,148
Likes
8,727
Location
Riverview FL
#8
I would suggest that you are misinterpreting the point of the thread. I would ask that you re-read my first post.

I am not talking about flavors, I am asking a legitimate scientific question: to be entirely clear, succintly: What measurements are actually most relevant to DAC sonic performance?
#1 Perceived Frequency Response would likely be most important, but they all (probably) measure ruler flat up to the filter. Then you feed the output to a couple of speakers in a room and it ain't what it was.

#2 No "noise" - Lower is better. That should be obvious. the amplifier will amplify noise, as well as contribute its own noise, and possibly to audible levels. Send it to speakers and you also have the ambient noise levels and the speaker's harmonics to contend with.

#3 What goes in comes out at the right time - "phase" response. Again, add speakers in a room and forget it.

---

I use DRC to modify the frequency response and phase at the listening position. Noise and harmonic distortions can't be fixed. Maybe that raises SINAD's import.

---

Or, I dunno. Sorry.

I bow out. I'm deaf anyway.
 

majingotan

Addicted to Fun and Learning
Forum Donor
Joined
Feb 13, 2018
Messages
862
Likes
785
Location
Laguna, Philippines
#9
Andreas, I do not disagree, necessarily. Your analysis supports my hypothesis even more, would you suggest then that all DACs for the green zone and above sound the same and are indistinguishable from each other?
Or, would you offer that, perhaps, something more is at work here, and what measurements might we use to describe this other thing (If it exists)?
I concur with the DACs on the green spot (I'd even wager as far as the Yellow ones or some of the Red ones) being indistinguishable from each other under the following conditions: DBT volume matched. Without precise controls, one can easily be misled by bias that a DAC has a different sound signature than the other. Having owned 5 different DAC brands from Red zone (Schiit Bifrost 2, FiiO BTR5) to Green zone (Apple Dongle, Chord Mojo and A&Ultima SP2000 DAP) and I cannot tell them differently when used as a DAC (no load condition) under volume matched conditions. Now take away all the controls and listen to a specific DAC for a few days then switch to another DAC by just remembering the volume by memory. The psychoacoustics of relying on our memories kicks in then you'll have an epiphany regarding what subjectivists are always clamoring on other forums, finding that this specific DAC that I just switched sounded inferior/superior than the other DAC that I had "critically" listened for just a few days. The key point is that sighted bias is NOT an measurement/indicator of what really matters when questioning the audible difference between DACs. That's why DBT volume matched test with various pools of trained and untrained listeners are needed to truly find out. Until then, the scientific papers about limits of audibility of distortion products under normal listening conditions correlates that DACs from yellow to blue are indistinguishable sonically from each other
 

Webninja

Active Member
Forum Donor
Joined
Oct 8, 2018
Messages
219
Likes
203
Location
Los Angeles
#10
Basically, once you have a DAC or a few candidates from the green zone, the next choice would be features and budget. Do you need a headphone amp? Balanced? Bluetooth?

The SINAD chart rankings ensure you’re getting a DAC that does its job with transparency, and as others have stated, look to speakers or the room for personal sound preferences.
 
OP
B

barrows

Active Member
Joined
Jan 24, 2020
Messages
116
Likes
138
Thread Starter #11
OK, so far it looks like the consensus is that DACs from the green zone and up are indistinguishable from one another for playing back music. I would suggest then, that the apparent importance that many take away for the SINAD chart is in error. To wit, I have seen folks, siting ASR as their reference, making claims such as: the topping d-90 wipes the floor of the benchmark DAC 3 (paraphrase). Perhaps it would be good idea to make it very clear, that despite the measured differences shown by the SINAD cart, all the DACs from the green level and up are equal in terms of sonic performance in music listening.

I am well aware of the need for carefully level matched comparisons for DAC evaluations. I prefer to measure DAC output voltage to level match.
 

Blumlein 88

Major Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
9,875
Likes
13,366
#12
Yeah, #1 is frequency response. You'd be surprised how little change we can hear, and how few DACs are really, really flat 20hz-20khz.

If you can't pass this hurdle then you can stop right there.

I pretty much concur with Ray in noise being next.

Not sure I agree with phase being that important to us humans. If we can keep it good until 2khz (not a typo) I don't think it is much important above that.

Now beyond that are numerous possible, but not common gotchas like impedance between pieces of gear, and gain levels that can make noise a problem when it otherwise wouldn't be. I could find a lower end green zone DAC that usually is just fine, but in the right misuse in a particular system noise would be a problem. That is where gain matching can matter.
 

Alexanderc

Senior Member
Forum Donor
Joined
Jun 11, 2019
Messages
328
Likes
391
Location
Florida, USA
#13
There are actual, scientific data that describe the thresholds of human hearing under a variety of conditions, yes? Best case perceptions of very quiet, very high frequencies, very low frequencies, tiny differences in various parameters of sound? Wouldn’t it be wise to compare those data to measurements of the equipment? Then you would have THE answer. If I’m not mistaken, Amir frequently cites this data in his reviews. “As you can see, this is beyond audible dynamic range threshold of human hearing....” that sort of thing.

I have seen several posts like this over the past few days. It seems barrows’ question is legitimate (I’m not sure all of them have been), but don’t we actually know the answers to these questions?

BTW, those test CDs are hard to find. Stereophile doesn’t appear to sell them anymore. Amazon has them for prices ranging from $70 to over $900. There are a few used on eBay costing more than the originally did new.
 

andreasmaaan

Major Contributor
Forum Donor
Joined
Jun 19, 2018
Messages
4,846
Likes
4,949
#14
There are actual, scientific data that describe the thresholds of human hearing under a variety of conditions, yes? Best case perceptions of very quiet, very high frequencies, very low frequencies, tiny differences in various parameters of sound? Wouldn’t it be wise to compare those data to measurements of the equipment? Then you would have THE answer. If I’m not mistaken, Amir frequently cites this data in his reviews. “As you can see, this is beyond audible dynamic range threshold of human hearing....” that sort of thing.

I have seen several posts like this over the past few days. It seems barrows’ question is legitimate (I’m not sure all of them have been), but don’t we actually know the answers to these questions?
Concerning nonlinear distortion, I tried to address these questions in some detail here.
 

Blumlein 88

Major Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
9,875
Likes
13,366
#15
There are actual, scientific data that describe the thresholds of human hearing under a variety of conditions, yes? Best case perceptions of very quiet, very high frequencies, very low frequencies, tiny differences in various parameters of sound? Wouldn’t it be wise to compare those data to measurements of the equipment? Then you would have THE answer. If I’m not mistaken, Amir frequently cites this data in his reviews. “As you can see, this is beyond audible dynamic range threshold of human hearing....” that sort of thing.

I have seen several posts like this over the past few days. It seems barrows’ question is legitimate (I’m not sure all of them have been), but don’t we actually know the answers to these questions?

BTW, those test CDs are hard to find. Stereophile doesn’t appear to sell them anymore. Amazon has them for prices ranging from $70 to over $900. There are a few used on eBay costing more than the originally did new.
Yes, this question always runs into the same problem. The gray area. The zone of uncertainty.

We can give pretty good numbers so that if you meet them there is just not any reason to think they could sound different. Any less and you have edge cases of excellent hearing combined with odd situations where it might become an audible difference though it probably will not be audible except maybe 1% of the time. And this is my gripe about the SINAD chart. Even two pieces of gear with the same SINAD score might be identical in system A and discernible in system B. You can have a wee bit of pinkish noise with vanishing distortion at -90 db SINDAD or you can have distortion with noise so low it will never be heard.

Someone around hear suggested turning up your system as loud as you'll ever listen. With silence do you hear noise? Nope, then noise is good. If you have flat FR, then you more than likely are all set.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
28,551
Likes
75,664
Location
Seattle Area
#16
Keep in mind that we have two types of audience here: those that use speakers and those that use headphones. The latter allows blocking of the room noise, ability to play at insane SPL levels, and doesn't allow one channel's content to mask the distortions of the other.
 

RayDunzl

Major Contributor
Central Scrutinizer
Joined
Mar 9, 2016
Messages
10,148
Likes
8,727
Location
Riverview FL
#17
Keep in mind that we have two types of audience here: those that use speakers and those that use headphones.
Good point.

I'm probably more observant of "noise" with headphones.

Sometimes I use them with TV, and the noise floor in the various segments of the broadcasts (studio, remote interview, commercials) that I might never notice when listening with speakers is often obvious with the phones.
 

Maxicut

Active Member
Joined
May 9, 2020
Messages
105
Likes
52
#18
In the digital world, by far the most important spec (in my near 40-year experienced opinion) is jitter. Converting the D to A is the 1st & most important step to an enjoyable analogue sounding playback, regardless of how the rest of it "measures".
 
Last edited:

solderdude

Major Contributor
Joined
Jul 21, 2018
Messages
6,350
Likes
11,772
Location
The Neverlands
#19
The question is meant to be open ended. The thread is going to be premised on listening to music, rather than measuring components, but is meant to make attempts at sorting out what measurements do matter, and at what level they matter. I will start with some simple observations, and suggest that those who might be intersted, please make some observations of your own before commenting.
All measurements matter, if only 1 would matter only that one would be made.
Consider test tones and music are 2 very different but closely related things.

Here at ASR there is published a SINAD chart, where DACs are ranked by this stat alone, how relevant is that to sound quality, meaning, in room playback performance over loudspeakers?
It has been explained a zillion times. SINAD is a simple number that says something about distortion levels at 1kHz only and noise levels and ONLY that. It is not the sole important number. It is an indication. It is easy to rank and says something.
Might as well have another 'number' which you can rank. Say 50mV performance (check is already ranked) or S/N ratio or how many dB roll-off in the lows or treble, or how many dB ripple, or filter steepness or channel separation, 19+20kHz IM distortion, FSD output voltage, intersample over handling. One can rank them all.
Overall, perhaps SINAD is the most indicative of technical performance. It has NO relation to how it is perceived in sighted comparisons.

At this point I would suggest that anyone interested, get a Stereophile test disc (I think mine is number 2), and find the -90 dB, 1 kHz tone. Set your system to a normal, critical listening level with some music, best to make this the loudest level you would actually use for listening. Now, play the -90 dB tone from the Stereophile Test Disc... can you hear it from your normal listening position? Here I can definitely hear it, but it is also very, very low in level. Note that 1 kHz is smack dab in the area where human hearing is very sensitive, so it is fairly safe to assume that a single tone of other frequencies at the same level would be no more audible than this 1 kHz tone. I do not have test tones at lower levels, but just roughly considering this, I very much doubt anything below -110 dB could be audible here in my room.
At this point you are already leaving your own premise. It is not impossible to hear -90dB 1 kHz continuous tone in very quiet circumstances. However, that ability has nothing to do with actual music.
If you really want to test you should mix that -90dB sine with music and then try to hear the 1kHz tone. Good luck with that.
And even that is cheating in quiet passages. More realistic would be to 'modulate' that 1kHz with the average music level so that it only is -90dB while the music is loud and lowered at quiet passages. I can make a prediction here when it comes to audibility of distortion levels when music is playing.

My takeaway from this is that I am not too concerned about distortion products below -110 dB.
The reason why a higher SINAD becomes important is simple. When you use digital volume control low noise levels become important. This is included in SINAD but a better ranking should be to use the S/N ratio in this case. It is far more important but the higher the SINAD is the better one can expect this to be.
Another reason is one might be inclined to play really softly and don't want to hear noise other than what's in the room already and at other times play enormously loud peaks in music (high DR or classical).
It would be wise to look at above 110dB SINAD gear (that also measures well in other departments)


So now the question becomes, do these two DACs sound identical? And if not, clearly the difference is not expressed by the SINAD numbers.
One can build a DAC with very high SINAD but have an audible roll-off response. When compared to the exact same SINAD DAC that does not roll-off (and using young ears) some will be able to reliably tell them apart in level matched blind conditions.
This COULD even be so using the same DAC but different (default) filter settings.
So the answer is... in most cases yes, when level matched and blind tested. Just not all cases.
Chances are something is 'f ed up with a high SINAD DAC is much smaller than with mediocre DACs.

A:Could another measurement actually more readily express any sonic differences?

B: would you just conclude that all DACs from the green zone and up sound identical to each other and cannot be distinguished by listening?
A: Yes, that's why not only SINAD is measured.

B: No, but under specific conditions might.
 
Last edited:

Maxicut

Active Member
Joined
May 9, 2020
Messages
105
Likes
52
#20
All measurement matter, but the question is "What Measurements Really Matter? I'm guessing he wants to look past all of the salad & get into the meat!
 
Top Bottom