Speed Reading DAC Audio Measurements

[EJH]

I am baaaaaaack!!! A new video in a series to become proficient in reading measurements, in this case, DAC measurements. The video itself is rather long but hopefully once you watch it, you can scan a text review in a minute or two.

Great little primer with which I had two issues from the subjective side.

You say about digital filters: "There's this fallacy where people think that the slow filters are better. No. They're not better...You want to have the sharpest, nicest filter."

I don't take issue with the fact that the sharp filter is better at filtering. Nevertheless, subjectively, I find the most gradual min. phase filter setting on Audirvana the most pleasing. It sounds more relaxing to me than the steepest possible linear filter setting. (Assuming I'm actually hearing any difference at all.) So --

The issue I have is not THAT you made a pronouncement about which filter is best at filtering, but HOW you did so, particularly as you become prescriptive (this is what you want). My recommendation is that you don't make people feel that there is something wrong with them for preferring a slower filter. Rather than saying "feel free to muck around with the filters," as if you're doing something wrong by doing so, you might say, "Feel free to experiment and see what sounds better to you. But there is no question that the sharpest filter does a better job of filtering ultrasonic noise."

My second issue is related to the first. When you come to the conclusions section, you say, "I sit back and just contemplate: what did I just see and hear?" Yet there is not a single word in your conclusion about what you heard, only about what you saw. Did you even listen to the thing? I can't tell from your review.

Now I know from other reviews that you do listen. But I'm not sure whether you always do so, or whether you always form your listening impressions before you measure so as to avoid having the measurements bias your impressions. That way, we have a better understanding of whether the issues in the measurements actually impinge on your enjoyment. Yes, they are only one man's impressions, but since nobody can borrow someone else's ears, it always comes down to what did Amir hear, what did Jack hear, etc. People decide which reviewer's listening impressions they value and trust.

 

[RandomEar]

Great little primer with which I had two issues from the subjective side.

You say about digital filters: "There's this fallacy where people think that the slow filters are better. No. They're not better...You want to have the sharpest, nicest filter."

I don't take issue with the fact that the sharp filter is better at filtering. Nevertheless, subjectively, I find the most gradual min. phase filter setting on Audirvana the most pleasing. It sounds more relaxing to me than the steepest possible linear filter setting. (Assuming I'm actually hearing any difference at all.) So --

The issue I have is not THAT you made a pronouncement about which filter is best at filtering, but HOW you did so, particularly as you become prescriptive (this is what you want). My recommendation is that you don't make people feel that there is something wrong with them for preferring a slower filter. Rather than saying "feel free to muck around with the filters," as if you're doing something wrong by doing so, you might say, "Feel free to experiment and see what sounds better to you. But there is no question that the sharpest filter does a better job of filtering ultrasonic noise."

This might be a misunderstanding. I think the critique from Amir is mostly that some people come along and actually say "slow filters are better" and then argue some nonsense about less filtering = more music or whatever. Then they might continue to argue that it also sounds better to them and just use their obscure arguments to explain why. So you're not wrong, but Amir isn't either. Slightly different (imaginary) situations.

Also, if you're older than about 40, it is extremely unlikely that you would be able to audibly distinguish slow roll-off from fast roll-off filters for CD quality material (44.1 kHz). For anything with higher sampling rates (>48 kHz), it's pretty much a given that there will be no audible difference at all. Apart from different delays which can influence production work flows, off course - but we are talking about at home listening here.

My second issue is related to the first. When you come to the conclusions section, you say, "I sit back and just contemplate: what did I just see and hear?" Yet there is not a single word in your conclusion about what you heard, only about what you saw. Did you even listen to the thing? I can't tell from your review.

Now I know from other reviews that you do listen. But I'm not sure whether you always do so, or whether you always form your listening impressions before you measure so as to avoid having the measurements bias your impressions. That way, we have a better understanding of whether the issues in the measurements actually impinge on your enjoyment. Yes, they are only one man's impressions, but since nobody can borrow someone else's ears, it always comes down to what did Amir hear, what did Jack hear, etc. People decide which reviewer's listening impressions they value and trust.

[Emphasis added]

Well, that's exactly the problem ASR is there to solve. ASR is about hard data. Things you can check, reproduce and rely on - in contrast to someone else's ears. You can listen to 50 subjective reviewers impressions about an amp and come away with an opinion. It might be a valid opinion about the usability, the build quality or the design. But subjetive impressions about the sound are nearly 100% bullshit. Most modern DACs and amps don't have disctinct differences in sound anymore. Apart from the noise level in some amplifiers (-> tweeter hiss), you will be hard pressed to differentiate between them in a blind test and people already failed at such tests 40 years ago.

So if there are no or at least no significant audible differences, what could you possibly gain from listening impressions by people who want to convince you of the opposite? You need to remember that for all those reviewers and magazines, their views and clicks and thereby their add revenue and income depend on it. Maybe they even believe themselves that those differences exist. But even if they don't, it is 100% in their best interest to tell you that there are differences even if there are none, because otherwise they would be out of a job.

 

[antcollinet]

Did you even listen to the thing? I can't tell from your review.

Typically Amir listens to speakers - where there is a difference to be heard.

Rarely speakers amps - pretty much no point.

I'm not sure he ever listens to dacs. Unless they are broken by design, they are going to sound the same as any other.

(edited for correction)

 

[formdissolve]

Typically Amir listens to speakers - where there is a difference to be heard.

Rarely speakers - pretty much no point.

I'm not sure he ever listens to dacs. Unless they are broken by design, they are going to sound the same as any other.

The only filter I ever heard a difference on was some dramatic and drastic "tube" or other esoteric idea. Mainly coloration, extreme distortion, etc. I can't hear a difference between fast or slow filters, post- or pre-ringing etc. I can hear up to 17kHz. I don't blame Amir or anyone for not wasting their time trying to hear differences between some filters.

 

[DWPress]

I don't blame Amir or anyone for wasting their time trying to hear differences between some filters.

In total agreement here. I know many are championing the new amp specs with no frequency dependencies under loads but +/- 1dB at 20Hz or 20K has also never been a big concern of mine....

 

[EJH]

So if there are no or at least no significant audible differences, what could you possibly gain from listening impressions by people who want to convince you of the opposite? You need to remember that for all those reviewers and magazines, their views and clicks and thereby their add revenue and income depend on it. Maybe they even believe themselves that those differences exist. But even if they don't, it is 100% in their best interest to tell you that there are differences even if there are none, because otherwise they would be out of a job.

Typically Amir listens to speakers - where there is a difference to be heard.

Rarely speakers amps - pretty much no point.

I'm not sure he ever listens to dacs. Unless they are broken by design, they are going to sound the same as any other.

(edited for correction)

I don't know about that, guys. Amir doesn't only give measurements; he also passes judgment, recommends or does not recommend that you buy a product. It's inconceivable to me that he would recommend or not recommend a piece of audio gear without bothering to listen to it. It is, after all, audio gear. I mean, even if it's to hear how bad something sounds.

The "if there's no difference, why bother" argument cuts both ways. If an item measures significantly worse than another, and the difference isn't audible, and audio equipment is meant to be heard, why pan or promote a product on that basis? Maybe manufacturer x decided to invest in other things rather than achieving perfect numbers, numbers that don't mean anything when you listen to music. By ranking equipment based on these measurements, you incentivize and reward manufacturers who waste their time on stuff that has no audible value. It's actually a lot like promoting high resolution recordings that reproduce frequencies that nobody can hear.

I object to the phrase "broken by design." Just because one person doesn't like something that veers from the perfectly transparent and neutral doesn't mean that another person can't buy and legitimately enjoy gear that's "colored" in one way or another. They're not buying broken gear; they're buying gear that works as intended.

Thank you for taking my little critique seriously enough to offer thoughtful responses. I enjoyed reading them. Cheers!

 

[Palmer]

Thank you so much everyone. What would you like the next speed run video be about? I am thinking of either headphone amplifiers or speakers.

Fast running turntables

 

[antcollinet]

It's inconceivable to me that he would recommend or not recommend a piece of audio gear without bothering to listen to it.

The measurements of a DAC or an amp tell you everything you need to know. Listening adds no more information. Remember Amir is subject to cognitive biases just like everyone else.

 

[EJH]

The measurements of a DAC or an amp tell you everything you need to know. Listening adds no more information. Remember Amir is subject to cognitive biases just like everyone else.

OK if the assumption is that all amps and dacs sound the same, then I see why there’s no point in listening. My experience has been that they all don’t sound the same, but I understand that that is just the kind of statement that would be rejected here. Still, if they all sound the same, I don’t see why bad measurements should be penalized.

 

[dc655321]

OK if the assumption is that all amps and dacs sound the same, then I see why there’s no point in listening. My experience has been that they all don’t sound the same, but I understand that that is just the kind of statement that would be rejected here. Still, if they all sound the same, I don’t see why bad measurements should be penalized.

DACs and amps are electrical, not acoustic devices. How would one listen to such a thing?
There is little to nothing that our senses could bring to bear on the signals produced.
So... we look at the electronic information.

 

[antcollinet]

OK if the assumption is that all amps and dacs sound the same, then I see why there’s no point in listening. My experience has been that they all don’t sound the same, but I understand that that is just the kind of statement that would be rejected here.

Your experience presumably based on uncontrolled sighted listening.

Yes - such comparisons will be rejected.

Still, if they all sound the same, I don’t see why bad measurements should be penalized.

Again with the straw man.

No-one ever says that - what is said is that all devices that measure with inaudible noise and distortion will sound the same. Devices with catastrophically bad measurements - can give themselves away audibly.

 

[Chagall]

OK if the assumption is that all amps and dacs sound the same, then I see why there’s no point in listening. My experience has been that they all don’t sound the same, but I understand that that is just the kind of statement that would be rejected here. Still, if they all sound the same, I don’t see why bad measurements should be penalized.

The more precise claim is: competently designed amps and DACs, operated within their limits, are often audibly transparent.
That is very different from saying listening has no value, or that gear can never sound different.

Amps can sound different if they have high output impedance, insufficient power, noise, distortion, clipping, channel imbalance, frequency-response interaction with the headphone/speaker load, DSP, filters, or deliberately colored design. DACs can also differ if something is broken, poorly implemented, level mismatched, or using unusual filtering/output stages.

The reason bad measurements are penalized is not because they are always instantly audible. They are penalized because they show poor engineering. A device with higher noise, distortion, output impedance, weak power delivery, or bad channel balance may still sound fine in one setup, at one volume, with one headphone. But it is less robust, less transparent, and more likely to create audible issues in harder conditions.

So the ASR position is not “never listen.” It is more like: listen, but don’t treat uncontrolled sighted impressions as reliable proof of audible difference. Measurements tell us whether a device is likely to be transparent, whether it has enough power, whether it behaves well into real loads, and whether the manufacturer did a competent job.

 

[MAB]

OK if the assumption is that all amps and dacs sound the same, then I see why there’s no point in listening. My experience has been that they all don’t sound the same, but I understand that that is just the kind of statement that would be rejected here.

Honestly, if you were to demonstrate that you could reliably tell the difference between DACs that were playing at the same volume, in a controlled test where you had no idea which one is playing, that would be remarkable. People here would be extremely interested in your results, the DAC used, what exact artifact you heard, which filter, etc.

Still, if they all sound the same, I don’t see why bad measurements should be penalized.

Bad measurements = poor engineering.

Take noise. Most people wouldn't know if they had some electronics with elevated noise. High sensitivity drivers will make noise apparent in many cases.

Since good engineering often uses the same component and design cost as poor engineering, it's useful to evaluate the resulting performance of the gear to the limit of the test equipment. Our ears are several orders less sensitive, accurate, and precise than test equipment. Therefore performance results extend way beyond human audibility. This is common when ranking results that extend beyond a threshold.

 

[EJH]

Again with the straw man.

No-one ever says that - what is said is that all devices that measure with inaudible noise and distortion will sound the same. Devices with catastrophically bad measurements - can give themselves away audibly.

Absolutely not a straw man. Here's Amir ranking the top 20 SINAD measurements, stating "higher Better." Presumably every single one of these amps is fully transparent with distortion that is completely inaudible. Yet they are being ranked based on these measurement, measurements that don't impinge in the slightest on what you can hear. Presumably, all these amps sound identical. Can you guys then just admit, based on the mantra "higher Better," that DSD256 and PCM384 are better than redbook?

Bad measurements = poor engineering.

Bad measurements=poor engineering. Is that always true? In any case, you can see from Amir's SINAD measurements above that even totally transparent, well engineered amps are being penalized by him. The top dog in such rankings is the guy with the best SINAD measurement. Is that really how we should be ranking audio gear?

TO BE CLEAR: I've never noticed (nor have I even looked for) a difference in stand-alone DACs but in single unit DAC/amps. Heck I am trying out Schiit's Asgard X and it makes my he560v1.2 headphones sound like two completely different headphones depending on whether I plug them into the balanced 4.4 "halo" or quarter inch jack.

 

[RandomEar]

Absolutely not a straw man. Here's Amir ranking the top 20 SINAD measurements, stating "higher Better." Presumably every single one of these amps is fully transparent with distortion that is completely inaudible. Yet they are being ranked based on these measurement, measurements that don't impinge in the slightest on what you can hear. Presumably, all these amps sound identical. Can you guys then just admit, based on the mantra "higher Better," that DSD256 and PCM384 are better than redbook?
View attachment 537218

That's a misleading example, because:

• "Higher is better" is a totally normal label for scientific graphs and plots. It helps the reader - who may not know the metric by heart - understand quickly what to look for.
• The full graphs span from "absolute train wreck" to "exceptional performance". You have deliberataly chosen the absolute top of the top of the graph to make a misleading point.

Yes, Amir generally concludes that anything with a SINAD of 115 dB or better is fully transparent. No, that does not mean that the statement "Higher is better" is in any way incorrect. The full ranking also includes devices which are far from transparent and if a device is ending up in the middle or at the bottom of the ranking, the graph will still have the same helpful and correct labelling. Prime example from [1]:

Bad measurements=poor engineering. Is that always true? In any case, you can see from Amir's SINAD measurements above that even totally transparent, well engineered amps are being penalized by him. The top dog in such rankings is the guy with the best SINAD measurement. Is that really how we should be ranking audio gear?

If you compare two devices of the same class and price bracket with similar internal structure and one performs significantly better in a specific metric while it's at least equal to the second unit in all other metrics, which would you conclude is engineered better? There's certainly other aspects like longevitiy, but you simply can't test those without investing unreasonable amounts of time and/or money. At least unreasonable for one person just doing this as a hobby.

I'm fine with ranking by one clear and relatively easy to measure metric. SINAD isn't everything and nobody here claims that it is. But it's 100 times better than "yeah, so Mike said he heard that the treble sounds smoother than on XYZ"-bulllsh*t. How would you prefer to rank them? Make a suggestion.

TO BE CLEAR: I've never noticed (nor have I even looked for) a difference in stand-alone DACs but in single unit DAC/amps. Heck I am trying out Schiit's Asgard X and it makes my he560v1.2 headphones sound like two completely different headphones depending on whether I plug them into the balanced 4.4 "halo" or quarter inch jack.

Balanced gives you four times the power. Unless you're running into a power limit and clipping with the unbalanced connection, there's no reason why it would sound any different. Also, not relevant to the topic being discussed.

 

[antcollinet]

Absolutely not a straw man. Here's Amir ranking the top 20 SINAD measurements, stating "higher Better." Presumably every single one of these amps is fully transparent with distortion that is completely inaudible. Yet they are being ranked based on these measurement, measurements that don't impinge in the slightest on what you can hear. Presumably, all these amps sound identical. Can you guys then just admit, based on the mantra "higher Better," that DSD256 and PCM384 are better than redbook?
View attachment 537218


Bad measurements=poor engineering. Is that always true? In any case, you can see from Amir's SINAD measurements above that even totally transparent, well engineered amps are being penalized by him. The top dog in such rankings is the guy with the best SINAD measurement. Is that really how we should be ranking audio gear?

TO BE CLEAR: I've never noticed (nor have I even looked for) a difference in stand-alone DACs but in single unit DAC/amps. Heck I am trying out Schiit's Asgard X and it makes my he560v1.2 headphones sound like two completely different headphones depending on whether I plug them into the balanced 4.4 "halo" or quarter inch jack.

Your statement that we say here "all DACs sound the same" is the straw man.

 

[EJH]

1) You have deliberataly chosen the absolute top of the top of the graph to make a misleading point.

2) Balanced gives you four times the power. Unless you're running into a power limit and clipping with the unbalanced connection, there's no reason why it would sound any different. Also, not relevant to the topic being discussed.

1) Wrong. I did not deliberately choose the absolute top of the top; it's what Amir himself chose to highlight with the caption saying that the unit "topped the charts" and promoting it as "finishing easily in our top 20 DACs ever tested (out of 400 units!!!)." Clearly, he is promoting certain products based on a metric that in the vast majority of cases and by your own admission is completely inaudible. By topping the chart, he's promoting and hyping it as though it were the best DAC ever, as in: the top dog slot goes to... So now manufacturers are incentivized to fall over themselves to earn that top dog status, to rank number one, even though the ranking is largely or completely irrelevant to the audible results.

2) It sounds like two completely different headphones. Why that is or what the "halo" feedback feature has to do with it, if anything, I don't know. I do want to correct something I wrote: the 4.4 jack requires a balanced cable but it itself is not balanced. The difference is the halo feedback. I think the amp itself is a no-feedback unit except through the 4.4 headphone jack. Schiit describes halo as a "motion feedback system that includes the transducer in the feedback loop."

Your statement that we say here "all DACs sound the same" is the straw man.

OK then I'm misunderstanding your own statement that the "measurements of a DAC or an amp tell you everything you need to know. Listening adds no more information" or dc's statement that "there is little to nothing that our senses could bring to bear on the signals produced." I'm pretty sure that I have read statements saying that "all DACs sound the same" here on the ASR forums, but I would have to go back and check.

 

[antcollinet]

OK then I'm misunderstanding your own statement that the "measurements of a DAC or an amp tell you everything you need to know.

Deliberately perhaps.

Measurements tell you everything you need to know becasue they tell you if a device is audibly perfect. There's no point in accepting lower performing DACS than this because the vast majority have inaudible noise and distortion. However DACs with poor measurements (very poor for most people in real world listening) may sound different.

 

[EJH]

Deliberately perhaps.

Measurements tell you everything you need to know becasue they tell you if a device is audibly perfect. There's no point in accepting lower performing DACS than this because the vast majority have inaudible noise and distortion. However DACs with poor measurements (very poor for most people in real world listening) may sound different.

I was not deliberately misunderstanding. Thanks for the clarification.

"DACs with poor measurements." I'd be interested to get your take on the claim made in the following video (starting at about 13:45) that "a small number of high-end DACs are doing something genuinely audibly distinct." In this category, he includes Chord DAVE, dCS Vivaldi/Bartok, and MSB Ref. Elsewhere in the video, he gives a friendly and respectful shout-out to Amir and Audio Science Review.

 

[RandomEar]

1) Wrong. I did not deliberately choose the absolute top of the top; it's what Amir himself chose to highlight with the caption saying that the unit "topped the charts" and promoting it as "finishing easily in our top 20 DACs ever tested (out of 400 units!!!)." Clearly, he is promoting certain products based on a metric that in the vast majority of cases and by your own admission is completely inaudible. By topping the chart, he's promoting and hyping it as though it were the best DAC ever, as in: the top dog slot goes to... So now manufacturers are incentivized to fall over themselves to earn that top dog status, to rank number one, even though the ranking is largely or completely irrelevant to the audible results.

He's cropping the top of the chart because, as my other example demonstrated, it's gotten impossible to read with every device showing. You chose this example to make a point, I showed a counter-example. Other members and me have already suggested to implement a different way of showing the chart, maybe interactive, but for now, it is what it is. I have no issue with the statements Amir made, if they are what you quoted. It's not a "promotion" as much as it sounds like he's a bit amazed / happy about the excellent results the DAC delivers. I do not see a problem with this. I also see no problem with manufacturers further improving the measured performance of their devices. It does not hurt the audible performance. I also don't need to switch to the latest "best" DAC, just because its SINAD is 3 dB higher. I know that the device I own is good enough. Amir stating that others perform better in a certain metric is not a problem from my perspective.

And again: What do you suggest ASR should rank the devices by? You can criticize this on and on, but it won't achieve anything positive until there is a better alternative. And since you are so against SINAD, what would be your preferred ranking method?

2) It sounds like two completely different headphones. Why that is or what the "halo" feedback feature has to do with it, if anything, I don't know. I do want to correct something I wrote: the 4.4 jack requires a balanced cable but it itself is not balanced. The difference is the halo feedback. I think the amp itself is a no-feedback unit except through the 4.4 headphone jack. Schiit describes halo as a "motion feedback system that includes the transducer in the feedback loop."

OK, if they implement some kind of proprietary filter and it's not even actually balanced: Needs to be measured to be sure, could sound different.

"DACs with poor measurements." I'd be interested to get your take on the claim made in the following video (starting at about 13:45) that "a small number of high-end DACs are doing something genuinely audibly distinct." In this category, he includes Chord DAVE, dCS Vivaldi/Bartok, and MSB Ref. Elsewhere in the video, he gives a friendly and respectful shout-out to Amir and Audio Science Review.

This is getting off-topic and into the territory of this old thread. The problem with many high-end DACs is that manfuacturers often do not care to have them measured by third parties and most owners of these devices aren't really amongst the ASR crowd. So we don't get to see them tested here. The Chord DAVE is an exception and - ironically - has several potentially audible flaws, including a modulating noise floor, mediocre power into 300 Ω and a badly documented treble roll-off filter. So in essence: Yes, expensive DACs will sound different if they are badly engineered. Just like cheap ones.

My opinion: The main problem with expensive DACs and their often obscure designs is that those manufacturers simply lack the engineering expertise to actually pull this off and design them properly. They end up with non-standard (a.k.a "unique") DACs and use the non-standardness for their marketing, but the objective performance of those devices is often worse than some 200 $ unit just implementing the ESS/AKM/Rohm/etc. reference design. The more you spend past some very reasonable threshold for DACs, the higher the likelyhood for significant design flaws. That's also not surprising at all, because ESS or AKM have been in the business of designing DAC chips for a couple of decades and each new chip builts on their existing expertise plus a couple 10 000 man hours of experienced engineers optimizing it even further. That's just orders of magnitude more engineering time and accumulated knowledge than some random niche manufacturer like CHORD could ever afford.