DAC audible differences are finally proven with ABX?

[jabbr]

That is the kind of question that would drive making good measurements to see what kind of influence it has. I don't think vague descriptions of listening impressions are going to do much for answering the question.

This is from the website of one of the DUT. I don't claim to have any experience with the Matrix. It seems to be accepted as a "transparent" DAC, but the successor (ND-1) has two listening modes which the manufacturer claims to have two sonic signatures. Is this still considered "transparent"? Not my vague description. Yes presumably each signature has a measurable correlate.

 

[danadam]

If so, what was the device attenuating so much the signal?

He wrote he was using the 2.5mm AUX input of the Cosmos, so that's probably part of the reason.

 

[pjug]

He wrote he was using the 2.5mm AUX input of the Cosmos, so that's probably part of the reason.

The 2.5mm is for measuring power amps, I thought.

 

[rjp]

Depends on which aspects. I explained the noise and distortion levels. Once they are below threshold, especially when ignoring masking for the latter, then that is that.

The situation we have is that some of these products don't just meet that target but go way past it:

Distortion products are at -145 dB or 30 dB better than we need them. SINAD is dominated by noise and clocks at around 120 dB:

Help me understand what I am missing please.
It is abundantly clear that nobody could hear any of the artifacts in these figures. of course.

So let's go over what we measured here.
(1) We know that when the input is a 1kHz sine wave, the output is also a 1kHz sine wave, and the noise level is still below -150dB, and THD is nearly as low.
(2) Even more impressive, when we input a complex signal consisting of the sum of 32 individual tones spanning 20-20kHz the output is extraordinarily clean, noise+harmonics are still far below the established limits of human hearing.

Based on these tests alone are we making the conclusion that, for a typical human listener, the output of this system is indistinguishable from its input for any audio signal?
This is the part I need help with.

I had previously thought that we made this conclusion based on the fact that blind listening tests correlate well with measurement thresholding, but it sounds like you may be saying the measurements alone are sufficient without the ABX backing. Am I following correctly?

 

[pjug]

This is from the website of one of the DUT. I don't claim to have any experience with the Matrix. It seems to be accepted as a "transparent" DAC, but the successor (ND-1) has two listening modes which the manufacturer claims to have two sonic signatures. Is this still considered "transparent"? Not my vague description. Yes presumably each signature has a measurable correlate.

If the manufacturer says it has two sonic signatures, then they should be able to show the differences with measurements and graphs. But if Matrix is doing something to the sound, maybe intentionally, possibly you are onto something. Did Josh say what mode he used?
edit: oh the model Josh used doesn't have the transformer mode, is that right?

 

[RexrothPigeon]

Josh did not seek scrutiny and did not post his findings here.

he did post his findings "here" (the publicly accessible internet)

 

[solderdude]

When you compare a DAC that has near perfect performance to one that doesn't the result is a noticeable difference is detected despite one DAC being near perfect.
One can have a preference and hear a difference but unless you trust one of the DACs to be (near perfect) you can't know how that what you heard is how it is supposed to be.

Also the floor of an FFT is NOT the actual noise floor and signals seen just above the noise floor in reality would be buried in the noise floor (when it isn't a constant tone).

 

[kemmler3D]

I don't see how either of those targeted specifically our ability to detect minute colorations within a device's frequency response.

Amir's point is the study he linked to would be considered as testing the absolute quietest thing people can hear. So any other test including what you're proposing would be considered less rigorous than that.

 

[solderdude]

he did post his findings "here" (the publicly accessible internet)

So everything that was ever posted somewhere and was linked to by someone on ASR is posting 'here' ?

Only after he became aware of this thread (being a member) did he engage in this discussion.

 

[RexrothPigeon]

So everything that was ever posted somewhere and was linked to by someone on ASR is posting 'here' ?

Only after he became aware of this thread (being a member) did he engage in this discussion.

"here" is an interesting word, requiring a "there" to differentiate and define it. in physical space, it represents some amount of distance from "there," sometimes even barriers (physical, psychological, legal) impeding movement between "here" and "there".

the internet, though, is all one big "here" though, no page farther away than any other, a child's blog a single click from a behemoth like Amazon.

so yeah, anyone posting anything on the public internet may as well be posting it on the entire public internet.

 

[NTTY]

He wrote he was using the 2.5mm AUX input of the Cosmos, so that's probably part of the reason.

The 43Vrms? Oh ok, got it. Indeed it is the 2.5mm input.

That makes no sense. But it explains the massive headroom of the ADC.

Isn’t all of this a bad joke?

 

[danadam]

The 2.5mm is for measuring power amps, I thought.

True. I don't know why he used that instead of XLR inputs. Maybe he didn't have the cable?

In any case, the Cosmos ADC DM page shows that the AUX level is 20 Vrms at the highest sensitivity (I assume that's the same as for the older models, for some reason their pages are not there anymore). Josh used Moondrop S8, with sensitivity 122 dB SPL/Vrms. Assuming he set the volume to get those 120 dB, that's already only -26 dBFS at the ADC. But that might be dangerous for the ears, so let's say he set the volume to "only" 100 dB SPL. That's -46 dBFS at the ADC. And then we don't really know what was the Cosmos sensitivity setting, so those values could be even lower.

 

[donjoe0]

It is not a matter of questioning the limits of human hearing, that seems to have been well established.

But it is partly that too. I have several connected concerns: (1) the limits haven't been as well established as some seem to think, since there are too few studies and most importantly too few participants to generalize the result, (2) ASR users who love using abusive language against those "denying the science" often don't even know what those limits are (for coloration detection ability) and (3) most measurements of DACs so far don't even zoom in enough to show how the device performs at those limits (for coloration detection ability, which would seem to need showing at least 0.08 dB resolution on the FR graph).

OR Josh hands over recorded outputs of the DACs via the comparator and proves that there were no other tells

Sure, I would love to see that myself, but it would not by itself address all my concerns above.

Everything has a cost so we don't take things to extremes. We test enough to build high confidence and stop there.
[...]
Audio Science has its standards and it doesn't cater to lay opinions in this manner.

Audio science is not some special kind of science that can handwave away the rules of the rest of science and the rules of statistics. An experiment with only tens of participants can't be hyper-confidently extrapolated to the whole 8.3 billion human population, this is as true in audio science as it is in medicine. Call it a "lay opinion" as much as you want, it does not excuse audio science from the rigors of science in general.

Amir's point is the study he linked to would be considered as testing the absolute quietest thing people can hear. So any other test including what you're proposing would be considered less rigorous than that.

It's not a test of the same ability, and there's no obvious way to translate between the two. It's simply missing the point of what I was asking.

 

[AndyBell]

So everything that was ever posted somewhere and was linked to by someone on ASR is posting 'here' ?

He posted them publicly.

That invites feedback and scrutiny.

Do that with anything other than a rock solid case and that scrutiny might become uncomfortable.

Such is life…

 

[kemmler3D]

there's no obvious way to translate between the two. It's simply missing the point of what I was asking.

The absolute limits of human hearing have been studied quite a bit, why would music be subject to different thresholds of detection?

there are too few studies and most importantly too few participants to generalize the result,

Hmm... how many studies have been conducted and how many participants were involved since... I dunno, 1920? And how many would be enough? Where are the actual statistical weaknesses? Small samples can still establish narrow confidence intervals if the values are close together.

If your argument is "there isn't enough science" can you at least say how much we have and how much more we need?

 

[solderdude]

Audio science is not some special kind of science that can handwave away the rules of the rest of science and the rules of statistics. An experiment with only tens of participants can't be hyper-confidently extrapolated to the whole 8.3 billion human population, this is as true in audio science as it is in medicine. Call it a "lay opinion" as much as you want, it does not excuse audio science from the rigors of science in general.

So does the test (which raised some eyebrows and concerns) done by one person invalidate all previous knowledge of audibility levels.
It requires much more info than we have now and at least a few more tests by others using a similar method and arriving at the same conclusion.

 

[solderdude]

He posted them publicly.

That invites feedback and scrutiny.

Yep on AS and CA.

Sure this invites scrutiny and feedback and that's what he got in spades and deservedly so.
Sadly there are no satisfying/clarifying answers to many questions posted at ASR nor elsewhere.
But indeed ... such is life

 

[NTTY]

True. I don't know why he used that instead of XLR inputs. Maybe he didn't have the cable?

In any case, the Cosmos ADC DM page shows that the AUX level is 20 Vrms at the highest sensitivity (I assume that's the same as for the older models, for some reason their pages are not there anymore). Josh used Moondrop S8, with sensitivity 122 dB SPL/Vrms. Assuming he set the volume to get those 120 dB, that's already only -26 dBFS at the ADC. But that might be dangerous for the ears, so let's say he set the volume to "only" 100 dB SPL. That's -46 dBFS at the ADC. And then we don't really know what was the Cosmos sensitivity setting, so those values could be even lower.

The ADCiso page (here) only mentioned the max, not the min.

Either way, it means tiny voltage at ADC input, and it is not a proper way to measure the DUT, since the ADC lost 8bits of resolution. That means it’s left with less than 13bits to analyze high resolution devices. This is wrong as no valid/meaningful measurements can be performed that way.

But the SFDR says it all, 76dB is stupidly low. The CDA (16bits) can do 110db to 136db. So what we’ve been provided as a proof of "the box is not asymmetrically altering the signal" is meaningless

 

[pma]

So what we’ve been provided as a proof of "the box is not asymmetrically altering the signal" is meaningless

And it only confirms that incompetent approach and lack of knowledge of the author.

 

[Plcamp]

1) the limits haven't been as well established as some seem to think, since there are too few studies and most importantly too few participants to generalize the result,

How do YOU know that is true? Why should we believe your wild claim that there have been “too few”? On what scientific basis are you relying for anything you say on this?

2) ASR users who love using abusive language against those "denying the science" often don't even know what those limits are (for coloration detection ability) and

That’s just a BS claim you have pulled from the ether, you have zero knowledge about what ASR members might know and don’t know.

And even if there are those who insult and use abusive language…that has nothing to do with the truth being sought here, and you know that.

“Colouration detection ability” is your made up term and it’s utterly undefined…so it is of zero impact. Be specific. Quote your sources. Stop the trolling nonsense.

3) most measurements of DACs so far don't even zoom in enough to show how the device performs at those limits (for coloration detection ability, which would seem to need showing at least 0.08 dB resolution on the FR graph).

Jibberish nonsense. Where did you pull 0.08 from? Explain your claim.