DAC audible differences are finally proven with ABX?

[rjp]

Note that the answer I gave you is far more powerful than what you ask. Instead of relying one expensive and error prone blind tests, we use threshold of hearing. Once a device's impairments land below that, then we know, with extreme confidence that listeners won't be able to detect them. That is the case because the analysis is conservative and assumes such things as masking does not exist.

I think I may understand what donjoe0 is getting at as I have had similar frustrations along the way.
It is not a matter of questioning the limits of human hearing, that seems to have been well established.
What I think some people are struggling to understand better is how do we justify the assertion that the differences between two products under test are indeed below this threshold? i.e., what make a set of tests sufficient?

 

[Thomas_A]

Josh did address this in the update. I am not familiar with how polarity is determined from multitone but I assume there is something in the software that gives this.

The RME, Matrix, Gustard, SMSL, and Schiit DACs all maintain absolute polarity, according to both published measurements and my own Multitone measurements on my specific units. I did not adjust the polarity of any DACs with an inversion option in the settings. The ABX box’s A and B paths share the same polarity, so polarity can’t differ between the two sides of a comparison
I can’t verify the D900’s filter or polarity, since I no longer own it. The D900 was measured by Audio Science Review, and inverted polarity wasn’t mentioned. However, polarity isn’t always addressed in reviews.

He states that the Matrix retains absolute polarity from published measurements. If someone could ask what publshed measurement he refers to it would be nice. Also to publish the waveform of source file and output.

 

[solderdude]

Josh did address this in the update. I am not familiar with how polarity is determined from multitone but I assume there is something in the software that gives this.

The RME, Matrix, Gustard, SMSL, and Schiit DACs all maintain absolute polarity, according to both published measurements and my own Multitone measurements on my specific units. I did not adjust the polarity of any DACs with an inversion option in the settings. The ABX box’s A and B paths share the same polarity, so polarity can’t differ between the two sides of a comparison.

I can’t verify the D900’s filter or polarity, since I no longer own it. The D900 was measured by Audio Science Review, and inverted polarity wasn’t mentioned. However, polarity isn’t always addressed in reviews.

The only way he can judge polarity without closely looking at the signal waveform in an editor or if he tested for this is if he nulled the DACs.
This was not done so the polarity was not checked, the comparator is not able to flip polarity either.
Whatever he described above will tell nothing about polarity.
There is also no indication that the D900 would be inverse polarity anyway.

Yes, OR stop using absolutist terms like "audibly transparent" and stop bullying people who claim to hear DAC defects like they're lunatics denying proven scientific facts corroborated by millennia of experimentation. They should either show how it's 'established science' or stop behaving like it is. Or, you know, stop claiming their approach is scientific, there's always that third option.

OR Josh hands over recorded outputs of the DACs via the comparator and proves that there were no other tells than small FR differences he picked up on successfully at the extremes of the audible band or some very small FR ripple.
Extraordinary claims deserve extraordinary evidence and as the evidence is lacking here the claim is nothing more than that. This works both ways doesn't it ?

 

[amirm]

What I think some people are struggling to understand better is how do we justify the assertion that the differences between two products under test are indeed below this threshold? i.e., what make a set of tests sufficient?

Depends on which aspects. I explained the noise and distortion levels. Once they are below threshold, especially when ignoring masking for the latter, then that is that.

The situation we have is that some of these products don't just meet that target but go way past it:

Distortion products are at -145 dB or 30 dB better than we need them. SINAD is dominated by noise and clocks at around 120 dB:

This is 5 dB better than we need it to be for absolute transparency.

Excitation with multiple signals shows the same:

There is just no way with a straight face one can say these are audible artifacts.

Now, when you lower the performance by say, 10 to 20 dB, then you are in a gray area. And the lower you go, the more gray it becomes. As such, I am not comfortable with statements such as "anything better than 80 dB is transparent." You are deep in the gray area.

As far as I am concerned then, buying state of the art DACs buys you assurance of transparency to the source. That assurance only comes if you can clear the threshold under all tests. Which is what we have here and in many other example DACs.

 

[Thomas_A]

The only way he can judge polarity without closely looking at the signal waveform in an editor or if he tested for this is if he nulled the DACs.
This was not done so the polarity was not checked, the comparator is not able to flip polarity either.
Whatever he described above will tell nothing about polarity.
There is also no indication that the D900 would be inverse polarity anyway.


OR Josh hands over recorded outputs of the DACs via the comparator and proves that there were no other tells than small FR differences he picked up on successfully at the extremes of the audible band or some very small FR ripple.
Extraordinary claims deserve extraordinary evidence and as the evidence is lacking here the claim is nothing more than that. This works both ways doesn't it ?

The uncertainty lies in that the specific Matrix model was released at the same time as the Matrix Element X. The Element DAC had inverted polarity according to Hifi News review. I have not been able to find polarity measurements of the Matrix model used by Josh although he claims it does not have that according to published data. A reference to that review or data, if it exists, is the least required. With both XLR and RCA out.

 

[amirm]

The moon has nothing to do with exploring the generally valid (or absolute) limits of the human organism. Testing lots of human organisms does.

That was a test of limits of human beings, i.e. whether they can breath on the moon.

I don't think asking for results based on thousands of patients would be out of place in medical science.

Everything has a cost so we don't take things to extremes. We test enough to build high confidence and stop there.

If someone like you is not convinced, you are welcome to chase that exception. Rest of us are going to assume the Lock Ness monster and sasquatch don't exist.

I say the same standard should be applied before recognizing anyone's right to talk in absolutist terms like "audibly transparent" and especially "you're an idiot for thinking you actually heard that defect".

Audio Science has its standards and it doesn't cater to lay opinions in this manner. Show that threshold of hearing is magically different for you or someone else and then we can talk. Until then, once we know what we know, we move on to other discoveries. But per above, the door is open for you to show otherwise. Just make sure, unlike our author, to proper respond to attempts to analyze and duplicate the results.

 

[RandomEar]

The uncertainty lies in that the specific Matrix model was released at the same time as the Matrix Element X. The Element DAC had inverted polarity according to Hifi News review. I have not been able to find polarity measurements of the Matrix model used by Josh although he claims it does not have that according to published data. A reference to that review or data, if it exists, is the least required. With both XLR and RCA out.

I have also searched for this and didn't find anything useful. Until we get ADC captures of the DAC output or someone at least shows the impulse response, I don't think we can get a satisfying answer.

 

[solderdude]

As far as I am concerned then, buying state of the art DACs buys you assurance of transparency to the source. That assurance only comes if you can clear the threshold under all tests. Which is what we have here and in many other example DACs.

When ABX'ed with PEQ disabled in standard polarity setting
The best thing is that this DAC does not have all the deviating reconstruction filter options.

 

[pjug]

The only way he can judge polarity without closely looking at the signal waveform in an editor or if he tested for this is if he nulled the DACs.

He said Multitone with capital M so I think he means Paul's software. As I said I am unfamiliar with that but looking at the documentation, Multitone displays input and output of a loopback. So to me it looks like Josh could test for polarity with Multitone as he said.

 

[solderdude]

Makes sense.
If that is the actual case and he verified all comparisons this way then the whole possible polarity issue can be forgotten and can not be the reason for reliable ABX results.
Looking at actual signal and intended signal is the best way to check this.
Unfortunately there aren't many reviews that check for polarity (RCA/XLR/Pentaconn).
Looking at a multitone plot will say nothing about polarity but looking at the waveforms using Multitone does.

 

[Thomas_A]

He said Multitone with capital M so I think he means Paul's software. As I said I am unfamiliar with that but looking at the documentation, Multitone displays input and output of a loopback. So to me it looks like Josh could test for polarity with Multitone as he said.

View attachment 548940

Yes it is simple. And even simpler is to link to the data or review that he claims to have shown no inversion of polarity at RCA output.

 

[pjug]

Yes it is simple. And even simpler is to link to the data or review that he claims to have shown no inversion of polarity at RCA output.

I agree that the update was somewhat helpful but not enough to really get towards answers about all this. Recordings, at the least, would allow looking into a lot more. But I don't think it is polarity now. If we don't want to believe he confirmed the DAC polarities then we have to question the reporting of everything, and I would rather trust that Josh really did what he says he did.

 

[Thomas_A]

I agree that the update was somewhat helpful but not enough to really get towards answers about all this. Recordings, at the least, would allow looking into a lot more. But I don't think it is polarity now. If we don't want to believe he confirmed the DAC polarities then we have to question the reporting of everything, and I would rather trust that Josh really did what he says he did.

Sure. It’s just a thing easy to check ans show. Looking at the PCB of the original matrix x-sabre Pro:

Matrix Audio X-SABRE Pro MQA: Best Audio DAC in the World?

Does anyone know if this is on sale anywhere other than Shenzhenaudio for 11.11 ?

www.audiosciencereview.com

And this one from YT about 5:10 for the Sabre 3 version:

There seem to be alterations around the outputs regarding opamps? But I can’t tell what it means. Two more opamps of the Sabre 3. Why? Needed to get the XLR and RCA the same polarity?

 

[RandomEar]

He said Multitone with capital M so I think he means Paul's software. As I said I am unfamiliar with that but looking at the documentation, Multitone displays input and output of a loopback. So to me it looks like Josh could test for polarity with Multitone as he said.

I've checked the recent comments on audiophilestyle and they confirm this. The Addendum was also pretty clear, but I initially overlooked it because I didn't know about Multitone.

Still, we would not need to have any of these discussions if Josh simply provided the recorded comparator output. He's got that captured and saved already. Because he used it to supply the additional evaluations in the addendum. I don't get why he would make this so difficult - it even creates more work and stress for him to continue supplying additional results piece by piece like he has down so far. Also some new measurement results just bring up additional questions, like this one. And comments about the unusual statistical power and the severe degradation of the comparator output when attenuated are totally ignored.

This all seems more and more like a method to create engagement on that site, which I wouldn't like to believe...

 

[pjug]

his all seems more and more like a method to create engagement on that site, which I wouldn't like to believe...

Well, Chris didn't bury it in the Object-ifi subforum that he created for such subjects as ABX.

 

[jabbr]

I remain of the opinion that most audible differences between high quality DACs ought to be from the analog output stages (say I-V outward). Looking at the Matrix X Sabre, I see that they have the ND-1 (and I'm admittedly entirely unfamiliar which is why I was doing research) and the ND-1 advertises two different output stages which they term "Transformer mode" vs "Op amp mode". They advertise each having a different sonic signature. Pardon my naivete but why isn't this an explanation for different products having slightly difference sonic signatures?

In the article under question, the differences were never claimed to be large, just audible, and why wouldn't that fall into slight output stage differences?

 

[NTTY]

From the addendum:

Does the highlighted text mean there was an attenuation of -50dBFS at the input of the Cosmos ADC?

If so, what was the device attenuating so much the signal?

But I guess it explains the SFDR (I suppose calculated relative to the carrier at -76dB) and the concerns I expressed pages before. The TD+N (-111dB) can’t be better than the SNR. But with such an attenuation, results are unreliable, and all the same because variances between DACs would be buried into the noise floor.

Assuming the Cosmos ADC was set to its highest sensitivity (1.7Vrms), -50dBFS attenuation at its input means the signal was at a very low 0.005Vrms. And it is not possible to test distortion and noise from a high resolution device with such an attenuation.

The added text with this view does not provide more information.

As is, this is non-informative. But I can repeat the same type of measurements, with the software I’m used to use and the same ADC, provided I understood the test parameters.

 

[pma]

I am not sure if it makes any sense and if it is worth any effort to analyze what they are writing there. I am not even curious about Josh’s experiment. The only thing I might be interested in are the properly done DBT tests, with statistically valid outcome.

 

[Chr1]

I remain of the opinion that most audible differences between high quality DACs ought to be from the analog output stages (say I-V outward). Looking at the Matrix X Sabre, I see that they have the ND-1 (and I'm admittedly entirely unfamiliar which is why I was doing research) and the ND-1 advertises two different output stages which they term "Transformer mode" vs "Op amp mode". They advertise each having a different sonic signature. Pardon my naivete but why isn't this an explanation for different products having slightly difference sonic signatures?

In the article under question, the differences were never claimed to be large, just audible, and why wouldn't that fall into slight output stage differences?

This.

 

[pjug]

I remain of the opinion that most audible differences between high quality DACs ought to be from the analog output stages (say I-V outward). Looking at the Matrix X Sabre, I see that they have the ND-1 (and I'm admittedly entirely unfamiliar which is why I was doing research) and the ND-1 advertises two different output stages which they term "Transformer mode" vs "Op amp mode". They advertise each having a different sonic signature. Pardon my naivete but why isn't this an explanation for different products having slightly difference sonic signatures?

In the article under question, the differences were never claimed to be large, just audible, and why wouldn't that fall into slight output stage differences?

That is the kind of question that would drive making good measurements to see what kind of influence it has. I don't think vague descriptions of listening impressions are going to do much for answering the question.