DAC audible differences are finally proven with ABX?

[krabapple]

There is a point at which obsessive preoccupation with a null hypothesis (e.g. it is not possible to discern the difference between two DACs) - to the point of insisting out of quasi-religious doggedness that there must, eventually, be found some explanation for a reported effect other than the one proposed by a competent researcher - becomes ego-driven Scientism rather than good Science.

It’s just as faulty to baulk irrationally at the possibility that an Independent Variable has not been fully isolated.

Arguably the best Science is collegiate in foundation - and people use common sense to know when to respect a p<0.05 - having afforded due scrutiny to a colleague's Method, Design, Procedure, Analysis. The best academic institutions understand “weight of evidence” – what it is – what it is not (in terms of proof).

I hope that Josh's discriminations are vindicated; otoh, I wouldn't be remotely dismayed if Amir's bench work discovered box anomalies.

Whatever the next Chapter, imo NTTY's generous investment would be optimally rewarded with (at least interim) broad consensus regarding possibilities, rather than interminable wrangling for righteousness’s sake.

Whatever this text is meant to convey; the fact is if a result as fantastically extraordinary as Josh's was published, scientists would be demanding (and pursuing) the same validations as here. That is not "scientism". It's plain science.

 

[krabapple]

When you have it drop shipped to Amir at least you'll know that the ABX box was up to snuff and at least the usability of it.
What better than to be able to say the device has been 'measured by Amir'.

Buh buh what if it gets damaged in transit in such a way that it no longer reveals the incredible differences Josh heard, huh?

 

[krabapple]

I also wish Josh would send audio files output from the DACs to be examined, as some people have been requesting.

Hey Josh, send those audio files. Even The Voice Of Temperence and Subjectivity on ASR wants you to.

The voice of me says:
You know it's a dick move not to send them, at this point . You've spent far more of your time on earth writing defensive posts, than it would take to upload audio files. It's just ego /pride stopping you. I hope

 

[NTTY]

But we just spent 50 pages talking about getting this box!

Soon enough.

 

[NTTY]

FWIW:

MUSINGS and LISTENING: On Absolute Polarity / Phase... (And on the joy of the modern audiophile.)

A blog for audiophiles about more objective topics. Measurements of audio gear. Reasonable, realistic, no snakeoil assessment of sound, and equipment.

archimago.blogspot.com

I still have an Asus Essence One that @Archimago used for this test. Should we include it in the testing?
I’m kidding of course

 

[NTTY]

That’s great. Sounds like there’s not really a need for me to test my ABX unless something shows up wrong with this one. Gotta say, I’m selfishly happy because I’m already stressed enough as it is trying to get through the things I have on hand. haha

My only request is that any subjective interpretation of the objective data is well tempered. The way that some of this discussion is gone, I’m concerned we’re gonna get to the level of picking nits beyond reasonable audibility and may be more critical of the ABX box than we might other pieces of electronic equipment.

Edit: speech to text misspellings

Agree, and your help will be much appreciated should we need to have more data from a second box. But that would anyway facilitate your measurements as they would be restricted to those requiring confirmation, if any.

 

[NTTY]

Hey Josh, send those audio files. Even The Voice Of Temperence and Subjectivity on ASR wants you to.

The voice of me says:
You know it's a dick move not to send them, at this point . You've spent far more of your time on earth writing defensive posts, than it would take to upload audio files. It's just ego /pride stopping you. I hope

Unfortunately, as @pma has shown, this is not trivial. The Cosmos ADC has a low impedance input which could impact the end result. Even if Josh had these files, they would be questionable, and he knows that.

Also, and it’s been mentioned before, clock drift between two DACs can be heard, especially in direct comparison. The Cosmos ADC has its own. Mine, after 2 hours of warm up, is +12ppm. That is why I always publish my result compensating the Cosmos drift from a measurement of my GPSDO (Leo Bodnar LBE1421, not to mention it).

What is the limit of audibility, say after 4min of one song played? In other words: what is the required drift difference between the two DACs under test to be easily heard? I think it is pretty low (4ppm?) and it might be one explanation. Eg: the SMSL DO400 and Topping DX1 II have an 8ppm difference between them. Can we hear it after 4min of playing a song?

I’m mentioning this for a reason, and that is because we suffered this issue with a friend, couple of years ago, when trying to test audibility of OPA. I was getting false 100% positive only because he and I were not listening the same way at the song. He was deciding at the beginning of it, thinking an element of the music was sufficient to decide, while I was going to the end. The root cause was clock drift, not OPA difference I suppose a tool like deltawave can easily reveal and even correct that (I’m not used to that tool, so I’m really unsure).

In addition, DACs don’t have the same clocks and their respective phase noise can vary. Josh said he used the Mutec 1.2 to feed the DACs. Maybe this one box has its own clock drift which will transition via SPDIF as I explained many times. Each DAC will slave itself to that clock but will be filtering it (via the PLL) differently depending on the phase noise of their respective internal clock, for a very different end result, on a pitch error perspective.

All that blah blah to say I totally agree that investigating the Van Alstine box will not reply to all questions. And that does not prevent us (me also) from testing independently what could be the root cause of easy DAC identification in live ABX session.

 

[MattHooper]

Hey Josh, send those audio files. Even The Voice Of Temperence and Subjectivity on ASR wants you to.

The voice of me says:
You know it's a dick move not to send them, at this point . You've spent far more of your time on earth writing defensive posts, than it would take to upload audio files. It's just ego /pride stopping you. I hope

LOL. I wonder which approach is more likely to work for inducing someone to do something for us on their free time.


The carrot

or

The Stick: “You need to do this thing for me while I’m calling you a dick”…

I haven’t tried that one yet ;-)

But on the subject of the testing, given the variety of suggestions about what might’ve affected the results, it will be interesting if anybody ends up satisfied in the end.

I hope so

Either way I’m sure there will be something to learn from all the efforts. At least for nincompoops like me.

 

[pablolie]

Buh buh what if it gets damaged in transit in such a way that it no longer reveals the incredible differences Josh heard, huh?

Very likely.

 

[jabbr]

Unfortunately, as @pma has shown, this is not trivial. The Cosmos ADC has a low impedance input which could impact the end result. Even if Josh had these files, they would be questionable, and he knows that.

Its not just the Cosmos ADC that might have affected results. Differences in output impedances between the two DUT when presented with a lowish input impedance amp, or via the passive ABX switch affect the sonic signature. The AP used to determine "transparency" has a 100kOhm input impedance, but DUTs might not have the same behavior with a low input impedance amp. This is also why a recording made with a high input impedance ADC might not show the same differences as a real world amp ... e.g. Genelec monitors which might be 3.6kOhm etc. or even lower with some devices. Input and output impedances are not simple numbers but curves and different curves present different measurable effects which can translate into audible effects.

 

[amirm]

Input and output impedances are not simple numbers but curves and different curves present different measurable effects which can translate into audible effects.

Careful there. Lower input impedance in the pre-amp brings down the levels but in a uniform, frequency independent way. This is a very different effect than output load on a power amplifier where the load represents a complex impedance and hence potentially distort the output.

What could be an issue is introduction of distortion and other anomalies in the unique, electronic pot used in the comparator. That is what we want to investigate. Not the generic case of a DAC and pre-amp having dependency on each other.

As to your comment about AP and its input impedance being 100K, it can down to 600 ohm if needed. Indeed, that is how I run these tests:

As you see, 600 ohm is still an easy load for the D900 and frankly, for any half decent DAC. There are probably crap designed DACs which may misbehave but that will not remotely be the norm. I run the above test so that we have something to compare to stereophile running the same test.

 

[jabbr]

Careful there. Lower input impedance in the pre-amp brings down the levels but in a uniform, frequency independent way. This is a very different effect than output load on a power amplifier where the load represents a complex impedance and hence potentially distort the output.

Who says there is an active preamp? Increasingly there aren’t. The ABX comparator is exactly the sort of device that would hi light this issue. There are real world amps with lower than ideal input impedance. Does a defined transparent DAC have to deal with eg tube amps etc or capacitative outputs etc?

What could be an issue is introduction of distortion and other anomalies in the unique, electronic pot used in the comparator. That is what we want to investigate. Not the generic case of a DAC and pre-amp having dependency on each other.

As to your comment about AP and its input impedance being 100K, it can down to 600 ohm if needed. Indeed, that is how I run these tests:

I’m not saying that these effects aren’t measurable, of course they are, but do you have?

As you see, 600 ohm is still an easy load for the D900 and frankly, for any half decent DAC.

Are these detailed measurements done for DACs which have been deemed “transparent”?

There are probably crap designed DACs which may misbehave but that will not remotely be the norm. I run the above test so that we have something to compare to stereophile running the same test.

Again I’m not saying that these measurements can’t be done but I didn’t see that as part of the definition of “transparent” … I’m looking for edge cases. I don’t want to miss simple explanations

 

[pma]

Amir, would you mind making a comparison of 100k/600ohm load impedance at 1kHz, not at 50Hz? I am usually getting rise of high order distortion components in such test. But not at 50Hz, as the feedback factor is usually higher at 50Hz than at 1kHz for a design based on opamps and also output impedance nonlinearity is thus more pronounced. I know that Stereophile makes that test at 50Hz, but then they rather test a DC nonlinearity at such low frequency.

 

[pma]

Excellent post, @NTTY . Thank you.

 

[mcdn]

There is an off-chance that there is a fault (volume control IC or setting) but that is unlikely.

Sorry but that is the most likely, possibly the only explanation for someone getting 12/12 on a level matched DAC ABX

 

[Jazigo]

I suppose a tool like deltawave can easily reveal and even correct that

I haven't seen the source code, but my guess is that it can estimate a clock drift once it has aligned the start of both samples. To compensate, it needs to do a Dynamic Time Warping, which in turn means aligning any data to fit the source over the given a given window (say 4ppm). This means that I can move freely in that window, but unfortunately would eliminate actual differences in the time domain.

That together with gain alignment, one could ask if we are actually comparing fairly with DeltaWave, since we adjust both axes at the same time to fit the source.

 

[solderdude]

Sorry but that is the most likely, possibly the only explanation for someone getting 12/12 on a level matched DAC ABX

It is, but If that were the case the null of the A and B input would not be -100 if that were to reach audible levels.
Of course that was just a number and was not mentioned at specific attenuation etc.
Anyway, looks like Amir is getting to measure technical performance and expect this to take out the guessing of possible technical issues.

 

[firedog]

Yet he has no time to provide a few files for members to analyze.


While I am qualified for Amazon Affiliate program, I don't use it. Reviews do not have any sponsorship links to any site, amazon or otherwise. It was suggested to use these links years ago to cover costs of running ASR but many did not like the idea, including myself so that was that.

As for "charging for measurements," once in a while a company reaches out to me and asks if I can test their prototype products. I say I will not review such products but can run private test for them. A few times a year they like me to do that. The above example is one such thing. Given how little I charge (fraction of what a lab would), it is a surprise to me that more companies don't take me up on the offer. The aim is to help companies higher performing products, not for me to make money.

If Josh thinks I should test other companies products as part of their R&D for free, let's have him step up for that and I will send all the referrals to him!

As to speakers, yes, I do have a store that sells Ascilab speakers. Product has very thin margins and importing speakers from Korea to sell in US is anything but easy work or money. You come help lift 400 pounds worth of speakers as I had to help my brother do between yesterday and today.


There is no "business model" for ASR or there would have been a lot of them. Members and companies send me products to test. I have invested some $200,000 in gear, tools, etc. and sit there for hours testing them and publishing openly for the world to use. Just today someone asked my permission for publish research on my speaker measurements for an AES Journal paper.

There is a reason we have grown so much as to dwarf AS which existed long before ASR was formed (daily visits):
View attachment 550126

That's a nice "unscientific" approach. You know very well that those numbers have nothing to do with actual visits - they include bots. And how many visitors at each site have tracking blocking software installed? Not reflected here.

As a general comment on the thread, it's certainly possible there were "tells" in the test that Josh unknowingly heard. But I find it interesting that some of the "tells" being discussed here are things that in other threads have been dismissed as inaudible. Well, are they possible tells or are they inaudible? Are they inaudible when you agree with the results and audible when you don't?
There's some entertainment "science" going on here. But hey, that gets those clicks, doesn't it.

 

[solderdude]

Who says there is an active preamp? Increasingly there aren’t. The ABX comparator is exactly the sort of device that would hi light this issue. There are real world amps with lower than ideal input impedance. Does a defined transparent DAC have to deal with eg tube amps etc or capacitative outputs etc?

I’m not saying that these effects aren’t measurable, of course they are, but do you have?


Are these detailed measurements done for DACs which have been deemed “transparent”?

Again I’m not saying that these measurements can’t be done but I didn’t see that as part of the definition of “transparent” … I’m looking for edge cases. I don’t want to miss simple explanations

You can't connect the ADC directly to the ABX box as the attenuator in it is passive and most likely is 20k.
A very low impedance load would affect the 0.1dB step size.
That's also not how Josh used it, he had a headphone amp between the output and the ADC.
Besides it is about the difference between A and B and if the same attenuation were used the effect would be the same on each output.

This is why I would like to see measurements of A versus B with differing attenuation levels (say A not attenuated and B attenuated) preferably with a realistic load (around 10k or so)

Buh buh what if it gets damaged in transit in such a way that it no longer reveals the incredible differences Josh heard, huh?

In my repair days I sometimes got in devices that did not seem to be defective even with weeks of continuous monitoring, went back to the owner only to return months later with the same complaints. Only to find out later that vibrations due to transport temporarily fixed a dodgy contact.
So far the little information provided by Josh does not seem to show issues with the device though.

There's some "entertainment science" going on here.

And there isn't on AS ?

 

[Talisman]

That's a nice "unscientific" approach. You know very well that those numbers have nothing to do with actual visits - they include bots. And how many visitors at each site have tracking blocking software installed? Not reflected here.

As a general comment on the thread, it's certainly possible there were "tells" in the test that Josh unknowingly heard. But I find it interesting that some of the "tells" being discussed here are things that in other threads have been dismissed as inaudible. Well, are they possible tells or are they inaudible? Are they inaudible when you agree with the results and audible when you don't?
There's some entertainment "science" going on here. But hey, that gets those clicks, doesn't it.

checkmate objectivists!!!!