DAC audible differences are finally proven with ABX?

[Leporello]

Or are they? Interesting experiment with positive results (according to the author):

https://audiophilestyle.com/ca/bits-and-bytes/can-i-tell-audibly-transparent-dacs-apart-blind-testing-with-the-van-alstine-abx-comparator-r1440/

As such, the results seem very convincing. In all the tests the author reached almost a full score. In 112 trials combined he only failed twice. This seems to suggest that the differences were pretty obvious and easy to hear.

 

[Mart68]

All credit to him for doing that. Amusing that even though he points out that he used headphones, that he has formal training and that the tells he found were irrelevant to the actual music, the subsequent comments ignore all that in favour of 'Measurements don't show everything' and 'I can hear the difference too!' That's the problem with doing this, it gives such people a straw to cling to.

 

[Redacted]

Would you trust a social scientist (an oxymoron right there, for me at least) to use instrument-based methods to check the volume matching or indeed do anything with kit convincingly?

I suspect the tester is someone who can tell the difference between two CD transports, from the same CD, feeding the same DAC and 'phones! And what happened to peer review? Pseudo-scientists are in a mutual support club on many 'findings', as there is huge pressure in universities to publish papers irrespective of reality, and of course the internet exists mostly to attract clicks.

I've done quite a few ABX blind and double-blind tests and the results were surprising (at least to my victims) but I have not tried several DACs, in this manner, but even if I could reliably tell which (quality, modern) DAC was which, I suspect after playing 20 fave tracks, I'd probably have no idea which sounded "best". In the end, the pressure is to buy more DACs at higher margins!

 

[Redacted]

[Edit - I pressed 'post' before editing]

 

[Leporello]

Would you trust a social scientist (an oxymoron right there, for me at least) to use instrument-based methods to check the volume matching or indeed do anything with kit convincingly?

I suspect the tester is someone who can tell the difference between two CD transports, from the same CD, feeding the same DAC and 'phones! And what happened to peer review? Pseudo-scientists are in a mutual support club on many 'findings', as there is huge pressure in universities to publish papers irrespective of reality, and of course the internet exists mostly to attract clicks.

I've done quite a few ABX blind and double-blind tests and the results were surprising (at least to my victims) but I have not tried several DACs, in this manner, but even if I could reliably tell which (quality, modern) DAC was which, I suspect after playing 20 fave tracks, I'd probably have no idea which sounded "best".

But the consensus here seems to be that even many inexpensive dacs are audibly transparent. I share that view too. But this guy has been able to tell several dacs apart with near perfect results. If they all sound different they cannot be audibly transparent.

 

[Mart68]

Would you trust a social scientist (an oxymoron right there, for me at least) to use instrument-based methods to check the volume matching or indeed do anything with kit convincingly?

I suspect the tester is someone who can tell the difference between two CD transports, from the same CD, feeding the same DAC and 'phones! And what happened to peer review? Pseudo-scientists are in a mutual support club on many 'findings', as there is huge pressure in universities to publish papers irrespective of reality, and of course the internet exists mostly to attract clicks.

I've done quite a few ABX blind and double-blind tests and the results were surprising (at least to my victims) but I have not tried several DACs, in this manner, but even if I could reliably tell which (quality, modern) DAC was which, I suspect after playing 20 fave tracks, I'd probably have no idea which sounded "best".

I give him the benefit of the doubt until shown otherwise.

It doesn't really surprise me that with training and headphones, he can find tells between DACS.

One thing that's missing from his write up is any specific details of the tells he found. That would make verification easier if anyone else wanted to replicate the experiment - which would be the next step in any truly scientific enquiry. Not sure why he left that info out.

 

[voodooless]

He has the right ADC... He should publish measurements of all the DACs through the ABX device. All it would need is the ABX device doing something audible to the signal of one of the two DUTs.

Also note that the ABX test protocol of that machine is totally non-standard, and in fact, not ABX at all

 

[RandomEar]

Based on statistics alone, I am extremely sceptical of these results. I have done a couple of "easier" ABX tests, for example between a signal where there was a small gradual drop of up to 0.5 dB in the response above 6 kHz. Even with these comparatively major differences, I did not score 10/10 in each test, repeatedly again and again. It is hard to distinguish such tiny differences and your ears occasionally will fail you. So 110 out of 112: I am extremely, extremely sceptical of that result, based on the numbers alone.

That being said, there's two major things not adressed in the article:

• Choice of reconstruction filters: The X Sabre Pro for example uses a slow roll-off filter by default, most other DACs in the test use a linear fast roll-off by default. Yes, the material was supposedly 96 kHz or upsampled to that. But I would nevertheless expect such essential settings to be listed and - if he was looking for a fair comparison - I would expect the author to select similar or if possible identical filters on all DACs.
• Delay: This is a big one. According to what I have seen in the article, the DACs were not time-aligned (-> directly fed by the same S/PDIF source). Due to the likely different filters in play, the effective delay between the DAC outputs will vary between around 150 µs up to roughly 900 µs, maybe more. Since humans can distinguish delays larger than around 7 to 18 µs depending on which trigger is investigated, this could potentially allow anybody to ABX these DACs under the given conditions. Simply because there's a time difference of multiple hundred microseconds between many pairs of them, or maybe all pairs - we can't know because as far as I could see, the reconstruction filters were not listed for any comparison and delays were not measured.

Soooo... This delay thing needs to be clarified. Otherwise, this looks like a false positive to me.

EDIT: Number of total test results corrected.

 

[voodooless]

Also note that the ABX test protocol of that machine is totally non-standard, and in fact, not ABX at all

Also:

If I understand the test protocol correctly, it should be impossible to have the first one correct all the time. Because you are simply presented with A or B, without having heard the other. Logically one would expect about 50/50 here.

 

[RandomEar]

Also:
[...]
If I understand the test protocol correctly, it should be impossible to have the first one correct all the time. Because you are simply presented with A or B, without having heard the other. Logically one would expect about 50/50 here.

Nah, it's true ABX it seems:

A random number generator assigns the “X” system each round. Using the remote, the listener can switch between the “A” system, the “B” system, and the “X” system.

Honestly: The device is really cool and the test was well done overall. He thought about level matching (maybe even too much ), blinding, quick switching, repetition and so on. If he really did what he wrote about, it's certainly miles above anything I have seen random audiophiles or even many serious creators or magazine writers do.

 

[Redacted]

I'm just glad DACs are as good as they are now.
I can thus expend more time, effort and money on recorded and live music.

 

[voodooless]

Nah, it's true ABX it seems:

You quoted the article, not the device manual..

There is no X at all

 

[RandomEar]

Interesting. Why call it ABX when it's not, though? Maybe the firmware has been updated and the manual is outdated? Certainly another point that needs to be clarified.

 

[killdozzer]

If they all sound different they cannot be audibly transparent.

You make a good point here (given I understood you correctly) - The DACs are either transparent or you can tell them apart. Right? If you've managed to tell them apart, they are obviously not transparent or not sufficiently transparent.

No one ever said it's never the case you can here the difference. The point was - if you can hear it (and you shouldn't), it's not working properly. Other point being that the difference maybe coming from an output stage, some gain or something and not converting zeros and ones into an analogue signal.

 

[Ruediger]

First impression: no mention of any settings on those DACs. I mean those are complex processors and not one-way headphone amps. Factory defaults out of the box? Customized boxes? Which filters have been used?

Second thought: people never make stuff like that up. That's a thing of the art world. No need for any proof it all did actually happen.

 

[voodooless]

Interesting. Why call it ABX when it's not, though? Maybe the firmware has been updated and the manual is outdated? Certainly another point that needs to be clarified.

The manual calls it ABX as well

 

[GGroch]

Interesting, his results were far more accurate than I would expect...so I am skeptical.

Because he is level matching, the resistance within the signal path in the Van Alstine are not identical, right, so how the preamp stage of the DAC responds to that could differ. (I am probably not using the right words).

Also, he does not describe I think doing a null test (2 identical sources) of the VanAlstine to be sure it was calibrated/working properly right?

 

[pjug]

You quoted the article, not the device manual..

View attachment 546462
There is no X at all

Next to last paragraph isn't ABX? (in the snippet above from the manual). I understand that as offerering for each guess the ability to freely switch between A, B, and X.

 

[gzost]

I applaud the author of the test for his efforts, and I do not doubt his sincerity. As others here have said, the results look too perfect, so I expect there to be some confounding variable.

There is a three part article on the comparator used on Dagogo:

Audio by Van Alstine ABX Comparator Review, Part 3: New Twists & Conclusion

Part 1: Audio Store & Wiring Part 2: Trials A new twist, amp comparisons I haven’t said much to this point about the amps used because this is where the ABX testing got downright weird. Though I was able to differentiate between all other components and speakers with some assurance, if not...

www.dagogo.com

(Link to third part, has links to the others at top)

In these Dagogo articles there are a few interesting bits:

There is mention that in the version the author used the relays for switching between A and B sound different, and that the manufacturer fixed this for later versions.

The author claims to not have noticed this, and is certain that he was not influenced by it (!). He also gets near perfect results for distinguishing sources and preamps (which both use the same part of the comparator).

Unlike with sources/preamps, the author was constistenly not able to distinguish diffetences between amps - which use a different path through the comparator.

This all strongly suggests that for comparing line-level signal sources the comparator itself produces some audible artefacts between the A and B paths, and that these are sufficient to allow clearly distinguishing the two.

My suggestion to the author of the audiophilestyle article would be to get somebody to assist him with his testing, and to include a test at random where the signal from one DAC is split so that A and B get an identical input. If he doesn't get random results with that it would prove that the ABX comparator is the issue.

 

[HalSF]

“The ABX Comparator’s controls had a paradoxical effect: it let me home in on the differences between DACs and, at the same time, underscored how small those differences really are.”

Smallness of differences seems just as significant in this test as the differences Perhaps even far more significant. I’m always skeptical of the overly categorical “all amps sound identical” and “DACs are now equally transparent regardless” proclamations, At the same time this test should give no comfort to the legions of audiophiles registering the night and day, vast, profound, non-trivial sound-quality contrasting-performance claims about different DACs. “Different but almost identical” seems like the headline takeaway here.