• Welcome to ASR. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Audio by Van Alstine ABX Comparator Review

Some DACs are likely to actually sound different, level matched, in a proper ABX, especially when using 44.1kHz sources.
This is evident from the many measurements done by Amir where a small roll-off (filter dependent) is present in certain selected filters.
There is no mystery there.
It is rather evident from the test #2 posted in this thread. Rather than from looking at measurements. It is also evident from the filter files created by @j_j in the same thread. It is the ripple in the audio band that has been responsible for the sound difference, in the 96/24 records. FIR Filter transfer functions are much more complicated to make a simple statement. Though 1-pole analog LPF is simple to understand (this is what I used in the white noise test in this thread), FIR digital filters have many variables, like same amplitude response with different phase responses. And you cannot see it in the standard set of measurements used in the ASR reviews.
 
Yep...

This means that there can be (and are) audible, or potentially audible to some, differences between DACs. A question could be how much one would care in practice.
The box also has (at least potentially) audible differences when different attenuation levels for A and B are used.

In the output of the amplifier behind the ABX box there thus will be a mix of (potentially) audible differences and those seem to result in 'reliable detectable' differences.
However, it is unclear which 'audible differences' are caused by the box and/or the DAC.

The conclusion that DACs can be distinguished in a properly conducted ABX test thus is not proven in Josh's article.
Simply because the box can (and does) impart audible changes and this is the key part.

But... might/could still be using a different (and verified) test method.
 
Last edited:
On this bit Josh is saying there:

"Finally, I reluctantly participated in PMA’s DAC->ADC “loopback” ABX tests on ASR. I scored an 8/10 (p = 0.055) on the test that compared the original file versus the DAC into the ADC, as well as a 9/10 (p = 0.011) on the comparison of two DACs."

First, I don't know why he would be reluctant if he is good at these things. Second, these would be good results for the average person. But not when same person claims 100% success across so many of the hardware tests:

ABXReultswithLogo-HR.png.696d6f243841ab4da6ca9c01c4eece3d.png


Where are the 8 out of 10 results, p = 0.05 there? It doesn't exist. The worst case p value for 15 out of 16 is a whopping 0.0005 or two orders of magnitude smaller! He should have been able get 11 out of 10 in the software ABX test to match those. :D Even 9 out of 10 with p value of 0.02, a far cry from what he got in the hardware AVA case.

Based on my subjective testing and finding, the only way to get results like this across 16 trials is with clear volume differences. Something that we could have figured out with just a pair of comparison files. In absence of that, we have clearly shown the potential for that and only have his word that it must not have happened to him. Yet he did not observe this problem despite doing the test two ways that would show it.

I should also note that detection in ABX is not the same as fidelity differences. Two DACs with different volumes can easily be detected in ABX testing. Doesn't mean they have different fidelity. Once a person finds differences, then a preference test can be conducted. As it stands now, nothing has been demonstrated as one DAC sounding better than another.

If Josh/Chris, etc. want to prove that fidelity is different, go ahead and conduct a blind test again and this time attach preference scores. You don't need or want the AVA box as it screws the heck out of the fidelity of the DAC. Use a simple AB switcher. Match levels in software/player/content and do the testing.
 
the only way to get results like this across 16 trials is with clear volume differences.
Yes. Because IF there is a really tiny tiny difference, then we, after getting something like 12/16 or 13/16, still feel insecure if it was real or partly by chance. 16/16 is only in case that the difference is immediately audible and could be 100/100 as well.
 
For me the bottom line of all this is, NEVER trust influencers. Yet another time I've seen a familiar pattern: The larger the reach and the higher the self-esteem of the reviewer, the sparser, usually, is his knowledge on the subject and the less solid are the results, obviously. In this case, there were at least a few details where the reviewer tried to check back with some real experts but it was no enough for such a delicate subject.
 
For me the bottom line of all this is, NEVER trust influencers. Yet another time I've seen a familiar pattern: The larger the reach and the higher the self-esteem of the reviewer, the sparser, usually, is his knowledge on the subject and the less solid are the results, obviously. In this case, there were at least a few details where the reviewer tried to check back with some real experts but it was no enough for such a delicate subject.
Time to bring back this Dunning-Kruger effect graph
1000007035.png
 
It’s one microphone, Michael. How much could it cost? $2,000?
Tobias: You know, Lindsay, as a reviewer, I have advised a number of people to explore audability differences between DACs where the listener follows scientific methodology, but is free to have their results verified by knowledgable individuals.

Lindsay: Well, did it work for those people?

Tobias: No, it never does. I mean, these people somehow delude themselves into thinking it might, but … But it might work for us.
 
Correct. Because it has a low and trim dependent output impedance, whatever you set it to with a high impedance meter, will be wrong. Actual connected device will pull the input voltage down, resulting in mismatched levels in actual usage.

You have to place a "buffer," i.e. another amp on its output and match levels there. Even then, the impact of the trim is frequency dependent which means you can never match levels across full audible band.

It should be possible to level match with the DUT connected using a splitter cable or breakout box. I'd be prepared to bet quite a lot though that wasn't done.
 
It should be possible to level match with the DUT connected using a splitter cable or breakout box. I'd be prepared to bet quite a lot though that wasn't done.
Still non flat FR. Non flat FR = impossible to match level properly.
 
Some DACs are likely to actually sound different, level matched, in a proper ABX, especially when using 44.1kHz sources.
This is evident from the many measurements done by Amir where a small roll-off (filter dependent) is present in certain selected filters.
There is no mystery there.
It should not be the case though, even with 44.1 if a (sampling theorem) compliant - sharp/fast - usually default - filter is used.
 
Still non flat FR. Non flat FR = impossible to match level properly.
Agreed - but the actual level change resulting from the change from high impedance DVM to DUT impedance can be eliminated.
 
It should not be the case though, even with 44.1 if a (sampling theorem) compliant - sharp/fast - usually default - filter is used.
Not a filter (per se) effect here, difference is because of the impedance interaction.

And that's the one weakness of the level matching demand, it can't always be the case.
And the irony is that is exactly on these cases an ABX is needed (flat FR vs non-flat FR gear for example, as some load-depended amps do up high)
 
It should not be the case though, even with 44.1 if a (sampling theorem) compliant - sharp/fast - usually default - filter is used.
The point is one can often select other filters in which case that DAC (device) can sound 'distinguishable' even from the same model with default filter.
So DAC's as a device can sound different from other DACs even when they are able to not have a 'sound signature'.
 
impedance interaction.
Impedance interaction at the output of a typical low-impedance DAC into a typical (10K+) input?

I'm probably showing my ignorance here, but I'd also not understand why that would be sampling frequency dependent.
 
The point is one can often select other filters in which case that DAC (device) can sound 'distinguishable' even from the same model with default filter.
So DAC's as a device can sound different from other DACs even when they are able to not have a 'sound signature'.
Oh of course. Agreed.

But selecting the correct filter should be part of the controls for a valid ABX. Or at least using the default filter for each device if that is what you are testing for.
 
Impedance interaction at the output of a typical low-impedance DAC into a typical (10K+) input?

I'm probably showing my ignorance here, but I'd also not understand why that would be sampling frequency dependent.
Not just DAC to amp, just add a typical passive variable attenuator between them (replicating thread's comparator) and you have the needed interaction (at its -6dB point would be the worst case of it)
 
  • Like
Reactions: pma
Not just DAC to amp, just add a typical passive variable attenuator between them (replicating thread's comparator) and you have the needed interaction (at its -6dB point would be the worst case of it)

Yes. Such units, if meant seriously (not a mere relay switch like my old Simplest AB(x) box), must have:

  • input opamp buffer to define constant input impedance
  • something to control volume, even if passive
  • output opamp buffer with high input impedance, very low input capacitance, and low constant output impedance
  • switching relays or MOSFET switches

These are necessary, however still not satisfactory conditions of a good AB comparator.
 
selecting the correct filter should be part of the controls for a valid ABX.
Absolutely ... when possible.

Or at least using the default filter for each device
Some DAC's have really weird choices for default filters (usually because the manufacturer wants it to sound different or has a theory).
In that case DACs could be told apart reliably by some folks with certain recordings and training solely by the manufacturers choice of default filter. Regardless if a 'proper' filter can be selected or not.

I believe Josh did that as good as he could and even went as far as using 96kHz files to ensure the reconstruction filters were not going to skew results.

That wasn't a problem in his test though, the alterations the ABX box made most likely were though.
We can't say whether or not his results came from the DAC or the ABX box or both.

When DACs would be indistinguishable, other than a little difference in output voltage, but are changed in such a way that is becomes distinguishable (due to the ABX box) and this is not recognized as a possible reason for the found distinct sound difference the test is not valid.
It only proves he could reliably tell the difference but this might have been the ABX box only or the box + actual difference.
 
Back
Top Bottom