DAC audible differences are finally proven with ABX?

[solderdude]

Yes, that's the topic. And one gaping hole I see in the denier side's arguments on this type of DAC controversy is they do not actually know what the limits are for the human ability to detect deviations from a perfectly flat FR. Yet they behave as if this was as clearly established as our inability to see UVC or gamma-rays, or hear 40k, and they proceed to hammer strangers over the head with this illusion of established knowledge.

What about current knowledge makes you doubt that 20Hz to 21kHz -0.5dB is not good enough ?
And in the case of this comparison for the signal DC to 40kHz (-0.1dB).

For the comparator FR extension could only matter when there was a substantial difference in volume control setting (say 70 vs 99) as the cable capacitance and input capacitance of the cable would start to matter ( 60ohm vs 5k + R out of the source).
Some math assuming a 1m fancy cable + PCB + input of amp is 500pF and Rout of a DAC is 100ohm:
Best scenario (setting 99) = 1.6MHz -3dB but specs of the 20k version says 310kHz, worst case (setting around 70) = 60kHz -3dB (around 20kHz -0.5dB) which was never set acc. to Josh.

Somehow I don't think FR flatness of the comparator could ever be an issue here unless the 50k or 200k version was used which is very unlikely from a design standpoint.

Still wondering if each AD5262 is used for for each input or that the volume control is only in the output path (so separate L + R chips)
Too bad there is no documentation available.

 

[ElJaimito]

Would you trust a social scientist (an oxymoron right there

A social scientist would have a science-based degree, i.e. BSc/BS, going to MSc/MS and eventually PhD. I have that in psychology, studied 25 years after me initial studies, and my work included bio-psychology, brain structures, psycho-neurology and neural network analogs, and a great deal of analytical and predictive statistics. It was quite compatible with (and I used a lot of) my electronics background.

However many people study social studies to BA and MA qualifications: these are arts degrees with rather limited technical content and a very different grading structure. Not knocking, but this style is not a "science" field of social study. My wife is an MA, ARCA, and she is a major expert in many topics including lace over the centuries. She had clear ideas of what she liked in sound and had a decent hifi when I met her (she's now deaf). But she has no technical analytical skills nor interest.

So, before arbitrarily throwing out babies, bathwater, a bit more background is called for. And not just cf. with "military intelligence" jokes

Not that that convinces me of the results of the study mentioned, mind you, though I'm trying to ignore my own knee-jerk responses to the word "audiophile"...

 

[Mnyb]

What about current knowledge makes you doubt that 20Hz to 21kHz -0.5dB is not good enough ?
And in the case of this comparison for the signal DC to 40kHz (-0.1dB).

For the comparator FR extension could only matter when there was a substantial difference in volume control setting (say 70 vs 99) as the cable capacitance and input capacitance of the cable would start to matter ( 60ohm vs 5k + R out of the source).
Some math assuming a 1m fancy cable + PCB + input of amp is 500pF and Rout of a DAC is 100ohm:
Best scenario (setting 99) = 1.6MHz -3dB but specs of the 20k version says 310kHz, worst case (setting around 70) = 60kHz -3dB (around 20kHz -0.5dB) which was never set acc. to Josh.

Somehow I don't think FR flatness of the comparator could ever be an issue here unless the 50k or 200k version was used which is very unlikely from a design standpoint.

Still wondering if each AD5262 is used for for each input or that the volume control is only in the output path (so separate L + R chips)
Too bad there is no documentation available.

+1 but for audiability small 0.1 dB deviations are clearly audible if they are over very wide range , say a filter setting that makes a gentle slope in fr from 10kHz

 

[ElJaimito]

A social scientist would have a science-based degree, i.e. BSc/BS, going to MSc/MS and eventually PhD. I have that in psychology, studied 25 years after me initial studies, and my work included bio-psychology, brain structures, psycho-neurology and neural network analogs, and a great deal of analytical and predictive statistics. It was quite compatible with (and I used a lot of) my electronics background.

However many people study social studies to BA and MA qualifications: these are arts degrees with rather limited technical content and a very different grading structure. Not knocking, but this style is not a "science" field of social study. My wife is an MA, ARCA, and she is a major expert in many topics including lace over the centuries. She had clear ideas of what she liked in sound and had a decent hifi when I met her (she's now deaf). But she has no technical analytical skills nor interest.

So, before arbitrarily throwing out babies, bathwater, a bit more background is called for. And not just cf. with "military intelligence" jokes

Not that that convinces me of the results of the study mentioned, mind you, though I'm trying to ignore my own knee-jerk responses to the word "audiophile"...

As a follow-up, I've read most of the article (should have done that before commenting I know) and the guy mentions statistics, so he may have a science variant of the degree.

But there's an immediate procedural issue: he sets levels using a digital multimeter.

I would never use a multimeter, digital or analog, for measuring audio levels. Reason being, most have a very limited frequency response and underspecifed rectification (multimeter True RMS circuitry is also frequency limited).

I suppose variance due to this is minimised by using the same meter in all cases and a 1kHz tone/shaped noise but I wouldn't trust a normal digital multimeter (make and model is not mentioned) for such duties @ 1kHz. I would only use a proper audio "VTVM" which now is solid-state of course. My poorest one has -0.5 dB at 150kHz, the best is good for that at 10 MHz. Whereas a typical DVM is lucky to reach 1 kHz: they are intended for low frequency AC mains power measurements, even the RMS models. Of course there are (expensive, name) DVMs that are entirely capable of these measurements, but not a digital multimeter (unspecified). Moreover the response to pink noise - inherently wide-band, though with a falling characteristic - is entirely unspecified with a digital multimeter, even if it is True RMS, because of (in almost all cases) a very limited frequency response.

If you have a digital multimeter check your spec sheet: some of mine spec to 400Hz, others to 1kHz. None of them are specified for pink noise measurements.

Here's a bit on this:

https://audiokarma.org/forums/threads/what-does-a-vtvm-do-that-a-dvm-cant.570178/

So, there is zero guarantee that the levels are in fact matched right at the beginning of the experiment. In particular, any anti-aliasing filter artifacts will not be seen by such a meter (there should not be any revealed by the pink noise, but still...)

AFAIAC that invalidates the whole of the rest of the results.

Of course, the fix for this is simple. Informing oneself over the lack of recognising this sort of issue, harder.

NB: measuring stuff well is hard.

 

[pma]

I would never use a multimeter, digital or analog, for measuring audio levels. Reason being, most have a very limited frequency response.

This is impossible to tell as a general statement. I have a DMM which is precise up to 100kHz, measures much higher. $200. And it is calibrated. I would rather say that laymen audience should not try to make sophisticated testing. Caveats and unexpected sources of errors.

 

[ElJaimito]

This is impossible to tell as a general statement. I have a DMM which is precise up to 100kHz, measures much higher. $200. And it is calibrated.

Oh really? Make and model? I'm interested.

 

[RandomEar]

As a follow-up, I've read most of the article (should have done that before commenting I know) and the guy mentions statistics, so he may have a science variant of the degree.

But there's an immediate procedural issue: he sets levels using a digital multimeter.

I would never use a multimeter, digital or analog, for measuring audio levels. Reason being, most have a very limited frequency response.

I suppose variance due to this is minimised by using the same meter in all cases and a 1kHz tone/shaped noise but I wouldn't trust a normal digital multimeter (make and model is not mentioned) for such duties @ 1kHz. I would only use a proper audio "VTVM" which now is solid-state of course. My poorest one has -0.5 dB and 150kHz, the best is good for that at 10 MHz. Whereas a typical DVM is lucky to reach 1 kHz: they are intended for low frequency AC mains power measurements, even the RMS models. Of course there are (expensive, name) DVMs that are entirely capable of these measurements, but not a digital multimeter (unspecified). Moreover the response to pink noise - inherently wide-band, though with a falling characteristic - is entirely unspecified with a digital multimeter, even if it is True RMS, because of (in almost all cases) a very limited frequency response.

If you have a digital multimeter check your spec sheet: some of mine spec to 400Hz, others to 1kHz. None of them are specified for pink noise measurements.

Here's a bit on this:

https://audiokarma.org/forums/threads/what-does-a-vtvm-do-that-a-dvm-cant.570178/

So, there is zero guarantee that the levels are in fact matched right at the beginning of the experiment. In particular, any anti-aliasing filter artifacts will not be seen by such a meter (there should not be any revealed by the pink noise, but still...)

AFAIAC that invalidates the whole of the rest of the results.

Of course, the fix for this is simple.

NB: measuring stuff well is hard.

Using a multimeter and a test tone within the specified bandwidth is absolutely the right approach. Also, Josh used multiple methods according to what he wrote, including pink noise evaluated by REW. It is, however, not clear to me which method he used to determine the final judgement of "good enough" matching.

 

[ElJaimito]

laymen audience should not try to make sophisticated testing. Caveats and unexpected sources of errors.

Yeah. They do. But we all start out as laymen, no? It's a reason for caution in presenting results, and requesting feedback for improvement...

 

[ElJaimito]

Using a multimeter and a test tone within the specified bandwidth is absolutely the right approach. Also, Josh used multiple methods according to what he wrote, including pink noise evaluated by REW. It is, however, not clear to me which method he used to determine the final judgement of "good enough" matching.

Without details of the equipment it is impossible to know if the signal was within the specified bandwidth. And wideband pink noise definitely is not.

 

[RandomEar]

+1 but for audiability small 0.1 dB deviations are clearly audible if they are over very wide range , say a filter setting that makes a gentle slope in fr from 10kHz

Sorry, but do you have any source for that? As far as I am aware, humans can generally not distinguish such small differences.

 

[KSTR]

If you have a digital multimeter check your spec sheet: some of mine spec to 400Hz, others to 1kHz. None of them are specified for pink noise measurements.

That's not the problem, the problem is using pink noise to begin with. The only way to get a precision reading (like 0.01dB or better) is using periodic pink noise (not random pink noise) and a RMS level meter with a integration time constant at least ten times larger than the noise period.
For precision level measurement single tone should be used, preferable a set of which, at different frequencies. If the DUTs have relevant level differences vs frequency against each other, then wide-band level matching is impossible by definition.

 

[KSTR]

Sorry, but do you have any source for that? As far as I am aware, humans can generally not distinguish such small differences.

0.3dB (3%) wideband level difference is an established threshold that many people can instantly detect (though not neccesarily as perceived level difference). Lower for well-trained listeners .

 

[RandomEar]

Without details of the equipment it is impossible to know if the signal was within the specified bandwidth. And wideband pink noise definitely is not.

He used a test tone at 1 kHz with a multimeter he bought brand new. Bandwidth is not the problem here, since he used the same signal to test both outputs.

He also used pink noise with the DVM, but then checked everything with pink noise via the ADC and REW. This is all perfectly fine.

The main problem here isn't the DVM. The main problem, if anything, is that the results of the level matching were not documented.

 

[ElJaimito]

the problem is using pink noise to begin with. The only way to get a precision reading (like 0.01dB or better) is using periodic pink noise (not random pink noise) and a RMS level meter with a integration time constant at least ten times larger than the noise period.

Agreed. The parameters of the experiment are ill-advised. I'd still like to know which meter you mentioned? The more expensive Flukes are good up to 20 kHz (within a dB or less) but most meters I have used are useless at that frequency. However, the Flukes aren't specified for RMS pink noise measurements in any case (or I can't find such), hence my interest in your device?

 

[ElJaimito]

0.3dB (3%) wideband level difference is an established threshold that many people can instantly detect

It'd be nice to have a reference? Don't disagree, but?

 

[RandomEar]

0.3dB (3%) wideband level difference is an established threshold that many people can instantly detect (though not neccesarily as perceived level difference). Lower for well-trained listeners .

OK, but that's still quite a bit away from 0.1 dB. I can barely pass a test with 0.2 dB with test tones, but 0.1 is simply inaudible to me. So I would still like a link to a publication which tested this for broad band signals.

 

[ElJaimito]

Using a multimeter and a test tone within the specified bandwidth is absolutely the right approach

Sorry, can't agree there!

 

[RandomEar]

Sorry, can't agree there!

Why not? What potential error do you expect?

 

[welwynnick]

As a follow-up, I've read most of the article (should have done that before commenting I know) and the guy mentions statistics, so he may have a science variant of the degree.
But there's an immediate procedural issue: he sets levels using a digital multimeter.
I would never use a multimeter, digital or analog, for measuring audio levels. Reason being, most have a very limited frequency response and underspecifed rectification (multimeter True RMS circuitry is also frequency limited).

Josh is way ahead of you.
1. He consulted with Amir and others to agree best methods for level setting LAST YEAR:

DAC audible differences are finally proven with ABX?

Sampling theorem, sampled signal must be limited to F Fs/2 and as such it cannot be reconstructed properly. Anti-aliasing input filter. Or oversampling etc., There is enough literature.

www.audiosciencereview.com

2. He measured a 1 kHz sine wave and broadband CTA-2034 pink noise with a DVM.
3. He measured the levels at the headphone amp output using a Cosmos ADC and REW.
4. He repeated the final test after balancing the channels to check they were matched.
5. He just published frequency response measurements showing the comparator outputs were very closely matched between channel A and channel B, between left and right, and at different comparator volume settings. At each volume setting, all the traces are on top of each-other.

https://audiophilestyle.com/forums/topic/72134-article-updated-can-i-tell-audibly-transparent-dacs-apart-blind-testing-with-the-van-alstine-abx-comparator/#findComment-1347665

 

[KSTR]

I can barely pass a test with 0.2 dB with test tones, but 0.1 is simply inaudible to me.

You don't use test tones for such tests, you use real music signal, preferably one with good stereo reverb tails. 0.1B (1%) level difference manifests itself as the louder one having more "resolution" and "depth" because the tails take longer until their components fall below thresholds (which are highly individual).
Further, one random single person's level difference detection threshold suggests that there are likely other people with lower thresholds. Not like 20dB lower of course, but several dB is certainly possible. And listening level is a major variable, of course.