GoldenSounds passes apparently ABX test for DACs (NOT Really)

[ctrl]

Well, it shows that reconstruction filters matter -- with regard to the frequency response and the images they allow to get through, and sometimes less obvious effects like distinct pre-echos (not to be confuses with pre-ringing).

That's my take... the results violate the local rumored belief system.

If the reconstruction filters used have a noticeable effect, can you describe how to proceed in this case (download of test files is here) to identify a difference.
I did an ABX test in foobar several times and the only surprising thing was that I got exactly 50% correct three times, so for me the test files are 100% indistinguishable.

As has already been said, the analysis in DeltaWave shows that there are only deviating amplitude signals and/or noise at all around 22kHz and phase deviation is negligible.



It's strange that the video never describes how you can distinguish between the different reconstruction filters in terms of sound. It's also strange that nobody else seems to be able to distinguish between the music samples.

UPDATE:
There is a second video from Goldensound that shows in detail how he proceeds with the ABX test and what he looks out for. I missed this information in the first video and I assume that his ABX test is genuine - no need to cheat like I do


That's why I repeated the ABX test again, but slapped myself in the face a few times beforehand and shouted "naive idiot, you believe everything...".
Suddenly it was no longer a problem at all to distinguish the samples. Very easy, here is the proof:

So I'm 57 years old and can easily distinguish between the sample files or any DAC (by the way). All one have to do is slap you in face and repeat the magic words...

You can check my results here (test result, see attachment):

foobar2000 ABX Log Signature Tool

Update: How I was able to get 10/10 in the ABX-Test is revealed here in post#251

 

[VientoB]

Find a small group of people who can hear up to 21kHz and give them the same test.

 

[oleg87]

bye

I'll take that as a "no" then.

His stated methodology is fine. But non-adversarial ABX tests by people making a living off clickbait and their golden ears are always going to be less-than-persuasive to outside observers, for very obvious reasons. If you diff the files, the differences are *extremely* low-level aside from being at the very bounds of human-audible frequencies. Which is why I'm asking if anyone here can *repeat* this feat - repeatability, as you hopefully know, is a thing in science.

 

[ads_cft222]

The differences in the graphs start from 10khz

 

[ctrl]

Find a small group of people who can hear up to 21kHz and give them the same test.

Even in the (not uncontroversial) sources on "listening above 20kHz" linked under the YouTube video (for examples this source), the tests were carried out with signals up to 100dB and the threshold for 24kHz listening was 88dB for 4 listener out of 15 - if I understood it correctly.

The first relevant measurable deviations of the two sample music files from the YouTube video are around 22kHz. These are about -55dB below the music signal at 1kHz:

So it should not be very easy (and this is meant very euphemistically) to perceive an interference signal around 22kHz if the actual music signal has a sound pressure level over 50dB higher.

 

[KSTR]

Still, it only proves that he can, not that people in general can.

If he can, it is proof (with the statistical limits of the given test) that it can be audible, nothing less and nothing more (unless it were staged). Whether it is significant to the average Joe Listener on the streets is a completely different question.

You know, our dear host Amir is well trained in identifying artifacts in lossy codecs and one of the very few than can hear it even at higher quality settings. To 99.9999% of the population it is as irrelevant as inaudible, but that does not take away from the fact that it can be audible to some.

We would also need for the original participant to replicate the results at least two more times. While unlikely that there was guessing involved, unless I am not understanding the testing protocol, we can't rule that out with just one example.

Good point. Normally, IME, once you have found what to listen for to pass an ABX with 18/20 or better, you can do than many times as well, with the occasional slip from lack of concentration. Training plays a big role and once you are confident with the procedure, I see no reason why one could not replicate the test with a video camera capturing the whole event, or even an in-person supervisor which of course would be the preferred situation.

 

[ctrl]

The differences in the graphs start from 10khz

Sorry, but that's not correct. The two music samples in the YouTube video only show differences in amplitude in the 20-22kHz range and virtually no phase differences.

 

[voodooless]

Sorry, but that's not correct. The two music samples in the YouTube video only show differences in amplitude in the 20-22kHz range and virtually no phase differences.

View attachment 367362

Check out the difference spectrogram. They indeed show differences lower down. Very minor though.

 

[spacevector]

Amir disclosed in his blind testing video (will add a link when I find it) that for the 'key chimes' example, he was able to pass the ABX test by focusing on the tail-end of the file and turning the volume way up.

I feel something similar may have been done here - focus on a 1-2 second section of the files that is predominantly high treble, use headphones with strong output in that region and crank the volume knob.

Sure we loose hearing in high frequencies as we age but isn't it more of a reduction in sensitivity and not a total loss of hearing that band? So if you cannot hear 20kHz played at normal levels, you still may be able to hear it at very high levels.

 

[Vitt]

I followed the instructions from this reddit post and made a .wav file of the differences, but boosted it by 47dB. The lower frequencies are quietly audible and visible on the spectrogram when amplified this much. The poster identifies a 0.0001dB difference between the two files, so some of this difference may be due to the volume difference.

Sorry, but that's not correct. The two music samples in the YouTube video only show differences in amplitude in the 20-22kHz range and virtually no phase differences.

 

[ctrl]

That's why I repeated the ABX test again, but slapped myself in the face a few times beforehand and shouted "naive idiot, you believe everything...".
Suddenly it was no longer a problem at all to distinguish the samples. Very easy, here is the proof:

Here is the explanation of how the music samples from the YouTube video can be distinguished with 10/10 in an ABX test - of course I cheated:



On the left you can see the test setup. Simply hold a mic to the headphones. And on the right you can see the measurable difference in the live spectrogram between sample A and B during the ABX test in foobar2000 - around 22kHz one sample shows a small step (see yellow curve), the other sample does not (green curve).

I am not suggesting that the YouTuber is using a similarly simple trick to deceive his viewers, but everything should be done to prevent such doubts from arising in the first place.
So, for example, show the test procedure in the video (perform the ABX-Test before the camera) and give a description of how others can also replicate the result, i.e. what to look out for in the music sample and which is the best place in the music sample to differentiate.

If something is too good to be true, then it usually isn't.

UPDATE:
There is a second video from Goldensound that shows in detail how he proceeds with the ABX test and what he looks out for. I missed this information in the first video and I assume that his ABX test is genuine - no need to cheat like I do

 

[SIY]

Here is the explanation of how the music samples from the YouTube video can be distinguished with 10/10 in an ABX test - of course I cheated:

View attachment 367367 View attachment 367369

On the left you can see the test setup. Simply hold a mic to the headphones. And on the right you can see the measurable difference in the live spectrogram between sample A and B during the ABX test in foobar2000 - around 22kHz one sample shows a small step (see yellow curve), the other sample does not (green curve).

I am not suggesting that the YouTuber is using a similarly simple trick to deceive his viewers, but everything should be done to prevent such doubts from arising in the first place.
So, for example, show the test procedure in the video and give a description of how others can also replicate the result, i.e. what to look out for in the music sample and which is the best place in the music sample to differentiate.

If something is too good to be true, then it usually isn't.

Exactly what I was thinking.

 

[ctrl]

Check out the difference spectrogram. They indeed show differences lower down. Very minor though.

I followed the instructions from this reddit post and made a .wav file of the differences, but boosted it by 47dB. The lower frequencies are quietly audible and visible on the spectrogram when amplified this much. The poster identifies a 0.0001dB difference between the two files, so some of this difference may be due to the volume difference.

To realistically hear a difference, the two samples must differ by at least about 0.5dB over 1-2 octaves, i.e. at least from 10kHz to 20kHz 0.5dB difference.
Recognizing this correctly 18 times in an ABX test with 20 attempts would be damn difficult to impossible.

But we are miles away from such a thing when the two music samples are analyzed, as DeltaWave clearly shows:


Of course, I can zoom into the delta spectrum to an extreme extent and then find sound pressure level differences <0.001dB between the two music samples in the 10-20kHz range, but these differences have no practical value, no one in the world can perceive such pressure differences.

 

[ads_cft222]

To realistically hear a difference, the two samples must differ by at least about 0.5dB over 1-2 octaves, i.e. at least from 10kHz to 20kHz 0.5dB difference.
Recognizing this correctly 18 times in an ABX test with 20 attempts would be damn difficult to impossible.

But we are miles away from such a thing when the two music samples are analyzed, as DeltaWave clearly shows:
View attachment 367387

Of course, I can zoom into the delta spectrum to an extreme extent and then find sound pressure level differences <0.001dB between the two music samples in the 10-20kHz range, but these differences have no practical value, no one in the world can perceive such pressure differences.

View attachment 367388

Goldensound is still a human. Indeed differences in magnitude seem miniscule, however as it seems humans are still sensitive to them

 

[VientoB]

It’s a shame he didn’t perform the test in his video like he made out he was going to at the start. And I’d have liked him to state what differences he was hearing.

 

[tmtomh]

If the test is a true and legitimate result, then it's evidence that some people can detect differences between certain digital reconstruction filters if they turn the volume way up in the highest audible octave - yes?

I see no reason why that should be a finding (if true) that in any way threatens or upends any of the core values or approaches to audio measurement and listening that we embrace here at ASR.

It certainly makes GoldenSounds' "Proof that DACs can make a difference" video title deeply misleading and problematic, because he's using the word "can" try to hide the fact that he's talking about reconstruction filters regardless of whether you're talking about two different DACs or the same DAC with multiple filter options.

 

[voodooless]

It certainly makes GoldenSounds' "Proof that DACs can make a difference' video title deeply misleading and problematic, because he's using the word "can" try to hide the fact that he's talking about reconstruction filters regardless of whether you're talking about two different DACs or the same DAC with multiple filter options.

Really if all is true, the main thing that is proven is that the human plays a larger role than the DAC.

 

[CedarX]

GoldenSounds ran his ABX tests with a Holo Audio May DAC—his preferred DAC. I noticed he was quite critical of the Denafrips R2R DACs he reviewed, particularly the “misleading” NOS mode.
Not surprisingly (in a good way!), he appears quite knowledgeable about R2R DAC technology.

Is the May DAC a potential factor here? Can some “glitch” exacerbate or transform the high-freq small differences into something much more audible?

EDIT: And is the 176.4 kHZ sampling a compounding factor for a R2R DAC?

 

[VientoB]

GoldenSounds ran his ABX tests with a Holo Audio May DAC—his preferred DAC. I noticed he was quite critical of the Denafrips R2R DACs he reviewed, particularly the “misleading” NOS mode.
Not surprisingly (in a good way!), he appears quite knowledgeable about R2R DAC technology.

Is the May DAC a potential factor here? Can some “glitch” exacerbate or transform the high-freq small differences into something much more audible?

EDIT: And is the 176.4 kHZ sampling a compounding factor for a R2R DAC?

I think only NOS DACs can completely bypass the onboard filter and get the full benefit of using HQ Player? Someone correct me if I’m wrong.

 

[voodooless]

I think only NOS DACs can completely bypass the onboard filter and get the full benefit of using HQ Player? Someone correct me if I’m wrong.

Your wrong. NOS does not mean no filter, it just means the filter is sample and hold. At high sample rates, the type of filter becomes way less important.