DAC listening test files - updated with new filters files

[NTTY]

It would be better if everone made 16 or 20 trials. I made 100 random tests with 10 trials and got 9/10 two times and 0/10 one time. So better use the standard 16 trials or 20 trials.

Like the group delay test here

Audibility of group delay at low frequencies

In the spectrogram tab You can change the appearance settings as well -- just change it to whatever method or resolution is needed to see things clearer. FDW doesn't affect the spectrogram at all, but it can clear up some of the reflection-induced rotations in seen in the phase traces. From...

www.audiosciencereview.com

This is why 16 is the somewhat golden number for that test.
That said, it should be easy to cheat A from B when using Foobar ABX, it just requires a monitoring tool of the output, as it’s already been discussed here. I did not try myself, but I might.

 

[huw]

Wow, new world record, 10sec total duration and 1sec between 2 trials. I’m not sure I can click that fast between the ABX buttons. Challenge accepted
For curiosity, how many times did you try to get that one positive result?

Yes I hoped the timing made it obvious that was just random clicking to generate the report. My point was that report also said "0126" instead of "2026" and "16-bit". As mentioned several times a single 9/10 result can be random chance if it was cherry-picked from multiple attempts.

 

[pma]

Wow!

 

[huw]

I'd never tried the foo_abx plugin before. One thing that surprised me is that the report does not indicate how much time was spent listening to A,B,X or Y or even (as in my case) if the samples were never played. Also, at least on macOS, the bit depth in the report reflects the setting in foobar2000 not that in the Audio MIDI Setup - I got a report with:

Output:
Core Audio : Default, 32-bit

when the output was set to 16-bit

 

[pjug]

@pma I hope you don't mind me sharing a modified version of your comp file from the 2_Dact-test comparison. I just ran the level EQ in Deltawave to get rid of the passband ripple difference along with any other amplitude difference in the spectrum. I hope that @Josh83 might do ABX iusing thi sinstead of the other comp file. If he fails then it would sem that he was hearing the passband ripple difference. If he passes then he must have been hearing something else. Or maybe we can find someone else who can hear the difference and have them help look into this. I will try again too, using the tip that Josh gave on how he was telling these apart.

dactest_comp with EQ.wav

drive.google.com

 

[pma]

No problem, a small note, the passband ripple was a part of the DAC behaviour.

 

[pjug]

No problem, a small note, the passband ripple was a part of the DAC behaviour.

Yes, exactly. So if Josh was able to tell differences from the ripple he is hearing differences between DACs. I am just hoping to get at what characteristics are allowing him to tell the difference.

 

[pma]

I am not sure if such filter manipulation is possible, because I do not think it is possible to mix 2 different tests (96kHz and 44.1kHz sampling). As I have repeatedly mentioned, I am on vacations with no acces to my test database. The dactest comp x ref files are about 3 years old and I do not have the filter setting in my memory. I can check it when I am back = next week.

 

[NTTY]

Yes I hoped the timing made it obvious that was just random clicking to generate the report.

It did

My point was that report also said "0126" instead of "2026" and "16-bit".

Point noted indeed.

As mentioned several times a single 9/10 result can be random chance if it was cherry-picked from multiple attempts.

We’ve been asking for the more standard 16 trials for quite some pages, which is one of the reasons why.

 

[pjug]

I am not sure if such filter manipulation is possible, because I do not think it is possible to mix 2 different tests (96kHz and 44.1kHz sampling). As I have repeatedly mentioned, I am on vscations with no acces to my test database. The dactest comp x ref files are about 3 years old and I do not have the filter setting in my memory.

We could do the same thing with the 96K files where the dac comparison shows the same kind of ripple (the 2nd pair of files in this thread). But I don't think Josh did ABX on that one (he passes the 96K comparison betwee a DAC and the original file). We need someone to pass the ABX test before we can start looking for what they were hearing.

 

[pma]

Right now I can only responsibly say that Topping D10s has easily measurable passband ripple, in both 44.1 and 96 kHz sampling, and of course at 48 kHz as well. I am not able to quantify it right now at the moment.

 

[Cbdb2]

@pma Do you happen to remember which filter was active on the DacMagic+ you used to capture the ABX test files? Because it looks like the minimum phase filter in that DAC can cause clipping:

Even if it wasnt clipping that ringing is huge (10khz/1v on a 3 v signal) and probably audible for some.

 

[solderdude]

I see 22kHz ringing (slightly less than 50us for 1 cycle) and when is there ever such a fast 0dBFS rise to fall in any music signal.

 

[RandomEar]

I see 22kHz ringing (slightly less than 50us for 1 cycle) and when is there ever such a fast 0dBFS rise to fall in any music signal.

If you count all 8 cycles at the top and assume that the time difference is about 730 - 310 = 420 µs, it would be 19.05 kHz. I'm not an expert in this, but I have seen before that minimum phase filters don't "ring" at Nyquist. Don't know why.

 

[solderdude]

Yep, no Nyquist indeed. I eyeballed it as 10kHz seemed a bit low.
Looking very closely using a ruler for only the last 7 rising edges I get 354µs = 50.57µs = 19.77kHz

 

[danadam]

We’ve been asking for the more standard 16 trials for quite some pages, which is one of the reasons why.

Seems to me you should state the p you are satisfied with rather than the number of trials. If 9/10, p = 1% is not good enough because it could happen by chance, then (unless I'm mistaken) 13/16, p = 1% will have the same problem.

 

[NTTY]

Seems to me you should state the p you are satisfied with rather than the number of trials. If 9/10, p = 1% is not good enough because it could happen by chance, then (unless I'm mistaken) 13/16, p = 1% will have the same problem.

I don’t get why you quote me because I literally said the opposite few pages ago. And I did not challenge a 9/10 as you make me say. But I agree with @pma that people should share their total number of trials and results, like I shared mine pages before and where I got a 13/16 by chance.

I’ve been calling for 16 trials, p-value < 0.05, especially with Foobar ABX calculation, from the beginning. And why are we not talking 16 trials and p < 0.05? I’ve been using Foobar ABX for years and it’s always been what I’ve used and asked people to use. I admit I don’t know where that number of trials come from, I thought it was the Foobar default. But I never went for higher or lower than 16.

On an additional personal note, if I don’t have a problem with only 10 trials provided we accept 9/10 or 10/10 as success (not 8/10), it appears that 10 trials in Foobar is too low because too easy to cheat, as we saw it. Reply randomly and it does not take long to have one positive. You also cheated the online test of @pma giving a hint on how to cheat whatever number of trials. And this is why I stated that we can’t trust online results performed that way anymore. I never thought people would cheat such tests that are performed for the interest of the community, but hey here we are, at 55, I’m still too naive.

So basically, "we" can put the number where we want, I recon I unfortunately can’t trust results collected that way anymore.

 

[pma]

Yes I hoped the timing made it obvious that was just random clicking to generate the report. My point was that report also said "0126" instead of "2026" and "16-bit". As mentioned several times a single 9/10 result can be random chance if it was cherry-picked from multiple attempts.

Is this (by random clicking achieved) 9/10 on macOS or iOS repeatable? Does it mean there is a bug in foo abx component for macOS?

 

[huw]

Is this (by random clicking achieved) 9/10 on macOS or iOS repeatable? Does it mean there is a bug in foo abx component for macOS?

No. I tried random guesses a few more times and got different scores as would be expected. The only bug is the incorrect date in the report.

 

[danadam]

I don’t get why you quote me

Sorry, I didn't mean to single you out. Yours was just the last message in the discussion. And AFAIU the discussion, it is about 9/10 happening by chance and 16-trials test helping with that:

@Thomas_A wrote
"It would be better if everone made 16 or 20 trials. I made 100 random tests with 10 trials and got 9/10 two times and 0/10 one time."
@huw wrote:
"As mentioned several times a single 9/10 result can be random chance if it was cherry-picked from multiple attempts."
you wrote:
"We’ve been asking for the more standard 16 trials for quite some pages, which is one of the reasons why."

I just don't see how 16-trials test helps with that, if p remains the same. It only helps when compared to 2 tests intended for summing, i.e. 2x 10-trials test vs 1x 16-trials test, because 2x10 requires trust that they happened one after the ohter.