DAC listening test files - updated with new filters files

[RandomEar]

Took a quick look at the Lagadec paper. I guess technically, it is an "echo". In the end, it just describes the deviation of the impulse response from an ideal (infinitely long) one, which any real filter will have. They relate it to passband ripple, but it could really be related to any filter deviation I would think. I don't see what would be special about the ripple - it is one error amongst many.

Coming back to my initial request: Listen to the delta for <7 s (for D10s vs DacMagic+). Go ahead. There is simply nothing there to "detect" in the first couple of seconds. It is below the audible threshold. This is not a question of training or being sensitive to this or that phenomenon: The absolute level of the differences in this case is below what one can hear even for very high listening levels of the original track.

It seems like a moot point to speculate on and on about possible causes if there is no actual deviation present in the delta track which you could relate them to.

 

[pjug]

Took a quick look at the Lagadec paper. I guess technically, it is an "echo". In the end, it just describes the deviation of the impulse response from an ideal (infinitely long) one, which any real filter will have. They relate it to passband ripple, but it could really be related to any filter deviation I would think. I don't see what would be special about the ripple - it is one error amongst many.

There is a big difference in the amound of passband ripple between different filters. The echoes are real. I don't get why everyone wants to say this isn't real. Audible -- who knows? But a real effect.

 

[pma]

Regarding audibility, passband ripple might be, in my opinion, a potential issue. I doubt about audibility of impulse/step response ringing. It needs very short impulse rise time to happen, something that is not happening in music, except transients like cymbal clash.

 

[Ageve]

These tests must be done in exclusive mode, bypassing the OS audio pipeline. Otherwise, both MacOS and Windows will resample the audio (to typically 48 kHz), and perform dithering, etc. These transforms could cause one sample to have artifacts and the other not. And at any rate, render any analysis of the original files moot as that is not what is being played.

macOS uses the bit depth and sample rate selected in Audio MIDI setup. If it's set up correctly, there shouldn't be any resampling done (unless it's done by the playback software, like a web browser or something).

The logs posted in this thread showing 16-bit output are correct btw. It's the default setting in foobar2000 on Mac.

Change it to 32-bit here:

For those who are wondering if macOS and foobar2000 are transparent (and useful for ABX testing):

I recorded the output (in Audacity, using BlackHole) during an ABX test between two identical files.

Details:

Spoiler

Macbook M1 Pro
macOS Tahoe 26.5.1
foobar2000 for Mac v2.24.1
Audacity 3.2.5 (recording, 32-bit float, 192 kHz)
BlackHole 2ch driver (192 kHz)
DeltaWave v2.0.25 (PC)

This is the "difference" between that recording and the original file:

Files are a bit-perfect match at 16 bits
Files are a bit-perfect match at 24 bits
Files are a bit-perfect match at 32 bits

Log:

Spoiler

DeltaWave v2.0.25, 2026-08-22T12:46:15.5193289+02:00
Reference: Test Eagles hotel california original.wav[?] 4000122 samples 192000Hz 32bits, stereo, MD5=00
Comparison: TEST ABX macos eagles 192khz 32 float.wav[?] 9215616 samples 192000Hz 32bits, stereo, MD5=00
Settings:
Gain:True, Remove DC:True
Non-linear Gain EQ:False Non-linear Phase EQ: False
EQ FFT Size:65536, EQ Frequency Cut: 0Hz - 0Hz, EQ Threshold: -500dB
Correct Non-linearity: False
Correct Drift:True, Precision:30, Subsample Align:True
Non-Linear drift Correction:False
Upsample:False, Window:Kaiser
Spectrum Window:Kaiser, Spectrum Size:32768
Spectrogram Window:Hann, Spectrogram Size:4096, Spectrogram Steps:2048
Filter Type:IIR, window:Kaiser, taps:262144, minimum phase=False
Dither:False bits=0
Trim Silence:True
Enable Simple Waveform Measurement: False
Correct for MP DC Filter = False, DC Filter frequency=0,5 Hz

Discarding Reference: Start=0s, End=0s
Discarding Comparison: Start=0s, End=0s

Initial peak values Reference: -1,159dB Comparison: -1,159dB
Initial RMS values Reference: -20,083dB Comparison: -20,697dB

Null Depth=42,118dB
X-Correlation offset: -323068 samples
Drift computation quality, #1: Excellent (0μs)


Trimmed 0 samples ( 0,00ms) front, 0 samples ( 0,00ms end)


Final peak values Reference: -2,106dB Comparison: -2,106dB
Final RMS values Reference: -20,209dB Comparison: -20,209dB

GS: -1682,645833 ms, 0 dB (L), 0 dB (R), -333,4 dBFS (L), -333,3 dBFS (R) [-333,6dBFSA (L) -333,6dBFSA (R)]

Gain= 0dB (1x) DC=0 Phase offset=-1682,645833ms (-323068 samples)
Difference (rms) = -333,45dB [-333,63dBA]
Correlated Null Depth=313,75dB [306,54dBA]
Clock drift: 0 ppm


Files are a bit-perfect match at 16 bits
Files are a bit-perfect match at 24 bits
Files are a bit-perfect match at 32 bits


---- Phase difference (full bandwidth): 0°
0-10kHz: 0,00°
0-20kHz: 0,00°
0-24kHz: 0,00°
Timing error (rms jitter): 0sec
PK Metric (step=400ms, overlap=50%):
RMS=-324,2dBFS
Median=-324,9
Max=-318,3

99%: -318,52
75%: -322,56
50%: -324,93
25%: -327,77
1%: -∞

gn=1, dc=0, dr=0, of=-323068

DONE!

Signature: a8eec0a95dded2c80042326dedaf1671

RMS of the difference of spectra: -389,648572048232dB
DF Metric (step=400ms, overlap=0%):
Median=-300dB
Max=-154,8dB Min=-300dB

1% > -300,0dB
10% > -300,0dB
25% > -300,0dB
50% > -300,0dB
75% > -157,04dB
90% > -155,95dB
99% > 0,0dB

Linearity 32,0bits @ 0.5dB error

This is the "difference" between A and B (from the same recording):

Spoiler

DeltaWave v2.0.25, 2026-08-22T12:51:01.5823483+02:00
Reference: TEST ABX macos eagles 192khz 32 float part 1.wav[?] 4270805 samples 192000Hz 32bits, stereo, MD5=00
Comparison: TEST ABX macos eagles 192khz 32 float part 2.wav[?] 4554265 samples 192000Hz 32bits, stereo, MD5=00
Settings:
Gain:True, Remove DC:True
Non-linear Gain EQ:False Non-linear Phase EQ: False
EQ FFT Size:65536, EQ Frequency Cut: 0Hz - 0Hz, EQ Threshold: -500dB
Correct Non-linearity: False
Correct Drift:True, Precision:30, Subsample Align:True
Non-Linear drift Correction:False
Upsample:False, Window:Kaiser
Spectrum Window:Kaiser, Spectrum Size:32768
Spectrogram Window:Hann, Spectrogram Size:4096, Spectrogram Steps:2048
Filter Type:IIR, window:Kaiser, taps:262144, minimum phase=False
Dither:False bits=0
Trim Silence:True
Enable Simple Waveform Measurement: False
Correct for MP DC Filter = False, DC Filter frequency=0,5 Hz

Discarding Reference: Start=0s, End=0s
Discarding Comparison: Start=0s, End=0s

Initial peak values Reference: -1,159dB Comparison: -1,159dB
Initial RMS values Reference: -20,367dB Comparison: -20,646dB

Null Depth=43,263dB
X-Correlation offset: -177348 samples
Drift computation quality, #1: Excellent (0μs)


Trimmed 0 samples ( 0,00ms) front, 0 samples ( 0,00ms end)


Final peak values Reference: -2,106dB Comparison: -2,106dB
Final RMS values Reference: -20,036dB Comparison: -20,036dB

GS: -923,6875 ms, 0 dB (L), 0 dB (R), -400,0 dBFS (L), -400,0 dBFS (R) [-400,0dBFSA (L) -400,0dBFSA (R)]

Gain= 0dB (1x) DC=0 Phase offset=-923,6875ms (-177348 samples)
Difference (rms) = -400dB [-400dBA]
Correlated Null Depth=300dB [300dBA]
Clock drift: 0 ppm


Files are a bit-perfect match at 16 bits
Files are a bit-perfect match at 24 bits
Files are a bit-perfect match at 32 bits


---- Phase difference (full bandwidth): 0°
0-10kHz: 0,00°
0-20kHz: 0,00°
0-24kHz: 0,00°
Timing error (rms jitter): 0sec
PK Metric (step=400ms, overlap=50%):
RMS=-∞dBFS
Median=-∞
Max=-∞

99%: -∞
75%: -∞
50%: -∞
25%: -∞
1%: -∞

gn=1, dc=0, dr=0, of=-177348

DONE!

Signature: 43df18655d25f5e38fc1c5e68f4d8a2f

RMS of the difference of spectra: NaNdB
DF Metric (step=400ms, overlap=0%):
Median=-300dB
Max=-156,5dB Min=-300dB

1% > -300,0dB
10% > -300,0dB
25% > -300,0dB
50% > -300,0dB
75% > -300,0dB
90% > -266,52dB
99% > 0,0dB

Linearity 32,0bits @ 0.5dB error

ABX log:

Spoiler

foo_abx 2.2.3 report
foobar2000 v2.24.1
0126-08-22 13:11:31

File A: Test 2 Eagles hotel california original (kopia).wav
SHA1: 1d0fcea1d2c9345c4e32754eb535229bff9f6721
File B: Test Eagles hotel california original.wav
SHA1: 1d0fcea1d2c9345c4e32754eb535229bff9f6721

Output:
Core Audio : Default, 32-bit
Crossfading: NO

13:11:31 : Test started.
13:11:38 : Test restarted.
13:11:38 : 00/01
13:11:41 : Test restarted.
13:11:41 : 01/02
13:11:43 : Test restarted.
13:11:43 : 02/03
13:11:45 : Test restarted.
13:11:45 : 03/04
13:11:47 : Test restarted.
13:11:47 : 04/05
13:11:49 : Test restarted.
13:11:49 : 05/06
13:11:51 : Test restarted.
13:11:51 : 05/07
13:11:53 : Test restarted.
13:11:53 : 05/08
13:11:55 : Test restarted.
13:11:55 : 05/09
13:11:57 : Test restarted.
13:11:57 : 06/10
13:12:01 : Test restarted.
13:12:01 : 07/11
13:12:02 : Test restarted.
13:12:02 : 07/12
13:12:05 : Test restarted.
13:12:05 : 07/13
13:12:09 : Test restarted.
13:12:09 : 08/14
13:12:11 : Test restarted.
13:12:11 : 08/15
13:12:14 : Test restarted.
13:12:14 : 08/16
13:12:14 : Test finished.

----------
Total: 8/16
p-value: 0.5982 (59.82%)

-- signature --
0cd2a0f1538172c4de7b933477c88b40f31e4f4b

I also tried playing the same file in Finder (play symbol on the icon).

This did not get a bit perfect result, but there's no resampling applied, and the difference is just -166dB. It's probably dither applied by Core Audio (and bypassed by foobar2000, I guess).

Spoiler

DeltaWave v2.0.25, 2026-08-22T12:56:25.6889688+02:00
Reference: Test Eagles hotel california original.wav[?] 4000122 samples 192000Hz 32bits, stereo, MD5=00
Comparison: Test eagles i findr 32 float.wav[?] 4615808 samples 192000Hz 32bits, stereo, MD5=00
Settings:
Gain:True, Remove DC:True
Non-linear Gain EQ:False Non-linear Phase EQ: False
EQ FFT Size:65536, EQ Frequency Cut: 0Hz - 0Hz, EQ Threshold: -500dB
Correct Non-linearity: False
Correct Drift:True, Precision:30, Subsample Align:True
Non-Linear drift Correction:False
Upsample:False, Window:Kaiser
Spectrum Window:Kaiser, Spectrum Size:32768
Spectrogram Window:Hann, Spectrogram Size:4096, Spectrogram Steps:2048
Filter Type:IIR, window:Kaiser, taps:262144, minimum phase=False
Dither:False bits=0
Trim Silence:True
Enable Simple Waveform Measurement: False
Correct for MP DC Filter = False, DC Filter frequency=0,5 Hz

Discarding Reference: Start=0s, End=0s
Discarding Comparison: Start=0s, End=0s

Initial peak values Reference: -1,159dB Comparison: -1,159dB
Initial RMS values Reference: -20,083dB Comparison: -20,704dB

Null Depth=43,623dB
X-Correlation offset: -334498 samples
Drift computation quality, #1: Excellent (0μs)


Trimmed 0 samples ( 0,00ms) front, 0 samples ( 0,00ms end)


Final peak values Reference: -1,159dB Comparison: -1,159dB
Final RMS values Reference: -20,039dB Comparison: -20,039dB

GS: -1742,177083 ms, 0 dB (L), 0 dB (R), -166,3 dBFS (L), -166,7 dBFS (R) [-175,5dBFSA (L) -175,9dBFSA (R)]

Gain= 0dB (1x) DC=0 Phase offset=-1742,177083ms (-334498 samples)
Difference (rms) = -166,25dB [-175,54dBA]
Correlated Null Depth=149,53dB [172,4dBA]
Clock drift: 0 ppm


Files are NOT a bit-perfect match (match=100%) at 16 bits
Files are NOT a bit-perfect match (match=97,92%) at 24 bits
Files are NOT a bit-perfect match (match=4,43%) at 32 bits
Files match @ 50,0138% when reduced to 27,82 bits


---- Phase difference (full bandwidth): 3,1323259055771E-05°
0-10kHz: 0,00°
0-20kHz: 0,00°
0-24kHz: 0,00°
Timing error (rms jitter): 0,9ps
PK Metric (step=400ms, overlap=50%):
RMS=-177,8dBFS
Median=-179,6
Max=-170,1

99%: -170,79
75%: -176,58
50%: -179,57
25%: -182,04
1%: -198,39

gn=0,999999972768692, dc=-1,09171566107602E-11, dr=0, of=-334498

DONE!

Signature: b460c6ed4ec6b4619a6bb03de7103137

RMS of the difference of spectra: -203,381801744576dB
DF Metric (step=400ms, overlap=0%):
Median=-300dB
Max=-139dB Min=-300dB

1% > -300,0dB
10% > -300,0dB
25% > -300,0dB
50% > -300,0dB
75% > -144,5dB
90% > -139,84dB
99% > 0,0dB

Linearity 30,6bits @ 0.5dB error

I tried using Quicktime Player as well, and got the exact same result - a tiny difference > 50 kHz (see Delta of spectra below):

100% match at 16 bits
97.92% match at 24 bits

Spoiler

DeltaWave v2.0.25, 2026-08-22T13:07:00.0062479+02:00
Reference: Test Eagles hotel california original.wav[?] 4000122 samples 192000Hz 32bits, stereo, MD5=00
Comparison: Test quicktime player eagles 192khz.wav[?] 5058176 samples 192000Hz 32bits, stereo, MD5=00
Settings:
Gain:True, Remove DC:True
Non-linear Gain EQ:False Non-linear Phase EQ: False
EQ FFT Size:65536, EQ Frequency Cut: 0Hz - 0Hz, EQ Threshold: -500dB
Correct Non-linearity: False
Correct Drift:True, Precision:30, Subsample Align:True
Non-Linear drift Correction:False
Upsample:False, Window:Kaiser
Spectrum Window:Kaiser, Spectrum Size:32768
Spectrogram Window:Hann, Spectrogram Size:4096, Spectrogram Steps:2048
Filter Type:IIR, window:Kaiser, taps:262144, minimum phase=False
Dither:False bits=0
Trim Silence:True
Enable Simple Waveform Measurement: False
Correct for MP DC Filter = False, DC Filter frequency=0,5 Hz

Discarding Reference: Start=0s, End=0s
Discarding Comparison: Start=0s, End=0s

Initial peak values Reference: -1,159dB Comparison: -1,159dB
Initial RMS values Reference: -20,083dB Comparison: -21,102dB

Null Depth=42,916dB
X-Correlation offset: -637602 samples
Drift computation quality, #1: Excellent (0μs)


Trimmed 0 samples ( 0,00ms) front, 0 samples ( 0,00ms end)


Final peak values Reference: -1,159dB Comparison: -1,159dB
Final RMS values Reference: -20,039dB Comparison: -20,039dB

GS: -3320,84375 ms, 0 dB (L), 0 dB (R), -166,3 dBFS (L), -166,7 dBFS (R) [-175,5dBFSA (L) -175,9dBFSA (R)]

Gain= 0dB (1x) DC=0 Phase offset=-3320,84375ms (-637602 samples)
Difference (rms) = -166,25dB [-175,54dBA]
Correlated Null Depth=149,53dB [172,4dBA]
Clock drift: 0 ppm


Files are NOT a bit-perfect match (match=100%) at 16 bits
Files are NOT a bit-perfect match (match=97,92%) at 24 bits
Files are NOT a bit-perfect match (match=4,43%) at 32 bits
Files match @ 50,0138% when reduced to 27,82 bits


---- Phase difference (full bandwidth): 3,1323259055771E-05°
0-10kHz: 0,00°
0-20kHz: 0,00°
0-24kHz: 0,00°
Timing error (rms jitter): 0,9ps
PK Metric (step=400ms, overlap=50%):
RMS=-177,8dBFS
Median=-179,6
Max=-170,1

99%: -170,79
75%: -176,58
50%: -179,57
25%: -182,04
1%: -198,39

gn=0,999999972768692, dc=-1,09171566107602E-11, dr=0, of=-637602

DONE!

Signature: 832ebbccd34339ab33633778b99bb3e5

RMS of the difference of spectra: -203,381801744576dB
DF Metric (step=400ms, overlap=0%):
Median=-300dB
Max=-139dB Min=-300dB

1% > -300,0dB
10% > -300,0dB
25% > -300,0dB
50% > -300,0dB
75% > -144,5dB
90% > -139,84dB
99% > 0,0dB

Linearity 30,6bits @ 0.5dB error

Original (HD Tracks, 192 kHz):

macOS Finder playback:

Conclusion:

foobar2000 on Mac is completely transparent. The ABX log is correct as well (bit depth).

Finder and Quicktime player are not bit perfect at 32-bit, but it's not a meaningful difference (no resampling, and < -166 dB).

 

[solderdude]

And once again, I believe that if Josh can hear a difference in those files, this is most likely what he is able to hear. Because removing the ripple with DW pushes the PK metric down to better than -110dB.

I don't think a PK metric of -60 or -70 is even audible there is no need to reach -110.
We can only guess what the reason might have been.

 

[pjug]

I don't think a PK metric of -60 or -70 is even audible there is no need to reach -110.
We can only guess what the reason might have been.

The question is just if you believe Josh passed the ABX. Even 9/10 is pretty strong evidence if you trust Josh, which I do. If we aren't going to trust anyone then we aren't going to get very far with what we can do online. So if Josh can tell the files apart, why is it? I say look at passband ripple since this is the main difference in the files.

edit: also, the -110dB was the PK metric. The RMS was not as good as that.

 

[danadam]

This ripple is in the freqency domain and doesn't affect the phase, so there will be no echo-like effect.

Here's white noise @44.1k vs the same noise with two -40 dB copies delayed by +25 and -25 samples respectively (about 0.5 ms):

 

[solderdude]

The question is just if you believe Josh passed the ABX. Even 9/10 is pretty strong evidence if you trust Josh, which I do.

The more important question is what he heard that told him the difference and why he did not do a 16 or 20 trials.
It has nothing to do with trusting or not trusting someone.

It would be great if Josh can tell differences others can't distinguish but again evidence and context.

 

[pjug]

The more important question is what he heard that told him the difference and why he did not do a 16 or 20 trials.
It has nothing to do with trusting or not trusting someone.

It would be great if Josh can tell differences others can't distinguish but again evidence and context.

As I see it, we don't need Josh to be able to describe what he heard, but if he is the only person who can hear any difference in the files we do need his continued help. If he passes ABX with the bassband ripple present, and fails with the passband ripple removed, then that is reason to believe he was hearing a passband ripple effect. In that case we could try to get at how the effect presents itself so that it can be heard. It seems to me that part might be difficult.

 

[solderdude]

But then 9/10 would not be sufficient for both files.... that's the point.
And yes the test would not be about testing the files but testing Josh's hearing abilities.
We would also need many others to do the same test to see if indeed the test is not possible to ace for 'ordinary people'.

 

[pma]

Back home. Working on summarization of foobar abx reports collected in this thread, some charts, plots.

Thanks to everyone who has already participated! Special thanks to @pedrojoaquimborges , @NTTY , @RandomEar , who have posted most foo abx reports.

 

[pma]

Again, thanks everyone who has participated in this thread. I have summarized the test results that have been posted as of today, August 23, 2026. First, I am posting a survey of the tests used, as it might become difficult to orientate in them (for better visibility and links, pdf file is at the bottom of this post):

And results of our members, including mine:

We have some results that (probably) might be worth of discussion, they are marked in light yellow and yellow. A single 9/10 result should be supported by repeating it, IMHO. I also cannot see a proof that some of the test files were discerned reliably enough, by listening.

Finally , I would like to apologize if I made some error in the charts here above, as it is a bit difficult to dig the results from this quite long thread. In case you see an error, please let me know and I will correct it.

 

[somebodyelse]

These tests must be done in exclusive mode, bypassing the OS audio pipeline. Otherwise, both MacOS and Windows will resample the audio (to typically 48 kHz), and perform dithering, etc. These transforms could cause one sample to have artifacts and the other not. And at any rate, render any analysis of the original files moot as that is not what is being played.

I don't entirely trust browser-based ABX for similar reasons - browsers have been known to resample too, and the details of their audio output pipeline are less well known than the various OS pipelines. The code is at least there for inspection for Chrome and Firefox though, so it could be checked.

 

[pma]

The question is just if you believe Josh passed the ABX. Even 9/10 is pretty strong evidence if you trust Josh, which I do. If we aren't going to trust anyone then we aren't going to get very far with what we can do online. So if Josh can tell the files apart, why is it?

The problem is a single, not-repeated 9/10 result. As we have explained, 9/10 one time may not be enough. See my 9/10 result in the post #332. You may again ask “why is it”.

 

[NTTY]

Again, thanks everyone who has participated in this thread. I have summarized the test results that have been posted as of today, August 23, 2026. First, I am posting a survey of the tests used, as it might become difficult to orientate in them (for better visibility and links, pdf file is at the bottom of this post):

View attachment 553762

And results of our members, including mine:

View attachment 553764

We have some results that (probably) might be worth of discussion, they are marked in light yellow and yellow. A single 9/10 result should be supported by repeating it, IMHO. I also cannot see a proof that some of the test files were discerned reliably enough, by listening.

Finally , I would like to apologize if I made some error in the charts here above, as it is a bit difficult to dig the results from this quite long thread. In case you see an error, please let me know and I will correct it.

Nice, thank you!
I’ll try to find time to test those that is missed.

 

[pma]

DacMagic+ vs. Toping D10s frequency responses

We have been talking abound passband ripple. Below please find frequency responses of DacMagic+ (linear phase and minimum phase filters) vs. Topping D10s (has one filter) measured with Cosmos ADC at 2.7V input range and Fs=44.1kHz.

I do prefer DacMagic+ filter responses. Its linear and minimum phase filter responses do overlap and have minimum of passband ripple. -0.2dB drop at 20Hz is definitely due to Cosmos input coupling capacitor. Sharp cut just below 20kHz is due to Cosmos ADC filter for Fs= 44.1kHz

Impulse responses

 

[NTTY]

Update August 11, 2026
Part III. Vincent CD-S3 vs. DacMagic+

alphabeta.zip

drive.google.com

Here are my results for this one (6/16), with Topping DX1 II (vol -20dB, low gain) and 7Hz Zero:2:


foo_abx 2.2.1 report
foobar2000 v2.1.5
2026-08-23 12:53:38

File A: beta.wav
SHA1: d6e8965b8daaeb938d38f415a952adbd5bd8df00
File B: alpha.wav
SHA1: 2edddc4333a3ee2f43f3d2c02602433f5b09f783

Output:
Default : Speakers (2- TOPPING USB DAC) [exclusive], 32-bit
Crossfading: NO

12:53:38 : Test started.
12:56:40 : Test restarted.
12:56:40 : 00/01
13:02:27 : Test restarted.
13:02:27 : 00/02
13:02:43 : Test restarted.
13:02:43 : 00/03
13:03:07 : Test restarted.
13:03:07 : 00/04
13:03:35 : Test restarted.
13:03:35 : 00/05
13:04:03 : Test restarted.
13:04:03 : 00/06
13:04:51 : Test restarted.
13:04:51 : 00/07
13:05:18 : Test restarted.
13:05:18 : 01/08
13:06:09 : Test restarted.
13:06:09 : 01/09
13:06:53 : Test restarted.
13:06:53 : 01/10
13:07:32 : Test restarted.
13:07:32 : 02/11
13:08:36 : Test restarted.
13:08:36 : 03/12
13:08:54 : Test restarted.
13:08:54 : 04/13
13:10:06 : Test restarted.
13:10:06 : 05/14
13:11:44 : Test restarted.
13:11:44 : 05/15
13:12:11 : Test restarted.
13:12:11 : 06/16
13:12:11 : Test finished.

----------
Total: 6/16
p-value: 0.8949 (89.49%)

-- signature --
9d247e24d5a2b3a0fa3ad7e977969a71cd8d5fe5

---

Online test Original data vs. Marantz SA7001 output


Here are my results for the one above (7/16), with Topping DX1 II (vol -33dB, low gain) and 7Hz Zero:2:


foo_abx 2.2.1 report
foobar2000 v2.1.5
2026-08-23 13:19:30

File A: original2.wav
SHA1: b2a7efd5dd335f2da2e5ef711efcbbbfd0bb2fce
File B: original1.wav
SHA1: d9841ad90e92c3b4a7b34a99beb6c7234a6c4f38

Output:
Default : Speakers (2- TOPPING USB DAC) [exclusive], 32-bit
Crossfading: NO

13:19:30 : Test started.
13:22:18 : Test restarted.
13:22:18 : 01/01
13:22:42 : Test restarted.
13:22:42 : 02/02
13:23:39 : Test restarted.
13:23:39 : 03/03
13:26:50 : Test restarted.
13:26:50 : 03/04
13:27:12 : Test restarted.
13:27:12 : 03/05
13:27:31 : Test restarted.
13:27:31 : 03/06
13:28:12 : Test restarted.
13:28:12 : 03/07
13:28:27 : Test restarted.
13:28:27 : 04/08
13:28:48 : Test restarted.
13:28:48 : 04/09
13:29:10 : Test restarted.
13:29:10 : 05/10
13:29:34 : Test restarted.
13:29:34 : 05/11
13:29:59 : Test restarted.
13:29:59 : 05/12
13:30:12 : Test restarted.
13:30:12 : 06/13
13:30:33 : Test restarted.
13:30:33 : 07/14
13:30:50 : Test restarted.
13:30:50 : 07/15
13:31:08 : Test restarted.
13:31:08 : 07/16
13:31:08 : Test finished.

----------
Total: 7/16
p-value: 0.7728 (77.28%)

-- signature --
eb49c2f61d56114e2cf09b0ddd2bfd9860d63ba1

---

Next the 6J1:

6J1_test.zip​

drive.google.com

Here are my results for the 6J1_Test (9/16), with Topping DX1 II (vol -33dB, low gain) and 7Hz Zero:2:



foo_abx 2.2.1 report
foobar2000 v2.1.5
2026-08-23 13:38:32

File A: file_x.wav
SHA1: 1941f50c2776fc7f5286573e57a59eddc5a866da
File B: file_y.wav
SHA1: 21f44ff35c139ae7bea391f0bf658b87dcbc06ec

Output:
Default : Speakers (2- TOPPING USB DAC) [exclusive], 32-bit
Crossfading: NO

13:38:32 : Test started.
13:41:47 : Test restarted.
13:41:47 : 01/01
13:41:58 : Test restarted.
13:41:58 : 02/02
13:42:14 : Test restarted.
13:42:14 : 03/03
13:42:35 : Test restarted.
13:42:35 : 03/04
13:42:55 : Test restarted.
13:42:55 : 04/05
13:44:30 : Test restarted.
13:44:30 : 05/06
13:44:58 : Test restarted.
13:44:58 : 05/07
13:45:31 : Test restarted.
13:45:31 : 06/08
13:46:13 : Test restarted.
13:46:13 : 06/09
13:46:53 : Test restarted.
13:46:53 : 07/10
13:47:36 : Test restarted.
13:47:36 : 08/11
13:47:56 : Test restarted.
13:47:56 : 08/12
13:48:13 : Test restarted.
13:48:13 : 08/13
13:48:20 : Test restarted.
13:48:20 : 08/14
13:48:36 : Test restarted.
13:48:36 : 08/15
13:48:51 : Test restarted.
13:48:51 : 09/16
13:48:51 : Test finished.

----------
Total: 9/16
p-value: 0.4018 (40.18%)

-- signature --
a5aa316218d41b755979aac19f7122f31750f7cf

 

[pma]

Thanks! I will be updating the results chart continuously.

 

[RandomEar]

Vincent CD-S3 vs DacMagic+ (#3)
8/16 - This one is deceiving if you analyze it in DeltaWave It looks and sounds (-> delta track) like there should be immediately obvious differences. For example a major peak around 44 s. But then you look at the phase and realize that it starts drifting apart at 1 kHz and that there are huge differences of >50° over 10 kHz. So I assume pretty much all you hear in the delta track is that phase difference and in the ABX, it's just irrelevant for me. Can't hear it.

CD rip vs Marantz SA7001 (#6)
8/16 - You may be able to tell these apart using the noise in the first 0.9 s. I tried and succeeded in training mode, but the actual test failed. Felt dumb anyway - that's not what we are looking for. The attached 8/16 result is from a different timestamp, where actual music was playing.

 

[danadam]

I know intuition is a bad predictor when it comes to statistics,

I fully admit that I'm not a statistician

but it just feels like tests with a smaller number of total trials are more error prone?

To me too, but I mean the opposite direction than the one discussed. Fewer number of trials means smaller statistic power, i.e. smaller probability of detecting the effect in case when the effect really exists. In other words, it is more prone to not detect the effect (e.g. when the effect is small - real but small), because you can't make any mistakes. One wrong result and you are out.

So if you expect on average 2.5/5 correct guesses for an infinite number of test runs using 5 trials per run, that means you are on average just 2.5 "lucky guesses" away from significance in each run.

I agree that you need fewer individual lucky guesses in 5 trials, but I feel that this is counteracted by the fact that you need to be lucky at each and every step. So double lucky, one could say

You also do have proportionally more trials and if you run either the 5 trial or the 16 trial test variant until you reach the same total (very high) number of trials, the results will, of course, converge. However, for a single run, it feels like the 5 trial test has a higher chance of producing outliers.

That doesn't make sense to me. The simulation simply sums the wins. If the chance of a win in a single run were higher, that would sum to higher number. There's nothing in the simulation to "cancel out" previous wins.

That being said, as I understand it 16 trials is the default in foobar2000

If I had to come up with a reason for that, I'd say 16 is high enough to be sensitive to detect small differences and low enough to not cause (overly big) fatigue in the tester. And not that p<5% from 16 trials is somehow better than one from 10 trials.