• Welcome to ASR. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

DAC listening test files - updated with new filters files

NEW TEST FILES

Hello all, after a short break that was needed to prepare and test new files, there is a link to new set of test files.


The test files were prepared by @j_j from my original file dchesorig1.wav, which is also included in the test. Zipped is this original file + 4 files with strange names (names generated randomly), and those 4 files use various types of digital filters. Your task is to test (ABX) the dchesorig1.wav file against any of the remaining 4 files, and post the result, if you are willing to participate. My 71 years old ears are able to tell only one of the filter files against the original, however with high success rate 38/48 (p = 0.00003085). I believe that you who are younger will be able to tell the difference in more files. Have fun and looking forward your results.

Note: Please use higher trial count than 10. If only 10 trials were used, please repeat the test and we use the cumulative result.
Nice. Will check but it may take a while. Traveling.
 
Had some spare time before travel. Difficult and I want to increase the number of trials.

foo_abx 2.2.3 report
foobar2000 v2.25.10
0126-09-21 14:44:28

File A: dchesorig1.wav
SHA1: 9578d6c3b0a59c80ea046a0ecf3fa6688a3e2aff
File B: cahadwlas.wav
SHA1: 2d4d7aa92e7c0f00e4ab61bee478956ba0039bb6

Output:
Core Audio : Default, 24-bit
Crossfading: NO

14:44:28 : Test started.
14:46:43 : Test restarted.
14:46:43 : 00/01
14:47:02 : Test restarted.
14:47:02 : 01/02
14:47:27 : Test restarted.
14:47:27 : 02/03
14:48:18 : Test restarted.
14:48:18 : 03/04
14:48:41 : Test restarted.
14:48:41 : 04/05
14:49:17 : Test restarted.
14:49:17 : 05/06
14:49:34 : Test restarted.
14:49:34 : 06/07
14:50:56 : Test restarted.
14:50:56 : 07/08
14:51:35 : Test restarted.
14:51:35 : 08/09
14:52:19 : Test restarted.
14:52:19 : 09/10
14:52:39 : Test restarted.
14:52:39 : 09/11
14:52:54 : Test restarted.
14:52:54 : 10/12
14:53:26 : Test restarted.
14:53:26 : 11/13
14:53:54 : Test restarted.
14:53:54 : 12/14
14:54:08 : Test restarted.
14:54:08 : 13/15
14:54:35 : Test restarted.
14:54:35 : 13/16
14:55:04 : Test restarted.
14:55:04 : 14/17
14:56:03 : Test restarted.
14:56:03 : 15/18
14:56:55 : Test restarted.
14:56:55 : 16/19
14:57:18 : Test restarted.
14:57:18 : 16/20
14:57:18 : Test finished.

----------
Total: 16/20
p-value: 0.0059 (0.59%)

-- signature --
813d1380eeee4d2eea77509baf2aeb9de5fc6e0d
 
Had some spare time before travel. Difficult and I want to increase the number of trials.
Out of curiosity, if you subtract one file from the other and play the result, do you hear anything?
 
  • Like
Reactions: pma
Out of curiosity, if you subtract one file from the other and play the result, do you hear anything?
All these files can be directly subtracted, that is how @j_j has prepared them. BTW, very good remark.
 
Out of curiosity, if you subtract one file from the other and play the result, do you hear anything?
I do not know how to do that properly; have not had the time to look at them.
 
Even Audacity can do it properly in this case.
 
Even Audacity can do it properly in this case.
Ok. I will not do any more now, time flies, earliest during weekend. I think I will go for 40 trials just to increase power, but it will take some concentration and energy to do it.
 
Last edited:
Great result! You are much better than me.
It can still be due to some issues like IMD. I managed two more sessions tonight.

This one I could not pass.

foo_abx 2.2.3 report
foobar2000 v2.25.10
0126-09-21 19:04:43

File A: dchesorig1.wav
SHA1: 9578d6c3b0a59c80ea046a0ecf3fa6688a3e2aff
File B: drvabdeal.wav
SHA1: 700cf4d9a7f6d72b00f199e71562c2110b753367

Output:
Core Audio : Default, 24-bit
Crossfading: NO

19:04:43 : Test started.
19:07:37 : Test restarted.
19:07:37 : 01/01
19:08:28 : Test restarted.
19:08:28 : 01/02
19:08:37 : Test restarted.
19:08:37 : 02/03
19:10:22 : Test restarted.
19:10:22 : 02/04
19:10:44 : Test restarted.
19:10:44 : 03/05
19:11:03 : Test restarted.
19:11:03 : 04/06
19:12:35 : Test restarted.
19:12:35 : 04/07
19:14:53 : Test restarted.
19:14:53 : 05/08
19:15:49 : Test restarted.
19:15:49 : 06/09
19:16:20 : Test restarted.
19:16:20 : 06/10
19:18:10 : Test restarted.
19:18:10 : 07/11
19:19:44 : Test restarted.
19:19:44 : 07/12
19:20:30 : Test restarted.
19:20:30 : 08/13
19:22:58 : Test restarted.
19:22:58 : 08/14
19:23:49 : Test restarted.
19:23:49 : 08/15
19:24:39 : Test restarted.
19:24:39 : 09/16
19:25:28 : Test restarted.
19:25:28 : 09/17
19:26:20 : Test restarted.
19:26:20 : 10/18
19:27:21 : Test restarted.
19:27:21 : 10/19
19:29:05 : Test restarted.
19:29:05 : 10/20
19:30:52 : Test restarted.
19:30:52 : 11/21
19:31:17 : Test restarted.
19:31:17 : 12/22
19:35:26 : Test restarted.
19:35:26 : 12/23
19:36:47 : Test restarted.
19:36:47 : 13/24
19:37:11 : Test restarted.
19:37:11 : 13/25
19:37:30 : Test restarted.
19:37:30 : 13/26
19:39:19 : Test restarted.
19:39:19 : 14/27
19:40:03 : Test restarted.
19:40:03 : 14/28
19:41:30 : Test restarted.
19:41:30 : 14/29
19:42:55 : Test restarted.
19:42:55 : 15/30
19:42:55 : Test finished.

----------
Total: 15/30
p-value: 0.5722 (57.22%)

-- signature --
4c911755cf78c49fe84029dec01a94db889a004f

This one had something, but I'm not satisfied with p-value.

foo_abx 2.2.3 report
foobar2000 v2.25.10
0126-09-21 19:46:10

File A: dchesorig1.wav
SHA1: 9578d6c3b0a59c80ea046a0ecf3fa6688a3e2aff
File B: pawbdkxmcw.wav
SHA1: ee0523ea6ff4e9ab2299c12ae626720048549d5e

Output:
Core Audio : Default, 24-bit
Crossfading: NO

19:46:10 : Test started.
19:49:05 : Test restarted.
19:49:05 : 00/01
19:50:15 : Test restarted.
19:50:15 : 01/02
19:53:28 : Test restarted.
19:53:28 : 02/03
19:54:40 : Test restarted.
19:54:40 : 03/04
19:55:26 : Test restarted.
19:55:26 : 04/05
19:55:54 : Test restarted.
19:55:54 : 05/06
19:56:15 : Test restarted.
19:56:15 : 06/07
19:57:17 : Test restarted.
19:57:17 : 07/08
19:57:43 : Test restarted.
19:57:43 : 07/09
19:58:27 : Test restarted.
19:58:27 : 07/10
19:59:19 : Test restarted.
19:59:19 : 08/11
19:59:35 : Test restarted.
19:59:35 : 09/12
19:59:44 : Test restarted.
19:59:44 : 10/13
19:59:52 : Test restarted.
19:59:52 : 11/14
19:59:59 : Test restarted.
19:59:59 : 12/15
20:00:07 : Test restarted.
20:00:07 : 13/16
20:00:18 : Test restarted.
20:00:18 : 13/17
20:00:23 : Test restarted.
20:00:23 : 14/18
20:00:47 : Test restarted.
20:00:47 : 15/19
20:01:10 : Test restarted.
20:01:10 : 16/20
20:01:25 : Test restarted.
20:01:25 : 16/21
20:02:18 : Test restarted.
20:02:18 : 17/22
20:02:50 : Test restarted.
20:02:50 : 18/23
20:03:15 : Test restarted.
20:03:15 : 19/24
20:03:34 : Test restarted.
20:03:34 : 19/25
20:04:06 : Test restarted.
20:04:06 : 19/26
20:04:37 : Test restarted.
20:04:37 : 20/27
20:04:49 : Test restarted.
20:04:49 : 21/28
20:05:16 : Test restarted.
20:05:16 : 21/29
20:06:27 : Test restarted.
20:06:27 : 21/30
20:06:27 : Test finished.

----------
Total: 21/30
p-value: 0.0214 (2.14%)

-- signature --
5862073a12a2f1104bb1f56116e13f624739b04b

I also tried the last one but could not find any tells to go after, gave up.
 
Out of curiosity, if you subtract one file from the other and play the result, do you hear anything?
I do not know how to do that properly; have not had the time to look at them.
Even Audacity can do it properly in this case.
Ok. I will not do any more now, time flies, earliest during weekend. I think I will go for 40 trials just to increase power, but it will take some concentration and energy to do it.
Simply put - there is nothing audible left between these files:
File A: dchesorig1.wav
File B: cahadwlas.wav


The whole difference content is well above 20kHz and with not very high level. So, if anything was found as discernable, it must have been due to:
  • intermodulation in electronics used
  • intermodulation in electro-acoustic transducer
  • intermodulation in the human ear
First two would be a systematic error. Only the last one would be of further interest. As this is the long-distance, remote uncontrolled test, it is difficult to derive anything in general. We just know that @Thomas_A has passed the ABX test with success, under his system conditions.
 
Simply put - there is nothing audible left between these files:
File A: dchesorig1.wav
File B: cahadwlas.wav


The whole difference content is well above 20kHz and with not very high level. So, if anything was found as discernable, it must have been due to:
  • intermodulation in electronics used
  • intermodulation in electro-acoustic transducer
  • intermodulation in the human ear
First two would be a systematic error. Only the last one would be of further interest. As this is the long-distance, remote uncontrolled test, it is difficult to derive anything in general. We just know that @Thomas_A has passed the ABX test with success, under his system conditions.
I will repeat that one with 30 trials, I'm quite sceptical I can repeat it. Will do at later time point - my last session was tiring at the end.
 
double post, slow at ASR tonight
 
I did a quick training run for each of the files and only heard a difference with one (pawbdkxmcw). So I ran the test for pawbdkxmcw.wav and got 16/16 (p = 0.0%). I might try some of the other ones again later on, but adding apamwdsaz.wav to the mix seems unnecessary to me. I think we can all expect the technical people on this forum to run an analysis on the files sooner or later, so it doesn't really fulfill its purpose I think.

@j_j I would be interested in hearing some variations of the pawbdkxmcw.wav alterations with reduced amplitude. Maybe 1 dB and 0.5 dB instead of 2dB of you know what ;)


Oh, yeah: Equipment still was the same. SU-9n, SH-9, Zero:2.
 

Attachments

So I ran the test for pawbdkxmcw.wav and got 16/16 (p = 0.0%). I might try some of the other ones again later on, but adding apamwdsaz.wav to the mix seems unnecessary to me
Very good. Both files are there intentionally. They must be there as a rule of DBT tests.
 
I did a quick training run for each of the files and only heard a difference with one (pawbdkxmcw). So I ran the test for pawbdkxmcw.wav and got 16/16 (p = 0.0%). I might try some of the other ones again later on, but adding apamwdsaz.wav to the mix seems unnecessary to me.
Very good. Both files are there intentionally. They must be there as a rule of DBT tests.

I know it is boring for testing. However, those two files, apamwdsaz.wav and pawbdkxmcw.wav MUST be a part of the set. They provide test control. Nobody should tell apamwdsaz.wav against the original and everybody should tell pawbdkxmcw.wav against the original in the ABX test. It is a negative and positive control and it tells us if the test person results are worth considering. This would allow us to leave the purely amateurish/laymen level. In fact then we are only interested in results of drvabdeal.wav and cahadwlas.wav files against the original. This might tell us something.
 
@pma While I understand the reasoning and I would also implement it that way in a controlled lab test, I think we need to be honest here and acknowledge that this is not that. It is a downloadable test on the internet, where we cannot prevent dishonest actors from cheating. Which would be the main reason to include all the files - any technical reason for significant results from identical input data can be excluded this way, but those seem exceedingly unlikely to me.

Nonetheless, I will try to motivate myself to do the additional tests soon :)
 
I managed to complete the set:

FileCorrect guessesp-value
apamwdsaz.wav11/1610.5%
cahadwlas.wav6/1689.5%
drvabdeal.wav8/1659.8%
pawbdkxmcw.wav16/160.0%

Not surprising, I would say. The 11/16 for the control shows that you can get lucky. But I don't feel lucky enough for my next casino visit, yet.
 

Attachments

I managed to complete the set:

FileCorrect guessesp-value
apamwdsaz.wav11/1610.5%
cahadwlas.wav6/1689.5%
drvabdeal.wav8/1659.8%
pawbdkxmcw.wav16/160.0%

Not surprising, I would say. The 11/16 for the control shows that you can get lucky. But I don't feel lucky enough for my next casino visit, yet.

Both of the test filters (and the positive control) cut off at 20khz, some more sharply than others.
Would it be interesting to do 18,16, 14, 12, 10 or some combination of that?
 
I managed to complete the set:

FileCorrect guessesp-value
apamwdsaz.wav11/1610.5%
cahadwlas.wav6/1689.5%
drvabdeal.wav8/1659.8%
pawbdkxmcw.wav16/160.0%

Not surprising, I would say. The 11/16 for the control shows that you can get lucky. But I don't feel lucky enough for my next casino visit, yet.
Your 11/16 result with two identical files (bitperfect) clearly shows why to strictly request for p < 0.05 (I would prefer p << 0.05) and “enough” trials and why 4/5 or 8/10 is absolutely inadequate. In fact only the 16/16 result with the “positive controls” file is significant and important and it confirms your ability to hear the difference between the files reliably.

One note to that file. It has +/-1dB ripple in the audio band and below 200Hz it is just that +1dB. This is why I can tell this file reliably on the sound of the double bass. No way to me to hear the 20kHz cut or “pre-ringing”.
 
Last edited:
Back
Top Bottom