• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

ABX complex non-linear distortion test, about 5% on a music sample

pma

Major Contributor
Joined
Feb 23, 2019
Messages
4,591
Likes
10,727
Location
Prague
This is the test on audibility of complex non-linear distortion. There are two test files, Quintet1 and Quintet2, one of them is the original and the other one is distorted. The link to the zipped test file is:

https://www.dropbox.com/s/kg8abarbdivr9na/quintet_test.zip?dl=0

If you would like to participate in the test, please download the test file (you do not need to have a dropbox account. Click somewhere outside log-in window and then click to download the quintet_test.zip file. Do no try to play the files directly from dropbox, it is impossible.)

There are 3 files in the zip, the 2 test files and a difference file, that demonstrates, by listening, how big (or small) was the distortion used. As a plot, please see the distortion profile below.

quintet_distortion.png


In case you want to participate, I would ask you to make an ABX test in Foobar 2000

http://www.foobar2000.org/components/view/foo_abx

and then to post the full protocol of the test, which we are able to verify at

https://www.foobar2000.org/abx/signaturecheck

I have borrowed the music sample from @Mario Martinez and I hope he would not mind.

Have fun listening the samples!
 
OP
pma

pma

Major Contributor
Joined
Feb 23, 2019
Messages
4,591
Likes
10,727
Location
Prague
OK I will start. This is my result

Code:
foo_abx 2.0.2 report
foobar2000 v1.4.8
2020-12-08 16:42:36

File A: Quintet1.flac
SHA1: 78941da5875a6dc51169e29bbbb909399b75c377
File B: Quintet2.flac
SHA1: 2cea48c9ec2b493807e65e1d2b49deb19b543939

Output:
ASIO : Creative Sound Blaster ASIO
Crossfading: NO

16:42:36 : Test started.
16:43:32 : 01/01
16:43:50 : 01/02
16:43:59 : 02/03
16:44:08 : 03/04
16:44:33 : 03/05
16:45:13 : 04/06
16:45:37 : 05/07
16:45:47 : 06/08
16:45:56 : 07/09
16:46:06 : 08/10
16:46:06 : Test finished.

 ----------
Total: 8/10
Probability that you were guessing: 5.5%

 -- signature --
0b4deacf8293b861f769b4e98db66bc60a645bf6
 
OP
pma

pma

Major Contributor
Joined
Feb 23, 2019
Messages
4,591
Likes
10,727
Location
Prague
So nobody yet, quite as expected. If there is hardly anybody who can hear such high distortion on music, so why there is a challenge of the -120dB THD??

I am posting distortion plots at lower level, still very very high.

quintet_distortion_-9dB.png


quintet_distortion_-20dB.png



The difference file in the zip is included for the reason to be able to find the loudest and most distorted parts of the music sample and to use these time interval in the ABX test. The distortion is even harmonics only and similar to SE tubes. There is a wrong belief that even harmonics make the sound more "pleasant". This is a nonsense, because the even harmonic also create difference IMD tones and these are the most audible distortion components.
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,523
Likes
37,056
The usual excuses. I used my Lenovo Ideapad on board sound to drive Sony headphones. It was late. I've had two glasses of wine. Now other than the wine, I don't think anything was an impediment. Certainly the gear is well below 5% thd.

EDIT:eek:ops I just noticed upon reading the original post you wanted me to post this at the signature check. Sorry. If I try again later, I will.

foo_abx 2.0.6d report
foobar2000 v1.5.3
2020-12-09 02:30:32

File A: Quintet1.flac
SHA1: 78941da5875a6dc51169e29bbbb909399b75c377
File B: Quintet2.flac
SHA1: 2cea48c9ec2b493807e65e1d2b49deb19b543939

Output:
DS : Primary Sound Driver
Crossfading: NO

02:30:32 : Test started.
02:35:17 : 01/01
02:35:29 : 02/02
02:35:39 : 03/03
02:36:02 : 04/04
02:36:15 : 05/05
02:36:23 : 05/06
02:36:40 : 06/07
02:36:49 : 07/08
02:36:59 : 08/09
02:37:09 : 08/10
02:37:09 : Test finished.

----------
Total: 8/10
p-value: 0.0547 (5.47%)

-- signature --
6f340f1633e5b00c14542ba09a3ef2f16a40fb2f
 
Last edited:
OP
pma

pma

Major Contributor
Joined
Feb 23, 2019
Messages
4,591
Likes
10,727
Location
Prague
The usual excuses. I used my Lenovo Ideapad on board sound to drive Sony headphones. It was late. I've had two glasses of wine. Now other than the wine, I don't think anything was an impediment. Certainly the gear is well below 5% thd.

EDIT:eek:ops I just noticed upon reading the original post you wanted me to post this at the signature check. Sorry. If I try again later, I will.

Thank you! Great result! I think that the wine makes the mind open ;). Your signature check is valid, no need to attach more.
------------------

I would like to show how the even harmonics create the IMD components

orig1.png

Original sample

dist1.png

Distorted sample. See the interleaved lines at 100-600Hz and 1kHz-5kHz
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,523
Likes
37,056
Yeah, with high even harmonics you get a kind of a calliope effect. Like a weird instrument is playing along you just barely hear and it doesn't fit. I've listened to a few S.E.T. amps which had too hefty a speaker attached which sounded that way.

I'm not surprised by the low participation. Whenever I've attached files and asked people to choose everyone is afraid.
 

MarsianC#

Active Member
Joined
Jul 23, 2020
Messages
232
Likes
264
Location
Austria
I'm super uncomfortable with this music. However, first try, Aurora DSP/Atom with 990 Pro, no EQ:
foo_abx 2.0.6d report
foobar2000 v1.6.2
2020-12-09 10:55:38

File A: Quintet1.flac
SHA1: 78941da5875a6dc51169e29bbbb909399b75c377
File B: Quintet2.flac
SHA1: 2cea48c9ec2b493807e65e1d2b49deb19b543939

Output:
Default : Primary Sound Driver
Crossfading: NO

10:55:38 : Test started.
10:56:01 : 00/01
10:56:22 : 00/02
10:56:40 : 01/03
10:56:56 : 01/04
10:57:13 : 01/05
10:57:52 : 02/06
10:58:20 : 03/07
10:58:34 : 04/08
10:59:02 : 04/09
10:59:18 : 04/10
10:59:18 : Test finished.

----------
Total: 4/10
p-value: 0.8281 (82.81%)

-- signature --
428be834cd9d13c9a87a4aaa74d62c84630e7289
4/10. Really, I am guessing. Switching tracks is NOT gapless, is it for you?
 
OP
pma

pma

Major Contributor
Joined
Feb 23, 2019
Messages
4,591
Likes
10,727
Location
Prague
I'm super uncomfortable with this music. However, first try, Aurora DSP/Atom with 990 Pro, no EQ:
foo_abx 2.0.6d report
foobar2000 v1.6.2
2020-12-09 10:55:38

File A: Quintet1.flac
SHA1: 78941da5875a6dc51169e29bbbb909399b75c377
File B: Quintet2.flac
SHA1: 2cea48c9ec2b493807e65e1d2b49deb19b543939

Output:
Default : Primary Sound Driver
Crossfading: NO

10:55:38 : Test started.
10:56:01 : 00/01
10:56:22 : 00/02
10:56:40 : 01/03
10:56:56 : 01/04
10:57:13 : 01/05
10:57:52 : 02/06
10:58:20 : 03/07
10:58:34 : 04/08
10:59:02 : 04/09
10:59:18 : 04/10
10:59:18 : Test finished.

----------
Total: 4/10
p-value: 0.8281 (82.81%)

-- signature --
428be834cd9d13c9a87a4aaa74d62c84630e7289
4/10. Really, I am guessing. Switching tracks is NOT gapless, is it for you?

Thank you for participating! To me, switching between tracks is quite smooth, it may be a case of system setting.
 

MarsianC#

Active Member
Joined
Jul 23, 2020
Messages
232
Likes
264
Location
Austria
it may be a case of system setting
I'll check that, Asio is not very stable. I tried cheating with mono and listing to a short piece only. My ears fatigue immediately! Same with Stax Lambda Nova Basic.
Btw I'm 25. Perfect hearing capability to 20 kHz. It must be the gear! :p:p:p
 

bobbooo

Major Contributor
Joined
Aug 30, 2019
Messages
1,479
Likes
2,079
The threshold for decent statistical significance should be at least 13/16 correct trials for an ABX.
 

hyperplanar

Senior Member
Joined
Jan 31, 2020
Messages
301
Likes
581
Location
Los Angeles
Code:
foo_abx 2.0.6d report
foobar2000 v1.6.2
2020-12-09 02:21:35

File A: Quintet1.flac
SHA1: 78941da5875a6dc51169e29bbbb909399b75c377
File B: Quintet2.flac
SHA1: 2cea48c9ec2b493807e65e1d2b49deb19b543939

Output:
Default : Primary Sound Driver
Crossfading: NO

02:21:35 : Test started.
02:25:16 : 01/01
02:25:39 : 02/02
02:25:47 : 03/03
02:25:54 : 04/04
02:26:07 : 05/05
02:26:16 : 06/06
02:26:30 : 07/07
02:26:44 : 08/08
02:26:57 : 09/09
02:27:04 : 10/10
02:27:10 : 11/11
02:27:14 : 12/12
02:27:20 : 13/13
02:27:28 : 14/14
02:27:36 : 14/15
02:27:41 : 15/16
02:27:48 : 16/17
02:27:53 : 17/18
02:27:57 : 18/19
02:28:07 : 19/20
02:28:07 : Test finished.

 ----------
Total: 19/20
p-value: 0 (0%)

 -- signature --
6439baea505b793ddd252eca948135f6c8e0db1b

Do I win something? :p
 

bigjacko

Addicted to Fun and Learning
Joined
Sep 18, 2019
Messages
721
Likes
359
Can someone explain the math behind p value? I am curious what p value would be if we get 0 out of 10.....breaking math?:eek:
 

MarsianC#

Active Member
Joined
Jul 23, 2020
Messages
232
Likes
264
Location
Austria
OP
pma

pma

Major Contributor
Joined
Feb 23, 2019
Messages
4,591
Likes
10,727
Location
Prague
foo_abx 2.0.6d report
foobar2000 v1.6.2
2020-12-09 02:21:35
----------
Total: 19/20
p-value: 0 (0%)

-- signature --
6439baea505b793ddd252eca948135f6c8e0db1b

Do I win something? :p

[/QUOTE]

Great!!!
 
OP
pma

pma

Major Contributor
Joined
Feb 23, 2019
Messages
4,591
Likes
10,727
Location
Prague
Can someone explain the math behind p value? I am curious what p value would be if we get 0 out of 10.....breaking math?:eek:

It may be googled from probability theory. Below is the chart for X successful trials from 10 trials.

probability.png
 

restorer-john

Grand Contributor
Joined
Mar 1, 2018
Messages
12,579
Likes
38,280
Location
Gold Coast, Queensland, Australia
Can I do it in the morning Pavel? It's been a hell of a day and it 10:41pm right now and I too, have had a wine or two... ;)
 
  • Like
Reactions: pma

SoundAndMotion

Active Member
Joined
Mar 23, 2016
Messages
144
Likes
111
Location
Germany
It may be googled from probability theory. Below is the chart for X successful trials from 10 trials.
The third column is not labeled correctly. It is the cumulative probability of guessing X or better.
The probability of guessing and getting 1 or more correct is indeed nearly 1. For 0 or better it is 1.
But if I get 5 profoundly deaf subjects and each answers 0 or 1 correct out of 10, I'm extremely confident, it was not due to guessing. It indicates a systematic error (several possibilities), not guessing. Which systematic error? ...I'd be guessing :D
 
  • Like
Reactions: pma
OP
pma

pma

Major Contributor
Joined
Feb 23, 2019
Messages
4,591
Likes
10,727
Location
Prague
The third column is not labeled correctly. It is the cumulative probability of guessing X or better.

Yes I know that I am not strong in proper math terms, I took the one from older ABX plug-in description, which was later corrected to "p-value". However, the method of calculation is valid for our purposes and remained the same in both plug-in versions.

Note: we should not make cherry-picking. I, myself, made one test run. It is possible to make some training to be familiar with the files but then please confess if there were more test runs. Personally, I would need at least 15 minute pause before repeating the test. On the other hand, 7/10 and 8/10, even with pause, may be summed to 15/20.

I just passed a 2nd test run, again 8/10

Code:
foo_abx 2.0.2 report
foobar2000 v1.4.8
2020-12-09 14:37:55

File A: Quintet1.flac
SHA1: 78941da5875a6dc51169e29bbbb909399b75c377
File B: Quintet2.flac
SHA1: 2cea48c9ec2b493807e65e1d2b49deb19b543939

Output:
ASIO : Creative Sound Blaster ASIO
Crossfading: NO

14:37:55 : Test started.
14:39:04 : 01/01
14:39:17 : 01/02
14:39:28 : 02/03
14:39:46 : 03/04
14:40:08 : 04/05
14:40:21 : 05/06
14:40:34 : 05/07
14:40:47 : 06/08
14:40:59 : 07/09
14:41:16 : 08/10
14:41:16 : Test finished.

 ----------
Total: 8/10
Probability that you were guessing: 5.5%

 -- signature --
562bcd39d8a48f15c5b057b7c46f4cf9918e15e1
 
Last edited:

pfzar

Member
Joined
Nov 4, 2020
Messages
13
Likes
19
Location
SF
If you would like to take the test below. It's available until 12-31-20, It's completely anonymous. I will post the data after the test is concluded.

updated LINK
 
Last edited:
Top Bottom