• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

ABX complex non-linear distortion test, about 5% on a music sample

bobbooo

Major Contributor
Joined
Aug 30, 2019
Messages
1,479
Likes
2,079
If the tracks are played online through a browser we have no idea if they were subject to sample rate conversion (common on Android), and on PC are likely played in Widows shared audio mode by most users. There's also no verifiable log with signature. The Foobar ABX Comparator is simple enough for anyone to do, so less controlled online tests just muddy the waters really.
 
  • Like
Reactions: pma

pfzar

Member
Joined
Nov 4, 2020
Messages
13
Likes
19
Location
SF
So glad that I offered an objective view to hosting the test.
 

bigjacko

Addicted to Fun and Learning
Joined
Sep 18, 2019
Messages
721
Likes
359
It may be googled from probability theory. Below is the chart for X successful trials from 10 trials.

View attachment 98204
But if I am purely guessing, getting 0 out of 10 and 10 out of 10 would be the same very low possibility. Getting 5 out of 10 should be most likely I am guessing. This chart does not look like it can go both ways, only one way which is getting it right, wonder if it is correct.
 

MarsianC#

Active Member
Joined
Jul 23, 2020
Messages
232
Likes
264
Location
Austria
Law of large numbers. Flip a coin 10^6 times and it will be more are less (very very close) 50/50, as one would expect. But it won't if you do it only 10 times. Hence the need for tests to make sure your result is likely to be correct or not. Bu I'm getting things mixed up. Look into it if you're really interested, it's no crazy math. Actually it's super boring math.
Never liked statistics and probability...
 

maty

Major Contributor
Joined
Dec 12, 2017
Messages
4,596
Likes
3,160
Location
Tarragona (Spain)
Come on Pavel, it is a very easy test. With only 5", JRiver MC. The difference is very evident!!! Who does not immediately notice the difference is that it has a large bottleneck in its audio system.

I want something like this: H2 at -70dB vs H2 at -80dB vs H2 at -90dB.

You know, I want H2 at -73dB as Nelson Pass (jazz, small group, instrumental and natural voice). Or something H2 at -83 dB to listen to ALL kind of very good recordings.
 
Last edited:
OP
pma

pma

Major Contributor
Joined
Feb 23, 2019
Messages
4,591
Likes
10,727
Location
Prague
If you would like to take the test here ABX test. It's available until 12-31-20, It's completely anonymous. I will post the data after the test is concluded.

No problem, thank you!

But, frankly, I prefer Foobar ABX, for the reason that one can make preliminary training and make a choice of the time interval where the difference is the biggest. In case of very similar samples, like here, it would be difficult to tell the difference by continuous streaming. Depends what we expect, to hear small differences or to hear big, substantial differences.
 
Last edited:

pfzar

Member
Joined
Nov 4, 2020
Messages
13
Likes
19
Location
SF
Just realized it's only doing 2 trials. Editing a new test that will have more,updated link
 
Last edited:
OP
pma

pma

Major Contributor
Joined
Feb 23, 2019
Messages
4,591
Likes
10,727
Location
Prague
Come on Pavel, it is a very easy test. With only 5", JRiver MC. The difference is very evident!!! Who does not immediately notice the difference is that it has a large bottleneck in its audio system.

I want something like this: H2 at -70dB vs H2 at -80dB vs H2 at -90dB.

You know, I want H2 at -73dB as Nelson Pass (jazz, small group, instrumental and natural voice). Or something H2 at -83 dB to listen to ALL kind of very good recordings.

You may say that, but you are not posting any ABX log signature, so unfortunately I cannot take your opinion into account. I also do not take the complaints that Foobar "does not sound good". I made numerous tests that confirmed that Foobar sends bit perfect 16 and 24 bit files. Data were sent through optical SPDIF, received, and the two files compared by subtraction and the result was 00000.................00000. Nothing. Nada. Of course you need properly installed drivers and no OS resampling and "enhancements" to get the proper result.

Regarding the music for the test, I beg to differ. I do not use (jazz, small group, instrumental and natural voice) for testing purposes. But please feel free to make your own test with the music that you like and at distortion levels that you have mentioned. There is a SW by @pkane that will allow you to make so (but it was not used in my test). However, unless there is not the ABX result, I cannot take it serious, I apologize.
 

escape2

Addicted to Fun and Learning
Joined
Mar 8, 2019
Messages
883
Likes
944
Location
USA
Since I can't hear any difference between the two files in the OP, I guess there is no point in me even attempting the ABX test.
 
OP
pma

pma

Major Contributor
Joined
Feb 23, 2019
Messages
4,591
Likes
10,727
Location
Prague
Since I can't hear any difference between the two files in the OP, I guess there is no point in me even attempting the ABX test.

Try the time interval selection between 16 s and 24.5 s, or 19.5 s and 24.5 s. Audible distortion is highest there.
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,524
Likes
37,056
Since I can't hear any difference between the two files in the OP, I guess there is no point in me even attempting the ABX test.
Maybe, but not necessarily. I've found on a few signals I will think I'm hearing nothing different yet somehow pick correctly almost all the time. Not just for one set of trials, but repeatedly. So do a quick run of 10 or 15.

BTW, my preference would have been 20 trials minimum. I stopped at 10 because the OP did, and I thought he'd want consistency of results. My usual method of doing 20 is to do 10, rest for a few minutes and do 10 more.
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,524
Likes
37,056
Try the time interval selection between 16 s and 24.5 s, or 19.5 s and 24.5 s. Audible distortion is highest there.

I did listen for a few minutes to pick where to test. I keyed in around those places you just listed, but ended up going with 11.9 to 13.9 seconds if I remember rightly.
 
  • Like
Reactions: pma
OP
pma

pma

Major Contributor
Joined
Feb 23, 2019
Messages
4,591
Likes
10,727
Location
Prague
Maybe, but not necessarily. I've found on a few signals I will think I'm hearing nothing different yet somehow pick correctly almost all the time. Not just for one set of trials, but repeatedly. So do a quick run of 10 or 15.

BTW, my preference would have been 20 trials minimum. I stopped at 10 because the OP did, and I thought he'd want consistency of results. My usual method of doing 20 is to do 10, rest for a few minutes and do 10 more.

In fact I did 20 trials as well, both 8/10 and posted, however there was a 1 day rest :D. Seriously, 20 trials in a row, in case of small sound differences like here, is too much, the concentration is fading.
 

maty

Major Contributor
Joined
Dec 12, 2017
Messages
4,596
Likes
3,160
Location
Tarragona (Spain)
Well, I will not participate more. If it had been more difficult I would have done several tests, with loudspeakers, headphones, foobar2000 (with and without ABX plugin), JRMC ... but the difference is so noticeable that it is not worth trying. With DISTORT I was complicating the tests but the initial one was not as easily detectable as yours.
 

LTig

Master Contributor
Forum Donor
Joined
Feb 27, 2019
Messages
5,760
Likes
9,442
Location
Europe
If you would like to take the test below. It's available until 12-31-20, It's completely anonymous. I will post the data after the test is concluded.

updated LINK
Sorry, can't hear any difference between them all... (too old, hearing too bad :confused:). Well, good to know that I can spare a lot of money, and the main reason to get the RME ADI-2 PRO fs was measuring.
 

CaféNoir

Member
Joined
Jul 17, 2020
Messages
6
Likes
1
Hi Pavel, nice to see you come back with some tests!
Ok here is my first attempt. I had to guess during the test that A stays A, and B stays B throughout, only X and Y changes, which did not help. I will try to install wasapi or ASIO next.

foo_abx 2.0.6d report
foobar2000 v1.6.2
2020-12-09 21:55:09

File A: Quintet1.flac
SHA1: 78941da5875a6dc51169e29bbbb909399b75c377
File B: Quintet2.flac
SHA1: 2cea48c9ec2b493807e65e1d2b49deb19b543939

Output:
Default : Primary Sound Driver
Crossfading: NO

21:55:09 : Test started.
22:01:50 : 01/01
22:02:41 : 02/02
22:08:40 : 03/03
22:09:06 : 04/04
22:11:10 : 04/05
22:12:12 : 05/06
22:13:07 : 06/07
22:13:44 : 06/08
22:14:10 : 06/09
22:14:33 : 07/10
22:14:33 : Test finished.

----------
Total: 7/10
p-value: 0.1719 (17.19%)

-- signature --
6a05d8090e3d3a2108473701283f1b6b9b189839

Music feels quite "distorted" or inharmonic in itself, which makes it a rather tricky and fatiguing test.
It is very strange that the difference is quite immediate at first, but ABX seems to make it abnormally difficult maybe due to repetition, listening fatigue, or other psychologic reasons. Like eating the same meal over and over, you lose the taste.

Can we talk about what we think is the distorted sample, and perceived subjective differences, or not yet?
 
  • Like
Reactions: pma

dougi

Addicted to Fun and Learning
Joined
May 31, 2020
Messages
845
Likes
763
Location
ACT, Australia
Here is mine just using ATH-ES7 headphones off the laptop into my 52 YO, 10kHz limited, tinnitus riddles ears.

Worse than guessing!

foo_abx 2.0.6d report
foobar2000 v1.5.5
2020-12-10 17:35:38

File A: Quintet1.flac
SHA1: 78941da5875a6dc51169e29bbbb909399b75c377
File B: Quintet2.flac
SHA1: 2cea48c9ec2b493807e65e1d2b49deb19b543939

Output:
DS : Primary Sound Driver
Crossfading: NO

17:35:38 : Test started.
17:37:37 : 00/01
17:38:00 : 01/02
17:38:14 : 01/03
17:39:02 : 02/04
17:39:59 : 02/05
17:40:43 : 02/06
17:42:04 : 03/07
17:43:00 : 04/08
17:43:37 : 04/09
17:44:27 : 04/10
17:45:33 : 04/11
17:45:55 : 04/12
17:46:42 : 05/13
17:47:10 : 06/14
17:48:19 : 06/15
17:48:59 : 07/16
17:48:59 : Test finished.

----------
Total: 7/16
p-value: 0.7728 (77.28%)

-- signature --
f1c6d1042cd1fd69e2f2c7336dc95666bbcf7ede
 
  • Like
Reactions: pma
OP
pma

pma

Major Contributor
Joined
Feb 23, 2019
Messages
4,591
Likes
10,727
Location
Prague
Hi Pavel, nice to see you come back with some tests!
Ok here is my first attempt. I had to guess during the test that A stays A, and B stays B throughout, only X and Y changes, which did not help. I will try to install wasapi or ASIO next.

Music feels quite "distorted" or inharmonic in itself, which makes it a rather tricky and fatiguing test.
It is very strange that the difference is quite immediate at first, but ABX seems to make it abnormally difficult maybe due to repetition, listening fatigue, or other psychologic reasons. Like eating the same meal over and over, you lose the taste.

Can we talk about what we think is the distorted sample, and perceived subjective differences, or not yet?

Thank you for your kind words. I understand the situation that you feel you can hear the clear difference by listening the files and then the ABX has a negative result. This is based on the fact that we use ear/brains processor for the listening and not some technical measuring instrument. Both ear (biological/neurological sensor) and brains (mind driven processor) have great influence to what we hear. It is difficult to get a non-biased result without a double blind test, and it is also difficult to get adapted to double blind test conditions.


So please feel free to describe your feelings but please take into account that I consider the ABX test result.


Regarding the “distortion” of the original music sample, this may be again related to high concentration to the test listening and ear/brains processing and mind evaluation. The question is what is perceived as a “distortion” - would it be real non-linear distortion, or reflections, room acoustics, or anything else? Again difficult to say. To me, the sample sounds very natural and that is why I have chosen it for the test.

Thank you and also @dougi and @LTig for participating in the test.
 
OP
pma

pma

Major Contributor
Joined
Feb 23, 2019
Messages
4,591
Likes
10,727
Location
Prague
One day pause, one more try, so I think it is proven. 8/10, 8/10 and 9/10. Tested interval was 7s - 12s. This time I took my headphone amp with close to zero output impedance (Sennheiser HD598 headphones).
BUF2_headamp1.jpg


Code:
foo_abx 2.0.2 report
foobar2000 v1.4.8
2020-12-10 20:26:03

File A: Quintet1.flac
SHA1: 78941da5875a6dc51169e29bbbb909399b75c377
File B: Quintet2.flac
SHA1: 2cea48c9ec2b493807e65e1d2b49deb19b543939

Output:
ASIO : Creative Sound Blaster ASIO
Crossfading: NO

20:26:03 : Test started.
20:28:32 : 01/01
20:29:04 : 02/02
20:29:28 : 02/03
20:29:44 : 03/04
20:30:07 : 04/05
20:30:24 : 05/06
20:30:48 : 06/07
20:31:19 : 07/08
20:31:35 : 08/09
20:31:59 : 09/10
20:31:59 : Test finished.

----------
Total: 9/10
Probability that you were guessing: 1.1%

-- signature --
05713a351055ed3def9a5c9b72de31182d3308f6
 
Last edited:

Thomas_A

Major Contributor
Forum Donor
Joined
Jun 20, 2019
Messages
3,422
Likes
2,407
Location
Sweden
No problem, thank you!

But, frankly, I prefer Foobar ABX, for the reason that one can make preliminary training and make a choice of the time interval where the difference is the biggest. In case of very similar samples, like here, it would be difficult to tell the difference by continuous streaming. Depends what we expect, to hear small differences or to hear big, substantial differences.

You can select the time interval where the difference is the biggest also with the linked trial.
 
  • Like
Reactions: pma
Top Bottom