• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required as is 20 years of participation in forums (not all true). Come here to have fun, be ready to be teased and not take online life too seriously. We now measure and review equipment for free! Click here for details.

Codec ABX testing

flipflop

Senior Member
Joined
Feb 22, 2018
Messages
308
Likes
257
#1
Time for some codec ABX testing.


Program material
The recordings are 24-bit .wav files with a sample rate of 96 kHz taken directly from the listener training software “Harman How to Listen.”
Conversion to MP3 was done using foobar2000 v1.3.16 with LAME 3.100 64bit.

Track 1: How Long by Bill Cantos
Bill is an extremely gifted singer, songwriter, arranger, and pianist. His fabulous arranging skills are truly shown on this classic pop hit from the 70’s.
https://mega.nz/#!cwBgkYyJ!pvwrdcSQmOKE4VITSY8JTmtsAAn7tTplhH8LkWqrf0E

Track 2: Clap Hands by George Krikes
George is a young indie/pop artist with a smooth voice, who’s a great arranger and a wonderful songwriter. His appeal will span a broad demographic.
https://mega.nz/#!lkZmkCLQ!rak2jLT7xxXhaOhcp1WEnEEvdxTrDrzYkLXOuxz_VV0

Track 3: Four Wheel Drive by Paul Thorn
Paul is a brilliant singer/song writer who forms his unique brand of American storytelling from the world around him. Descriptive terms such as inspiration, eccentricity, and character all tell part of this story. But hearing him live says it all!
https://mega.nz/#!R0ZQnADJ!p9w7N-_Kf3wkqEl0wxDhPiVPXXK75rjlDnKQWJDPPzk

Track 4: Fiesta Linda by Open Hands
Led by the legendary Abraham Laboriel of whom Larry Carleton once said. “There are a lot of great bass players in this world, but there is one, and only one Abraham Laboriel.” Everything this band plays embraces the joy of music making to the fullest!
https://mega.nz/#!osJkwI7J!9bw_PH4ogMX8-r4megZFcgt_xWqHkmx7rgmkEylTY78


Codec & trial recommendations
Use the original .wav files for the blind test along with an .mp3 file of your choice.
Trained listeners* should start with V6, experienced** listeners with V7, inexperienced with V8. V9 is only for confirming your programs and equipment are set up correctly or that you are not totally deaf.
In order for the probability of guessing to be lower than 1%, a minimum of 7 trials is required in which you get the answer right in all of them. Here's a list with examples of other possible outcomes that also meet the <1% criteria:
8/8
9/9
10/10
≥10/11
≥11/12
≥12/13
≥12/14
≥13/15
≥14/16
http://djcarlst.provide.net/abx_bino.htm

* Skill level eight or higher in Harman How to Listen.
** As defined by yourself ;)


ABX software
I believe the ABX Comparator for foobar is still the go-to tool for audio file ABX testing.

A log from a successfully conducted blind test should look something like this:
foo_abx 2.0.6c report
foobar2000 v1.3.16
2019-05-12 13:58:58

File A: (V9) Bill Cantos - How Long.mp3
SHA1: 7e9771357896c734433f62da9cc6166e62c2a953
File B: (wav) Bill Cantos - How Long.wav
SHA1: 0329f08e2447afac7f80c17a3d7357b4b7724605

Output:
DS : Primær lyddriver
Crossfading: NO

13:58:58 : Test started.
13:59:09 : 01/01
13:59:17 : 02/02
13:59:27 : 03/03
13:59:34 : 04/04
13:59:39 : 05/05
13:59:48 : 06/06
14:00:10 : 07/07
14:00:23 : 08/08
14:00:34 : 09/09
14:00:45 : 10/10
14:00:45 : Test finished.

----------
Total: 10/10
p-value: 0.001 (0.1%)

-- signature --
d741baa837eaad11773f7951364595e76b496480
It's mandatory for you to post your log if you claim to hear a difference between the formats of the same recording.


Break a leg.
 
Last edited:

flipflop

Senior Member
Joined
Feb 22, 2018
Messages
308
Likes
257
#2
Bumping with the fact that I have now failed 3 ABX tests of V6 against lossless.
I would be extremely impressed if someone could demonstrate a perceivable difference between the two.
 
Joined
May 31, 2019
Messages
23
Likes
6
#3
Hi

Good idea...
BUT, could you be so kind as doing the preparation work for us?
I have not enough time to perform the full test.
So why not to post shortened samples with just the 20s of each containing the most relevant differences ?
Would that make the test less relevant ? (I don't think so)
 

flipflop

Senior Member
Joined
Feb 22, 2018
Messages
308
Likes
257
#5
So why not to post shortened samples with just the 20s of each containing the most relevant differences ?
I didn't want to tamper with the original files and the songs are already quite short as they are, ranging from 18 to 43 seconds.
What you are asking for can actually be done with the foobar ABX Comparator, though:
ABX Comparator.PNG

also - add AAC 320 ABR
Your wish is my command. I updated the folders and they now include AAC 320 ABR.
 
Top Bottom