• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required as is 20 years of participation in forums (not all true). Come here to have fun, be ready to be teased and not take online life too seriously. We now measure and review equipment for free! Click here for details.

Audio Blind Tests and Listener Training (video)

Newman

Addicted to Fun and Learning
Joined
Jan 6, 2017
Messages
946
Likes
1,070
Good write up. Worth noting that MP3 encoders improved in the years since 20 years ago, when those tests were done, so MP3 encoded today at the same bitrate would be harder to identify from source than it was in that write up.

Also, MP3 has been surpassed in audio terms. In fact MP3 is officially obsolete. Modern codecs like AAC at a constant bitrate of 128 or 256 kbps would be much harder to identify from uncompressed source music than MP3 was in that write up. Although nobody today would use constant bitrate at a medium-low bitrate like 128 kbps, so using variable bitrate AAC would improve the audio quality again (ie making it even more difficult to detect than constant bitrate), if the target file size was the same as 128 kbps constant bitrate MP3 file sizes in 2000.

cheers
 
Last edited:
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
35,410
Likes
136,173
Location
Seattle Area
we can all distinguish 16 bit vs 24 bit by playing a LSB signal and amplifying the output by 100 dB
No such effort was made whatsoever. I simply turned up the analog volume control from normal listening level to a bit higher. This was using my laptop. No way it had 10 dB of headroom let alone 100 dB. Indeed, I explained that I discarded an artificial test where I used software amplification (which I think was 30 or 60 dB). So please don't make up scenarios like this.

we can all distinguish 32 bit vs 24 bit by playing a LSB signal and amplifying the output by 150 dB
If you mean software amplification, then you are violating the test by playing a different set of files than what was provided. So that would be a clear cheat. If you mean you have the ability to provide analog amplification of 150 dB, then you must be living in a different universe than the rest of us!

However, this is a real scenario to the extent to you do have much amplification. Any noise you hear then is an artifact that is audible and means the format you are playing is not transparent.

I suggest you actually try to see if you can pass the test as I did. That would train you for what to listen for which was the purpose of the video.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
35,410
Likes
136,173
Location
Seattle Area
Good write up. Worth noting that MP3 encoders improved in the years since 20 years ago, when those tests were done, so MP3 encoded today at the same bitrate would be harder to identify from source than it was in that write up.
Whether you can pass such a test highly depends on training and the content. This is my passing one such test at 320 kbps when challenged last by the late Arny Krueger:

foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/07/19 19:45:33

File A: C:\Users\Amir\Music\Arnys Filter Test\keys jangling 16 44.wav
File B: C:\Users\Amir\Music\Arnys Filter Test\keys jangling 16 44_01.mp3

19:45:33 : Test started.
19:46:21 : 01/01 50.0%
19:46:35 : 02/02 25.0%
19:46:49 : 02/03 50.0%
19:47:03 : 03/04 31.3%
19:47:13 : 04/05 18.8%
19:47:27 : 05/06 10.9%
19:47:38 : 06/07 6.3%
19:47:46 : 07/08 3.5%
19:48:01 : 08/09 2.0%
19:48:19 : 09/10 1.1%
19:48:31 : 10/11 0.6%
19:48:45 : 11/12 0.3%
19:48:58 : 12/13 0.2%
19:49:11 : 13/14 0.1%
19:49:28 : 14/15 0.0%
19:49:52 : 15/16 0.0%
19:49:56 : Test finished.

----------
Total: 15/16 (0.0%)

I took one of his files which had posted per above for another test and passed the ABX test. This way, I was not picking some special file that made the job easier for me.

Also, MP3 has been surpassed in audio terms. In fact MP3 is officially obsolete. Modern codecs like AAC at 128 or 256 kbps would be much harder to identify from uncompressed source music than MP3 was in that write up. Although nobody today would use constant bitrate at a medium-low bitrate like 128 kbps, so using variable bitrate would improve the audio quality (ie making it even more difficult to detect), if the target file size was the same as 128 kbps constant bitrate MP3 file sizes in 2000.
The original FHG reference encoder was actually quite good. A lot of people messed around with it but only made improvements at higher bit rates. At lower bit rates by relaxing the bandwidth limitations, they actually made things worse.

AAC is much, much better codec. No one who has access to one should be using MP3. MP3 was never meant to be transparent (at any bit rate) and hence the development of AAC to replace it. I was challenged to pass AAC test using VBR coding at 192 kbps and I passed that as well:

foo_abx 1.3.4 report
foobar2000 v1.3.2
2017/11/07 21:50:26

File A: C:\Users\Amir\Documents\Test Music\AAC test\01-01 Cadenza I (FLAC 96.0 kHz 24-bit)_02.flac
File B: C:\Users\Amir\Documents\Test Music\AAC test\01-01 Cadenza I (FLAC 96.0 kHz 24-bit)_02_01_192aac_01.wav

21:50:26 : Test started.
21:52:06 : 01/01 50.0%
21:52:24 : 02/02 25.0%
21:52:37 : 03/03 12.5%
21:52:57 : 04/04 6.3%
21:53:11 : 04/05 18.8%
21:53:22 : 05/06 10.9%
21:53:36 : 06/07 6.3%
21:53:48 : 07/08 3.5%
21:54:01 : 08/09 2.0%
21:54:22 : 09/10 1.1%
21:54:40 : 10/11 0.6%
21:55:14 : 11/12 0.3%
21:55:50 : Test finished.

----------
Total: 11/12 (0.3%)

The source was some high-res file for another challenge.

Anyway, the topic of this video was what a critical listener is and proof of the same in the form of controlled testing. If you think all of these formats sound the same, then you are not a critical listener.
 
D

Deleted member 27694

Guest
Amir do you think there is a significant difference between 24/16 under normal listening conditions? by normal I mean listening to music with headphones or speakers. Lets say you use your favorite headphones and use your favorite song (or 1min snippets) at normal or high volume? would you say that you would detect a difference and pass a test in a “normal” listening situation? Thank you!
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
35,410
Likes
136,173
Location
Seattle Area
I think your question needs expansion. How did you create the 16 bit and 24 bit files? Was the former created using the latter? If so, what was the conversion? Truncation? Dither? Noise shaping? And at what playback level.

If you mean it in a less precise manner, then no, I don't think I can because a lot of content has a lot of noise in it already. I am certainly quite happy listening to 16 bit content.
 

Newman

Addicted to Fun and Learning
Joined
Jan 6, 2017
Messages
946
Likes
1,070
@amirm Analog or digital, today’s tech or tomorrow’s tech, 100 dB or 150 dB or 75 dB or 60 dB, my point stands. It wasn’t a test of a competent recording played at a level where the primary program is listenable.

It wasn’t make clear enough that you didn’t demonstrate whether you can distinguish the two in real listening scenarios.
 
D

Deleted member 27694

Guest
I think your question needs expansion. How did you create the 16 bit and 24 bit files? Was the former created using the latter? If so, what was the conversion? Truncation? Dither? Noise shaping? And at what playback level.

If you mean it in a less precise manner, then no, I don't think I can because a lot of content has a lot of noise in it already. I am certainly quite happy listening to 16 bit content.
Thank you! i can not specify i just don’t have that technical knowledge. But I know when YOU are happy listening to 16 bit then it is perfectly fine for me:)
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
35,410
Likes
136,173
Location
Seattle Area
It wasn’t make clear enough that you didn’t demonstrate whether you can distinguish the two in real listening scenarios.
It doesn't get more real than the way I passed the test. Every listener has a volume control for their headphone. If you know what to look for, have proper training, you can pass the test showing the format is not transparent. No trickery was used as you were suggesting with software manipulation.

Remember, there are people with better hearing than me. So if I can pass the test, others may be able to pass it with even less preconditions.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
35,410
Likes
136,173
Location
Seattle Area
@amirm Analog or digital, today’s tech or tomorrow’s tech, 100 dB or 150 dB or 75 dB or 60 dB, my point stands. It wasn’t a test of a competent recording played at a level where the primary program is listenable.
No, your point doesn't stand. You modified the file if using software which is not allowed in controlled testing. And if you used analog amplification, you had to demonstrate that you could pass such listening test as I did. No pontification about 150 dB, noise-free amplification. I suggest you watch the video again as you have lost the plot here.
 

Newman

Addicted to Fun and Learning
Joined
Jan 6, 2017
Messages
946
Likes
1,070
@amirm OK I have had another look at the video and I am happy to retract. You do make the statement (approx 18m50s timestamp) that, if you listen to the musical piece without cranking the volume of the trail-out, you would be hopeless at detecting a difference. But IMHO by the end, you had repeated so often that one can detect differences, (if one has the innate ability, plus well-trained, plus knowing where to focus one's listening, plus in this case cranking the volume in a near-silent passage), I felt that the earlier message, that it would be hopeless in real listening to musical pieces, was kind of drowned out on balance. It would be too easy, IMHO, for some audiophiles to watch your video and tell themselves "these things sound detectably different, so lesser tech is going to make my music sound lesser", even though that is obviously not your opinion.

You also mention (approx 13m timestamp) that Archimago's 16/24 bit test was wrong to ask for preference, and I get your point if the goal is to know if the difference is detectable, but it is also a suitable goal (dare I say a more relevant goal for the audiophile mainly focused on ultimate music-listening experiences) for Archimago to say that there was not a detectable preference when listening to actual music at 16 vs 24 bit.

I hope I am not misrepresenting you to say that this video is emphasising that it is wrong for some scribes to run around saying it is completely impossible to detect when something is 16 bit vs 24 bit, nth-generation re-encoding, strong jitter added, etc, but those things won't create preferred/less preferred music listening experiences.

cheers
 
D

Deleted member 27694

Guest
@amirm OK I have had another look at the video and I am happy to retract. You do make the statement (approx 18m50s timestamp) that, if you listen to the musical piece without cranking the volume of the trail-out, you would be hopeless at detecting a difference. But IMHO by the end, you had repeated so often that one can detect differences, (if one has the innate ability, plus well-trained, plus knowing where to focus one's listening, plus in this case cranking the volume in a near-silent passage), I felt that the earlier message, that it would be hopeless in real listening to musical pieces, was kind of drowned out on balance. It would be too easy, IMHO, for some audiophiles to watch your video and tell themselves "these things sound detectably different, so lesser tech is going to make my music sound lesser", even though that is obviously not your opinion.

You also mention (approx 13m timestamp) that Archimago's 16/24 bit test was wrong to ask for preference, and I get your point if the goal is to know if the difference is detectable, but it is also a suitable goal (dare I say a more relevant goal for the audiophile mainly focused on ultimate music-listening experiences) for Archimago to say that there was not a detectable preference when listening to actual music at 16 vs 24 bit.

I hope I am not misrepresenting you to say that this video is emphasising that it is wrong for some scribes to run around saying it is completely impossible to detect when something is 16 bit vs 24 bit, nth-generation re-encoding, strong jitter added, etc, but those things won't create preferred/less preferred music listening experiences.

cheers

I also watched it again. He did say that and he also said "I went to the end of the track and got lucky because there was a fade to zero" (19m36s)

@amirm I hope I am not misrepresenting you to say that this video is emphasizing that it is wrong for some scribes to run around saying it is completely impossible to detect when something is 16 bit vs 24 bit, nth-generation re-encoding, strong jitter added, etc, but those things won't create preferred/less preferred music listening experiences.[

That was also my point from the beginning. I understand that from an engineering standtpoint you thrive for absolute perfection. And the goal is transparency - under all circumstances. But I think transparency should be measured against "preferred/less preferred music listening experiences." So what I would suggest is we need a definition for "the perfectly clean apartment, done by the best cleaning service in the U.S." Yes, you can still argue it is not perfect and a guy who knows what to look for with a black-light and a microscope (volume knob) can proof it.

But I understand NOW that it was never Amirs point to say "Hey guys I am 40+ and even I can hear a difference between 24/16 so get rid of that crap an stop ruining your ears"

IMHO there is absolutely no difference for music enjoyment between 16/24 (and for me personally even [email protected]). I think because of the loudness war CD-quality got a bad rep and 24 catered for an "audiophile" audience an AFAIK it is not the difference between the bit depths that people hear, but rather the difference in the quality of the digital mastering. (better dynamic range) Because to hear the difference in dynamic range between a 16-bit and a 24-bit recording in a normal quiet listening room, you would have to go so loud it would cause permanent hearing loss. I think we can agree most people don`t want that.
 

Fokken_Ossom

New Member
Joined
Nov 15, 2021
Messages
2
Likes
1
Hi @amirm, thanks for the video! About the long-term memory. Years ago I had Asus Xonar soundcard with Cirrus Logic CS4398 dac chip in several setups. For the last 5 years I didn't used it at all and switched to ES9018/ES9038- based dacs. But a few weeks ago my friend invited me to check out his setup, the speakers and receiver were new to me, the dac was hidden somewhere, but after 1 minute I've got a very strong nostalgic feeling and I knew it was a signature of my old soundcard. I was sure there was Cirrus Logic dac somewhere in his setup and that was true! Do you know what kind of bug could give such a strong signature to those CS4398 dacs? IMD in some freq region, jutter? It's obviously not the frequency response, I believe it's extremely difficult to spoil it in a dac and opamp buffer, right?
I have exactly the same case.
Korg DACs with CS4398 give me 3D sound. I have not heard Xonar but tried other CS4398 based DACs. They all have that distinct pleasant signature, though only Korg gives me the proper 3D sound.
 

Yevhen

Active Member
Forum Donor
Joined
Dec 12, 2020
Messages
145
Likes
51
Location
Netherlands
I have exactly the same case.
Korg DACs with CS4398 give me 3D sound. I have not heard Xonar but tried other CS4398 based DACs. They all have that distinct pleasant signature, though only Korg gives me the proper 3D sound.
Finally, I'm not alone! Do you have any idea about the reason for this sound signature of the Cirrus Logic DACs? Even at Topping, they mention it:
1637519001920.png

 

Fokken_Ossom

New Member
Joined
Nov 15, 2021
Messages
2
Likes
1
Finally, I'm not alone! Do you have any idea about the reason for this sound signature of the Cirrus Logic DACs? Even at Topping, they mention it:
Not sure for the reasons but definitely some ancient witchcraft have taken place when the chip was designed.
 
Top Bottom