Audio Blind Tests and Listener Training (video)

[amirm]

Good write up. Worth noting that MP3 encoders improved in the years since 20 years ago, when those tests were done, so MP3 encoded today at the same bitrate would be harder to identify from source than it was in that write up.

Whether you can pass such a test highly depends on training and the content. This is my passing one such test at 320 kbps when challenged last by the late Arny Krueger:

foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/07/19 19:45:33

File A: C:\Users\Amir\Music\Arnys Filter Test\keys jangling 16 44.wav
File B: C:\Users\Amir\Music\Arnys Filter Test\keys jangling 16 44_01.mp3

19:45:33 : Test started.
19:46:21 : 01/01 50.0%
19:46:35 : 02/02 25.0%
19:46:49 : 02/03 50.0%
19:47:03 : 03/04 31.3%
19:47:13 : 04/05 18.8%
19:47:27 : 05/06 10.9%
19:47:38 : 06/07 6.3%
19:47:46 : 07/08 3.5%
19:48:01 : 08/09 2.0%
19:48:19 : 09/10 1.1%
19:48:31 : 10/11 0.6%
19:48:45 : 11/12 0.3%
19:48:58 : 12/13 0.2%
19:49:11 : 13/14 0.1%
19:49:28 : 14/15 0.0%
19:49:52 : 15/16 0.0%
19:49:56 : Test finished.

----------
Total: 15/16 (0.0%)

I took one of his files which had posted per above for another test and passed the ABX test. This way, I was not picking some special file that made the job easier for me.

Also, MP3 has been surpassed in audio terms. In fact MP3 is officially obsolete. Modern codecs like AAC at 128 or 256 kbps would be much harder to identify from uncompressed source music than MP3 was in that write up. Although nobody today would use constant bitrate at a medium-low bitrate like 128 kbps, so using variable bitrate would improve the audio quality (ie making it even more difficult to detect), if the target file size was the same as 128 kbps constant bitrate MP3 file sizes in 2000.

The original FHG reference encoder was actually quite good. A lot of people messed around with it but only made improvements at higher bit rates. At lower bit rates by relaxing the bandwidth limitations, they actually made things worse.

AAC is much, much better codec. No one who has access to one should be using MP3. MP3 was never meant to be transparent (at any bit rate) and hence the development of AAC to replace it. I was challenged to pass AAC test using VBR coding at 192 kbps and I passed that as well:

foo_abx 1.3.4 report
foobar2000 v1.3.2
2017/11/07 21:50:26

File A: C:\Users\Amir\Documents\Test Music\AAC test\01-01 Cadenza I (FLAC 96.0 kHz 24-bit)_02.flac
File B: C:\Users\Amir\Documents\Test Music\AAC test\01-01 Cadenza I (FLAC 96.0 kHz 24-bit)_02_01_192aac_01.wav

21:50:26 : Test started.
21:52:06 : 01/01 50.0%
21:52:24 : 02/02 25.0%
21:52:37 : 03/03 12.5%
21:52:57 : 04/04 6.3%
21:53:11 : 04/05 18.8%
21:53:22 : 05/06 10.9%
21:53:36 : 06/07 6.3%
21:53:48 : 07/08 3.5%
21:54:01 : 08/09 2.0%
21:54:22 : 09/10 1.1%
21:54:40 : 10/11 0.6%
21:55:14 : 11/12 0.3%
21:55:50 : Test finished.

----------
Total: 11/12 (0.3%)

The source was some high-res file for another challenge.

Anyway, the topic of this video was what a critical listener is and proof of the same in the form of controlled testing. If you think all of these formats sound the same, then you are not a critical listener.

 

[Deleted member 27694]

Amir do you think there is a significant difference between 24/16 under normal listening conditions? by normal I mean listening to music with headphones or speakers. Lets say you use your favorite headphones and use your favorite song (or 1min snippets) at normal or high volume? would you say that you would detect a difference and pass a test in a “normal” listening situation? Thank you!

 

[amirm]

I think your question needs expansion. How did you create the 16 bit and 24 bit files? Was the former created using the latter? If so, what was the conversion? Truncation? Dither? Noise shaping? And at what playback level.

If you mean it in a less precise manner, then no, I don't think I can because a lot of content has a lot of noise in it already. I am certainly quite happy listening to 16 bit content.

 

[Newman]

@amirm Analog or digital, today’s tech or tomorrow’s tech, 100 dB or 150 dB or 75 dB or 60 dB, my point stands. It wasn’t a test of a competent recording played at a level where the primary program is listenable.

It wasn’t make clear enough that you didn’t demonstrate whether you can distinguish the two in real listening scenarios.

 

[Deleted member 27694]

I think your question needs expansion. How did you create the 16 bit and 24 bit files? Was the former created using the latter? If so, what was the conversion? Truncation? Dither? Noise shaping? And at what playback level.

If you mean it in a less precise manner, then no, I don't think I can because a lot of content has a lot of noise in it already. I am certainly quite happy listening to 16 bit content.

Thank you! i can not specify i just don’t have that technical knowledge. But I know when YOU are happy listening to 16 bit then it is perfectly fine for me

 

[amirm]

It wasn’t make clear enough that you didn’t demonstrate whether you can distinguish the two in real listening scenarios.

It doesn't get more real than the way I passed the test. Every listener has a volume control for their headphone. If you know what to look for, have proper training, you can pass the test showing the format is not transparent. No trickery was used as you were suggesting with software manipulation.

Remember, there are people with better hearing than me. So if I can pass the test, others may be able to pass it with even less preconditions.

 

[amirm]

@amirm Analog or digital, today’s tech or tomorrow’s tech, 100 dB or 150 dB or 75 dB or 60 dB, my point stands. It wasn’t a test of a competent recording played at a level where the primary program is listenable.

No, your point doesn't stand. You modified the file if using software which is not allowed in controlled testing. And if you used analog amplification, you had to demonstrate that you could pass such listening test as I did. No pontification about 150 dB, noise-free amplification. I suggest you watch the video again as you have lost the plot here.

 

[Newman]

@amirm OK I have had another look at the video and I am happy to retract. You do make the statement (approx 18m50s timestamp) that, if you listen to the musical piece without cranking the volume of the trail-out, you would be hopeless at detecting a difference. But IMHO by the end, you had repeated so often that one can detect differences, (if one has the innate ability, plus well-trained, plus knowing where to focus one's listening, plus in this case cranking the volume in a near-silent passage), I felt that the earlier message, that it would be hopeless in real listening to musical pieces, was kind of drowned out on balance. It would be too easy, IMHO, for some audiophiles to watch your video and tell themselves "these things sound detectably different, so lesser tech is going to make my music sound lesser", even though that is obviously not your opinion.

You also mention (approx 13m timestamp) that Archimago's 16/24 bit test was wrong to ask for preference, and I get your point if the goal is to know if the difference is detectable, but it is also a suitable goal (dare I say a more relevant goal for the audiophile mainly focused on ultimate music-listening experiences) for Archimago to say that there was not a detectable preference when listening to actual music at 16 vs 24 bit.

I hope I am not misrepresenting you to say that this video is emphasising that it is wrong for some scribes to run around saying it is completely impossible to detect when something is 16 bit vs 24 bit, nth-generation re-encoding, strong jitter added, etc, but those things won't create preferred/less preferred music listening experiences.

cheers

 

[Deleted member 27694]

@amirm OK I have had another look at the video and I am happy to retract. You do make the statement (approx 18m50s timestamp) that, if you listen to the musical piece without cranking the volume of the trail-out, you would be hopeless at detecting a difference. But IMHO by the end, you had repeated so often that one can detect differences, (if one has the innate ability, plus well-trained, plus knowing where to focus one's listening, plus in this case cranking the volume in a near-silent passage), I felt that the earlier message, that it would be hopeless in real listening to musical pieces, was kind of drowned out on balance. It would be too easy, IMHO, for some audiophiles to watch your video and tell themselves "these things sound detectably different, so lesser tech is going to make my music sound lesser", even though that is obviously not your opinion.

You also mention (approx 13m timestamp) that Archimago's 16/24 bit test was wrong to ask for preference, and I get your point if the goal is to know if the difference is detectable, but it is also a suitable goal (dare I say a more relevant goal for the audiophile mainly focused on ultimate music-listening experiences) for Archimago to say that there was not a detectable preference when listening to actual music at 16 vs 24 bit.

I hope I am not misrepresenting you to say that this video is emphasising that it is wrong for some scribes to run around saying it is completely impossible to detect when something is 16 bit vs 24 bit, nth-generation re-encoding, strong jitter added, etc, but those things won't create preferred/less preferred music listening experiences.

cheers

I also watched it again. He did say that and he also said "I went to the end of the track and got lucky because there was a fade to zero" (19m36s)

@amirm I hope I am not misrepresenting you to say that this video is emphasizing that it is wrong for some scribes to run around saying it is completely impossible to detect when something is 16 bit vs 24 bit, nth-generation re-encoding, strong jitter added, etc, but those things won't create preferred/less preferred music listening experiences.[

That was also my point from the beginning. I understand that from an engineering standtpoint you thrive for absolute perfection. And the goal is transparency - under all circumstances. But I think transparency should be measured against "preferred/less preferred music listening experiences." So what I would suggest is we need a definition for "the perfectly clean apartment, done by the best cleaning service in the U.S." Yes, you can still argue it is not perfect and a guy who knows what to look for with a black-light and a microscope (volume knob) can proof it.

But I understand NOW that it was never Amirs point to say "Hey guys I am 40+ and even I can hear a difference between 24/16 so get rid of that crap an stop ruining your ears"

IMHO there is absolutely no difference for music enjoyment between 16/24 (and for me personally even AAC@HQ). I think because of the loudness war CD-quality got a bad rep and 24 catered for an "audiophile" audience an AFAIK it is not the difference between the bit depths that people hear, but rather the difference in the quality of the digital mastering. (better dynamic range) Because to hear the difference in dynamic range between a 16-bit and a 24-bit recording in a normal quiet listening room, you would have to go so loud it would cause permanent hearing loss. I think we can agree most people don`t want that.

 

[Fokken_Ossom]

Hi @amirm, thanks for the video! About the long-term memory. Years ago I had Asus Xonar soundcard with Cirrus Logic CS4398 dac chip in several setups. For the last 5 years I didn't used it at all and switched to ES9018/ES9038- based dacs. But a few weeks ago my friend invited me to check out his setup, the speakers and receiver were new to me, the dac was hidden somewhere, but after 1 minute I've got a very strong nostalgic feeling and I knew it was a signature of my old soundcard. I was sure there was Cirrus Logic dac somewhere in his setup and that was true! Do you know what kind of bug could give such a strong signature to those CS4398 dacs? IMD in some freq region, jutter? It's obviously not the frequency response, I believe it's extremely difficult to spoil it in a dac and opamp buffer, right?

I have exactly the same case.
Korg DACs with CS4398 give me 3D sound. I have not heard Xonar but tried other CS4398 based DACs. They all have that distinct pleasant signature, though only Korg gives me the proper 3D sound.

 

[Yevhen]

I have exactly the same case.
Korg DACs with CS4398 give me 3D sound. I have not heard Xonar but tried other CS4398 based DACs. They all have that distinct pleasant signature, though only Korg gives me the proper 3D sound.

Finally, I'm not alone! Do you have any idea about the reason for this sound signature of the Cirrus Logic DACs? Even at Topping, they mention it:

TOPPING D30 PRO CS43198*4 Chips XMOS XU208 DSD256 & PCM384kHZ Hi-Res Decoder

Features: 1. Preamplifier function: D30Pro has preamplifier function so you can control the output volume on the front panel and also at the remote control. Si

shenzhenaudio.com

 

[Fokken_Ossom]

Finally, I'm not alone! Do you have any idea about the reason for this sound signature of the Cirrus Logic DACs? Even at Topping, they mention it:

Not sure for the reasons but definitely some ancient witchcraft have taken place when the chip was designed.

 

[Fokken_Ossom]

Finally, I'm not alone! Do you have any idea about the reason for this sound signature of the Cirrus Logic DACs? Even at Topping, they mention it:
View attachment 167163

TOPPING D30 PRO CS43198*4 Chips XMOS XU208 DSD256 & PCM384kHZ Hi-Res Decoder

Features: 1. Preamplifier function: D30Pro has preamplifier function so you can control the output volume on the front panel and also at the remote control. Si

shenzhenaudio.com

I bought Topping D30. I tried tried it for 3 weeks.
Could not get the 3D sound.
It produces no noise, has excellent drivers, remote etc. Everything is perfect except no 3D sound.
Korg is less clean, has horrible drivers, no preamp output. But it produced that 3D sound even with the non hi res sources.
I could not understand why and how it is happening.

Then I have accidently found the answer.

I recorded a large bell sound with Focusrite Clarett 8PreX+ sE x1 mic. Simple loud sharp bang sound.
And played it via Pass Labs XA25+ Infinity Reference 11i with both DACs and the Focusrite interface.
I have noticed that D30 played it in a kind of "step ladder" style, that is the loudness of the sound was increasing in steps.
Korg D10R played it natural, no steps. Just sharp bang and smooth decay indistinguishable from the live bell.
Focusrite played it somewhere in between the Korg and D30.

Only Korg produced that 3D sound.

Cheaper D100m and D100 produced the 3D sound.

I then bought Korg NU-1. Everything was perfect but no 3D sound.

So weird.

 

[Larry B. Larabee]

How is I/V conversion implemented in these dacs. Is it in some cases a simple resistor, op amps or discrete class A gain stages? Obviously there is DA conversion and if the digital domain is audibly transparent aren't you just evaluating the analog part of the dac and if you are how is the additional resolution from the dac any better than a typical analog component. That is, how does old school analog in the sgnal chain allow someone to differentiate between, say, 16 and 24 bits.

 

[Fokken_Ossom]

How is I/V conversion implemented in these dacs. Is it in some cases a simple resistor, op amps or discrete class A gain stages? Obviously there is DA conversion and if the digital domain is audibly transparent aren't you just evaluating the analog part of the dac and if you are how is the additional resolution from the dac any better than a typical analog component. That is, how does old school analog in the sgnal chain allow someone to differentiate between, say, 16 and 24 bits.

Sorry, I wish I was technical enough to answer your question.
There some pictures of the insides here
It looks so basic.
I have noticed that when I use a non Pass Labs or Threshold power amp ( e..g. Marantz SM-10, SR7012, Yamaha AS-3200, A-3080 and many others) then the 3D feeling almost disappears, the sound feels flat, compressed, and unnatural.

I have been playing live instruments for many years and the bloody Korg plays the records exactly like a real instrument would.
CS4398 based Korgs (100m, 100, 10R, MR2000S) are not clean, they do ouput some noise (except MR2000S), but it is not affecting the 3D sound stage and naturality of the sound.
And just I don't understand how and why.
And most importantly I can't find a replacement cause Korg made me badly addicted to that 3D sound.
Somebody, help me please!

 

[TheLastGerman]

Sorry, I wish I was technical enough to answer your question.
There some pictures of the insides here
It looks so basic.
I have noticed that when I use a non Pass Labs or Threshold power amp ( e..g. Marantz SM-10, SR7012, Yamaha AS-3200, A-3080 and many others) then the 3D feeling almost disappears, the sound feels flat, compressed, and unnatural.

I have been playing live instruments for many years and the bloody Korg plays the records exactly like a real instrument would.
CS4398 based Korgs (100m, 100, 10R, MR2000S) are not clean, they do ouput some noise (except MR2000S), but it is not affecting the 3D sound stage and naturality of the sound.
And just I don't understand how and why.
And most importantly I can't find a replacement cause Korg made me badly addicted to that 3D sound.
Somebody, help me please!

Seems I'm late to the party, but ... Try the Marantz HD-DAC1. I had one and regret having sold it to partially raise the money for a RME ADI-2 DAC. The HD-DAC1 (with CS4398 of course) gives that 3D feeling we miss. Dead silent, "black" background, no noise at all. Volume Control is analog btw. At least for me it worked flawlessly with a Windows 10 PC, the provided driver is rock solid.