Let's develop an ASR inter-sample test procedure for DACs!

[KSTR]

Here we go. I did 2 passes. First pass scoring 9/10, taking time to get familiar with the tracks; then a second pass scoring 10/10, during which I was faster.

In this instance, it was all about the transient. The reference track has more energy at that peak you're showing in your graph, compared to the clipped track. Subtle but audible.

Great, a first data point that suggests audibility. Thanks for taking the time. May I ask the age of your pair of ears? Mine are closing in on 60 and have seen some abuse, no output above 13kHz.
Inspired by your result I just took a try with fresh ears this morning... but no way.

 

[melowman]

I’m 7 bruh! Just kidding, I’m in my forties.

 

[bennetng]

In the attachment there are 44.1 kHz and 48 kHz files with Fs/4 tones which:

• start at -0.5 dBTP,
• in the last 0.1 of each second ramp up by 0.1 dB
• until they reach +2.9 dBTP.

(there are also 4 pulses at the beginning and at the end: for aligning, to "wake-up" the DAC, etc)

View attachment 323287
View attachment 323288

Below are spectrograms of outputs of a few things I have lying around:

• DragonFly Red
• Fiio K3 (USB-C) with and without gain
• Cowon Plenue D
• Samsung dongle
• Speaka (AFAIK it's a clone of DragonFly Black)
• Lenovo T480s

The 44.1 kHz file was played on Linux through ALSA using aplay command and the outputs were captured with ADI-2 Pro FS BE:

• 192 kHz sampling rate,
• ref level +13 dBu,
• different Trim Gain settings to have at least 3.5 dB of headroom when playing calibrating 220 Hz tone (3 dB for true peaks, 0.5 dB as a spare)

For spectrograms I downsampled the captured files to 96 kHz and used only the left channel, to save some vertical space. I also aligned them so that during the 10-th second the signal should be at 0 dBTP [0].

[0] For those more observant among you: yes, if the signal starts at -0.5 dBTP then it should reach 0 dBTP during the 9-th second. Well, I miscalculated a bit and in the file that I used the signal was from -0.6 dBTP to +2.8 dBTP ¯\_(ツ)_/¯

View attachment 323289View attachment 323290View attachment 323291View attachment 323292View attachment 323293View attachment 323294View attachment 323295

For ADI-2 itself I only could do a loopback recording at 44.1 kHz sampling rate. I used:

• Main out at +2 dB, ref level +4 dBu
• Phones out at 0 dB, low power setting

For phones it looks at first like it doesn't clip, so I also added spectrograms with zoomed dB scale (for both main and phones):

View attachment 323296View attachment 323297View attachment 323298View attachment 323299

And now I'll go listen to my volume-normalized music and contemplate the point of this exercise

Could you test your devices with the file below as well?

https://stash.reaper.fm/32679/intersample.flac

 

[Adis]

Could you test your devices with the file below as well?

https://stash.reaper.fm/32679/intersample.flac

My Windows is running at 48kHz, when I play the track in MPC there's nothing wrong I can hear. In Foobar (which switches system SR to 44.1 kHz automatically), it goes bonkers at about 9th second.

 

[bennetng]

My Windows is running at 48kHz, when I play the track in MPC there's nothing wrong I can hear. In Foobar (which switches system SR to 44.1 kHz automatically), it goes bonkers at about 9th second.

Thanks for trying this, likely the effect of CAudioLimiter and therefore MPC is undistorted. But anyway, I should post full instructions so that others can use this test signal safely:

Audible intersample clipping test signal - Cockos Incorporated Forums

Audible intersample clipping test signal REAPER General Discussion Forum

forum.cockos.com

 

[melowman]

I can try to provide something publicly shareable, and my own ABX test results. (Don't hold your breath for it to happen right now, though. )

Meanwhile, I'm trying to cook up the same test but with a more challenging track (at least I think), a track that is already pretty crushed and full of clipping. It should be more difficult (or maybe easier, who knows in advance..).

As promised, I did something challenging: ABXing ISO clipping in an already-crushed track with lots of clipping already baked in. To challenge me even more, I took a snippet where I didn't get more than around 1dB of ISO clipping. In the end, the difference was really subtle, but after a few minutes of training, I felt confident to take the ABX test, and I scored 9/10.
There were mainly two focus points when listening: the clipped version felt duller (not because of a lack of high end, but because of some transients being 'weaker' so to speak), and it had some artifacts that are not present in the reference track. Subtle, but even in that case, audible.

Here are the files (ASR won't allow the upload, file is too big) -- I don't like the song, but I thought it was a good candidate. Link to audio files
And my ABX test results are attached below.

 

[bennetng]

The right solution is to have an option in the music player to perform the dynamic range reduction

Yes, I always do this on the playback software (foobar), especially, because I also play lossy files and MIDI files via some synth plugins which involve floating point processing and the rendered peaks can go beyond 0dBFS (not dBTP), therefore headroom in DAC is not going to help.

If at some point you start to measure intersample overs and if I am seeing excessive headroom which is not part of the DAC chips' built-in behaviour (like some of the AKM and Wolfson chips mentioned by others), I'd tend to specifically avoid these products because I prefer to have full control of the digital volume myself.

On the other hand, I suggested adding DAC passband ripple measurements years ago instead of simply using white noise, I do appreciate the advice is being taken, so thanks.

 

[KSTR]

On the other hand, I suggested adding DAC passband ripple measurements years ago instead of simply using white noise

Yes, notably because linear phase passband ripple is often misunderstood in its effects. It's not the tiny level wiggles of 0.0x dB that might be audible, it's the pre-echo in the impulse response which tends to be invisible in the usual impulse response plots with their linear scale.

 

[danadam]

First, some update to the previous post, because I recorded ADI-2 main out with EQ still on. Here's with EQ disabled:

Could you test your devices with the file below as well?

https://stash.reaper.fm/32679/intersample.flac

I've added a scale with true peaks based on 8x sox upsampling. Strangely only FiiO and Speaka show results similar to the previous test, while the rest reach higher before starting to clip.


For ADI-2 nothing hides in the background, so no zooms this time:

 

[bennetng]

Thanks. My test signal is asymmetrical, if the device allows changing the filter's phase it may show a different result.

Anyway, the Samsung dongle result is the strangest one.

 

[bennetng]

Today, most properly designed DACs are well above the human hearing threshold, but sometimes fail to provide adequate protection for a necessary by-product of PCM encoding. Only by publishing objective data and reviews can we encourage more manufacturers to solve this problem. ASR has changed many things for the better in the hi-fi/pro audio world, so I'm sure this would be a great next challenge to tackle.

OK, I see why you posted here, but this Gearspace post is not particularly interesting. Could you go back and tell Alexey that a 2-pass AAC/MP3 enocoding from lossless source is not necessary, because there are something like MP3Gain/AACGain?

How to normalize many mp3 songs to for example -10 LUFS - Cockos Incorporated Forums

How to normalize many mp3 songs to for example -10 LUFS REAPER General Discussion Forum

forum.cockos.com

As for Ian's worry about decoder clipping, this thread should be relevant:

For those who worry about quality of software volume control

Using Audacity, the visual effect of -60dB on a music waveform. Of course, a visual comparison isn't very useful, since you're only using a couple of hundred pixels for the vertical scale (so, about 8-bit resolution). Since you've attenuated by 60dB, you've dropped the bottom 10 bits. You...

www.audiosciencereview.com

I'd recommend trying the software and the tests files, as well as reading all posts (only a few pages, not as long as the currently 19 pages Gearspace thread) in the links above, and report the results to your Gearspace friends. Headroom in DAC is a non-solution to these issues, if anything, all the issues are on the software side. Streaming services are business, not charity, they have the responsibilty to make the content encode and decode properly.

 

[splitting_ears]

Could you go back and tell Alexey that a 2-pass AAC/MP3 enocoding from lossless source is not necessary, because there are something like MP3Gain/AACGain?

Sorry but I don't get your point. The post I linked to (from Paul Frindle by the way, not Alexey...) mentioned how manufacturers can be reluctant to implement headroom at different stages. That's all.

I'd recommend trying the software and the tests files, as well as reading all posts (only a few pages, not as long as the currently 19 pages Gearspace thread) in the links above, and report the results to your Gearspace friends. Headroom in DAC is a non-solution to these issues, if anything, all the issues are on the software side.

And where in this thread has anyone suggested that turning down the volume degrades quality? Even I mentioned in the first post of this thread that reducing the volume practically solves the problem in most (but not all) cases

Streaming services are business, not charity, they have the responsibilty to make the content encode and decode properly.

I agree with this. I'd like to see streaming services 1) scan the peak headroom of the files submitted, 2) reduce the gain if necessary, 3) encode to whatever lossy format they want/need. But that's not how things work in the real world. They just encode what the aggregators give them, even if it's a known fact that some codecs sound worse with too little headroom.

 

[bennetng]

That's why I am saying the post from Paul Frindle you linked to is not particularly interesting. The thread started from Ian, Alexey and others about the confusion between ISP and codec clipping. Codec clipping cannot be solved on the DAC's side, and when codec clipping can be solved, ISP on lossless sources can be solved on the playback software as well, all the issues are on the software side.

As long as you are sending fixed-point data to the lossy encoders, all encoded 0dBFS+ peaks can be reduced during decoding on the playback software. If the playback software cannot do this, it is broken, and when the DAC receives these broken decoded PCM data, headroom in DAC is not going to help, therefore headroom in DAC is a non-solution.

The test files I posted can be used to check if the playback software is capable of playback unclipped 0dBFS+ data or not. I have several freeware capable of doing this. Yes, freeware in the real world, and if commercial playback clients fail to do so, it is their problem.

 

[splitting_ears]

As long as you are sending fixed-point data to the lossy encoders, all encoded 0dBFS+ peaks can be reduced during decoding on the playback software. If the playback software cannot do this, it is broken, and when the DAC receives these broken decoded PCM data, headroom in DAC is not going to help, therefore headroom in DAC is a non-solution.

And what about this?
If you're going to answer 'turn down the volume first', how about a CD player connected to this unit via S/PDIF?

II think it's only sensible to provide headroom where it's needed. Be it in the ASRC chips or anywhere else. I'm just asking for linear and predictable behaviour -but on the other end of where we usually test it, the most significant one.

 

[bennetng]

Yes I already read these posts before, in fact, I even posted in another thread:

Help explain intersample overs, please?

One thing I don't understand is why everyone mentions Benchmark's illustrations when talking about ISP. I mentioned it in 2006 here for example: https://forums.dearhoney.idv.tw/viewtopic.php?t=52943&start=18 Yes, I mentioned the "0dBFS+ Levels in Digital Mastering" paper (at 2000) as well...

www.audiosciencereview.com

Help explain intersample overs, please?

One thing I don't understand is why everyone mentions Benchmark's illustrations when talking about ISP. I mentioned it in 2006 here for example: https://forums.dearhoney.idv.tw/viewtopic.php?t=52943&start=18 Yes, I mentioned the "0dBFS+ Levels in Digital Mastering" paper (at 2000) as well...

www.audiosciencereview.com

I read in Gearspace that you think that posting in ASR can potentially result in some sort of big changes due to ASR's popularity, but turns out the replies somehow disappointed you.

How about this? You find some of your Gearspace friends to measure ISP on the classes of devices you mentioned (e.g. CD players), and report your findings here? If you've collected sufficient amount of data, and presented in an organized way, your posts can draw some attention (e.g. being put on the front page).

Measuring ISP does not require expensive hardware like Audio Precision, and even within ASR, several members also publish their measurements periodically, you and your friends can do these measurements as well.

 

[splitting_ears]

I read in Gearspace that you think that posting in ASR can potentially result in some sort of big changes due to ASR's popularity, but turns out the replies somehow disappointed you.

How about this? You find some of your Gearspace friends to measure ISP on the classes of devices you mentioned (e.g. CD players), and report your findings here? If you've collected sufficient amount of data, and presented in an organized way, your posts can draw some attention (e.g. being put on the front page).

Measuring ISP does not require expensive hardware like Audio Precision, and even within ASR, several members also publish their measurements periodically, you and your friends can do these measurements as well.

Ok, I've tried to ignore the snarky tone of your previous messages, but let's set a few things straight:

Yes, I'm a Gearspace user and member too. Is that a problem? Should I disclose this critical information before any interaction?

Yes, I have mentioned this thread on Gearspace in two posts, here and here.
- The first is a summary of my original post here, in case anyone else wanted to contribute to the conversation too. I had an idea and I wanted people to discuss it (which might include telling me it wasn't a good idea).
- In the second, I summarised some of the replies I had received since the discussion started, because I thought they were informative. If you sense a tone of disappointment in my post, could you please explain how and why that is?

I started a thread here because I enjoy reading ASR, and because I thought the topic might be of interest to this community too. It's as simple as that.
And if you want to join the discussion on Gearspace, you're welcome to.

Now could we get back to the original discussion please?

 

[bennetng]

OK, I apologize if some of my expressions made you unhappy, probably a language barrier. However, instead of reading a single post from Gearspace, I decided to read more into the main thread to understand the flow of messages from other Gearspace members, not only you, so it is nothing personal.

So I already made a suggestion that you are welcomed to post a detailed report after doing the aforementioned research with your friends.

 

[splitting_ears]

OK, I apologize if some of my expressions made you unhappy, probably a language barrier. However, instead of reading a single post from Gearspace, I decided to read more into the main thread to understand the flow of messages from other Gearspace members, not only you, so it is nothing personal.

So I already made a suggestion that you are welcomed to post a detailed report after doing the aforementioned research with your friends.

Thank you, I appreciate it. On the other hand, I'm not a native English speaker, so I may misunderstand the tone and intent of some messages too.

To be honest, I'm not sure if my colleagues on Gearspace will start this research. Even I have barely had time to try ABXing the test files that were recently shared here. I'll try to catch up!

 

[splitting_ears]

Ok, here are some ABX results with the first test from post #217.

I did a little training first, then 2 sets of 10 trials, a little break and then 2 more sets. Here are my results:

1. 7/10
2. 10/10
3. 10/10
4. 7/10

I chose to identify the reference file, which was easier for me due to the increased 'transient energy' as @melowman also explained. I couldn't use Foobar (I can't manage to play mono files at 176kHz SR with the ASIO driver...) so I used HOFA 4U+ BlindTest in my DAW instead.

... and just to be on the safe side, I did another set just before posting this message. Got 10/10 again.

I'm going to try the other test as soon as I can.

 

[spigot]

https://www.audiosciencereview.com/forum/index.php?threads/are-we-measuring-enough-parameters.56699/
"I was reading an interview with John Siau, chief designer at Benchmark and he was asked if he had ever learned something from a subjective test that was not captured by measurements. Here is his answer:

"A few years back I was evaluating an asynchronous sample rate conversion chip (ASRC) and I noticed a difference when I switched the chip in and out in a listening test. The music sounded brighter when the chip was inserted. The measurements showed a ruler-flat frequency response with THD+N well below the threshold of hearing in our playback system. Additional measurements confirmed that there was no change in the transient response. Something was happening, but we could not see it in the measurements. Eventually, we were able to prove that the audible difference was caused by the clipping of intersample peaks that exceeded 0 dBFS. The DSP in the ASRC was clipping intersample peaks and this added the impression of brightness. We now have a test signal that we can use to measure this defect which can occur whenever a digital source is upsampled. Starting with the DAC2, all of our sigma-delta D/A converters have an “extra” 3 dB of DSP headroom in order to accommodate the highest possible intersample peaks. The elimination of intersample clipping is a gamechanger, but most of our competitors still seem to be ignoring this problem."