Let's develop an ASR inter-sample test procedure for DACs!

[kemmler3D]

To recap if I understand the discussion correctly:

• @splitting_ears suggests we start testing the output of DACs with intersample overs as part of the standard measurements
• Many members jump in saying this is a dumb idea for various reasons:
  • Masters shouldn't have intersample overs in the first place, therefore we should not test DACs for how they handle them. (Roads shouldn't have bumps so don't test the shocks on a car?)
  • They're not audible, so we shouldn't test DACs for how they handle them. (still under debate apparently)
  • They're rare, so we shouldn't test DACs for how they handle them. (not THAT rare, I think?)
• Amir responds saying he could do the test, but chooses not to because he doesn't want to encourage DAC makers to cut headroom in order to pass the intersample over test.

I think Amir's reason for not performing or publishing the test is the only one I understand in the context of ASR. We eagerly read reviews comparing -98 to -112dB SINAD but yet another test of inaudible technical flaws is dumb and bad for some reason?

Anyway, I wonder if it would be helpful and not hurtful to the industry if Amir performed some of the tests and published a one-off survey of category-wide performance but didn't name names? This way we could give guidance to the industry without creating a perverse incentive for each individual model of DAC to 'game the system' for this test. If we could find some link between audibility and intersample overs, then ASR could publish an industry report and guideline without creating a "hoop" to jump through.

It would be something similar to how Backblaze publishes SSD reliability reports, but doesn't say which models are which. It's interesting and somewhat useful to the consumer but doesn't make manufacturers scurry around doing weird things to their products.

 

[amirm]

I don’t get why the hand is trembling that much. Manufacturers used to be afraid by you, but now the situation seems to be have been inverted. No offense or lack of respect here. I’m not understanding your last two sentence..

Trembling? There must be some communication issue as I am telling you the opposite. I am saying you that we have a lot of influence in the market. That if I start testing for this, it will result in a lot of products snapping to it. With this influence, comes responsibility and ability to justify change.

SNR seems to have become an irrational golden end goal making people blind to other important issues.

SNR is a far more rational goal than what is being discussed here. I routinely get email from people, including one last week, that folks are hearing noise at their speakers.

 

[amirm]

Yes you’re right: let not music equipment « ruin » audio material by adding unwanted (and avoidable) distortion that’s not here in the first place.

Tell me again if this is an important issue for you, you are not using a DAC with volume control to solve it.

 

[mdsimon2]

BTW, I have had the Benchmark test for 2 to 3 years now. The reason I have not deployed it is above. Us forcing it will mean all DACs will eventually lose 3 dB of dynamic range or whatever this needs to be. So best think about what you are asking. There are plenty of tracks I avoid because fidelity is so poor. I don't mind having tracks with audible issues due to clipping added to that list.

If you are talking about the +3 dBFS 11.025 kHz test signal, why not only run it for DAC's with volume control and at a volume control setting of -4 dB? Seems to me what you want to know is if the DAC volume control will solve the issue or not.

Every DAC (except for the miniDSP 2x4HD) I've run the +3 dBFS 11.025 kHz test signal on is fine when using the DAC volume control and none show nasty clipping but rather all show an elevated noise floor without attenuation. The miniDSP 2x4HD shows nasty clipping because it has an ASRC upstream of the volume control (at least in the current firmware, they are working on fixing this). It seems like Benchmark also has an issue here as they use an ASRC but obviously have built-in headroom (at least in the DAC2 and DAC3). Would be good to weed out those few DACs using ASRC upstream of volume control without thinking about headroom.

Michael

 

[NTK]

• Many members jump in saying this is a dumb idea for various reasons:
  • Masters shouldn't have intersample overs in the first place, therefore we should not test DACs for how they handle them. (Roads shouldn't have bumps so don't test the shocks on a car?)
  • They're not audible, so we shouldn't test DACs for how they handle them. (still under debate apparently)
  • They're rare, so we shouldn't test DACs for how they handle them. (not THAT rare, I think?)
• Amir responds saying he could do the test, but chooses not to because he doesn't want to encourage DAC makers to cut headroom in order to pass the intersample over test.

Bad analogy. A better one would be finding the most comfortable fabric to make body bags so that the corpses will feel more comfortable in them

Intersample overs means the analog signals (original or intended for synthesized signals) is over full scale. Here are some illustrations of the often used fs/4 sine wave to demonstrate ISO's. Here we have our samples taken at the perfect timing. The reconstruction matches the original. Woo-hoo!
[Edit] Since there are some confusion (see post #108) that I was presenting "fake news", I have replaced my original plots with new ones that highlight the zones that are over full scale. The digitized samples clip at +/-1.

Shift the sampling time a little bit, the reconstructed wave has a different (reduced) magnitude from the original. Note that this waveform is "ideally reconstructed" as it is done mathematically with no headroom limit of intersample overs.

With a sampling timing shift 1/2 of a sampling period (1/8 of the period of the sine wave) from the 1st example, the reconstructed wave is a full 3 dB lower than the original. So, exactly the same input waveform, exactly the same method of reconstruction, just a little shift in the start time of the A/D sampling process and you get different results.

[Edit] Add an animated version to illustrate the effect of A/D sampling timing shift.

With real life signals (e.g. frequency ≠ fs/4 or a "nice even faction" of fs), the differences will be more prominent and will also include harmonic distortions. So for ISO's, we are talking about choosing which of the different reconstructions of signals irreversibly damaged at the source, with none of them being anything like the original.

Tell me this is not a tempest in a teapot.

 

[thecheapseats]

Tell me again if this is an important issue for you, you are not using a DAC with volume control to solve it.

^^^that!... boom...

 

[cyril_meysson]

If the solution is to reduce dynamic range, it means that you disadvantage huge amount of music that doesn't have this problem in order to deal with small amount of content. In other words, if the headroom is permanently built into the DAC, then you lose that dynamic range for all uses, not just in the case of intersample overs. This is one of the reason that the DACs that have this feature are not at the top of our chart.

The right solution is to have an option in the music player to perform the dynamic range reduction on a per track basis. An online database could be built to instruct the player to do this automatically.

Thanks for joining the conversation Amir, it's much appreciated.

I understand your point very well. However, I don't see how it makes sense to treat the least significant bit with the same 'care' as the most significant bit. In the S/N ratio, what I care about (and what I hear as a listener) is the S part. Especially in a DAC. Also, the distortion caused by ISPs is spread over the entire audible range. I think that's a valid concern?
And by the way, this '1-bit pad' could be user selectable. Personally I would certainly trade 3-6dB of increased noise (well below the threshold of hearing) for reduced distortion.

The database is a nice idea, but the problem is that it has to take into account the differences between lossy files (which can overshoot even more) and lossless files. Probably a better idea would be to do the ITU-R BS-1770-4 true-peak calculations locally, on a per-track basis, to dynamically adjust the output gain.

And what about CD players? They may not have a volume control, and they're designed with 16-bit dynamic range in mind, so the dynamic range argument doesn't apply here. Wouldn't it be great to at least encourage a little more headroom with these?

 

[KSTR]

Intersample overs means the analog signals (original or intended for synthesized signals) is over full scale. Here are some illustrations of the often used fs/4 sine wave to demonstrate ISO's. Here we have our samples taken at the perfect timing. The reconstruction matches the original. Woo-hoo!
View attachment 322599

Shift the sampling time a little bit, the reconstructed wave has a different (reduced) magnitude from the original. Note that this waveform is "ideally reconstructed" as it is done mathematically with no headroom limit of intersample overs.
View attachment 322600

With a sampling timing shift 1/2 of a sampling period (1/8 of the period of the sine wave) from the 1st example, the reconstructed wave is a full 3 dB lower than the original. So, exactly the same input waveform, exactly the same method of reconstruction, just a little shift in the start time of the A/D sampling process and you get different results.
View attachment 322601

With real life signals (e.g. frequency ≠ fs/4 or a "nice even faction" of fs), the differences will be more prominent and will also include harmonic distortions. So for ISO's, we are talking about choosing which of the different reconstructions of signals irreversibly damaged at the source, with none of them being anything like the original.

Tell me this is not a tempest in a teapot.

[^Bold mine]

This is not correct. You changed sample time, ok. But you also changed input level (but have not updated the "Original" trace) so that the largest samples hit 0dBFS and guess what, then obviously the level of the reconstructed wave is different by exactly that amount.
Very embarrasing move IMHO, presenting fake news, not helping the reputation of the site.

 

[cyril_meysson]

Some time ago I did ISO measurements on the DAC bundled with a Google Pixel 3, a Moshi DAC, and a cheap chinese VIMVIP DAC. Some of these measurements show waveform distortion that is significantly worse than what would be caused by hard clipping alone. At the highest TP level listed above, all devices I tested exhibit extreme non-linear distortion all over the spectrum that is not even harmonically related to the original 12 kHz tone - a phenomenon that cannot be explained by clipping alone.

Thanks for linking these threads and SoX commands, it's extremely useful and informative!

At least it shows that it's possible to interpret ISP tests in a constructive and meaningful way, especially in the context of a review. I'm also going to look more into the distortion you measured, because if it can't be accurately simulated by hard-clipping, then it means we need to think differently about how to set up a blind test.

 

[KSTR]

I'm also going to look more into the distortion you measured, because if it can't be accurately simulated by hard-clipping, then it means we need to think differently about how to set up a blind test.

Those erratic patterns, quite similar to the wraparound I once observed, can be modelled as well.
And SRC is always a problem. RME have introduced a -6dB prescaling in the ADI-2 Pro's to catch all but the most extreme IS-overs.

 

[cyril_meysson]

Those erratic patterns, quite similar to the wraparound I once observed, can be modelled as well.
And SRC is always a problem. RME have introduced a -6dB prescaling in the ADI-2 Pro's to catch all but most most extreme IS-overs.

Ah, please go on! How would you model those erratic patterns?

That's one thing that bothered me about your first proposed test. Applying 6dB of headroom at the beginning (which is exactly what I'm advocating, headroom where it's needed) would mean that the upsampling stage would be 'clean' and that only one non-linearity would occur at a very high sample rate. But if there are two stages, then we know for sure that they won't add, but multiply, which takes the problem up an order of magnitude.

And by the way, thanks for your contribution to the discussion. I really appreciate your help.

 

[Lambda]

Quite sad how this forum ore some user are reluctant to adopt new tests and metrics that would be able to differentiate DACs.
Is it copium because you can’t boil down performance to a single number?

 

[pLudio]

The database is a nice idea, but the problem is that it has to take into account the differences between lossy files (which can overshoot even more) and lossless files. Probably a better idea would be to do the ITU-R BS-1770-4 true-peak calculations locally, on a per-track basis, to dynamically adjust the output gain.

Sounds like you're asking for ReplayGain scanning with loudgain.

 

[Sokel]

Sounds like you're asking for ReplayGain scanning with loudgain.

I knew there would be a foobar plug-in for this!

 

[cyril_meysson]

Tell me again if this is an important issue for you, you are not using a DAC with volume control to solve it.

Yes, but it still puts the onus on the end user to be aware of the ISPs (this discussion shows that this is *not* an intuitive concept) and then act accordingly. We want thing to be foolproof, isn't it?

When you say that ISPs are only a problem for 0.001% of the content out there, allow me to strongly disagree. In fact, I think the opposite is much, much closer to the truth. At least for music released after the 90s and older, badly remastered music. I certainly don't want to turn the industry upside down, only help people to get a better sound. In fact, I think writing an ASR article on ISPs to help people understand the issue and solve it on their end would also be beneficial. I would happily contribute to it.

I've given your arguments some thought, and I can see why you wouldn't want SOTA products to implement this headroom. While I disagree, I can understand the logic.
It's also fair to say that most digitally controlled DACs are likely to be operated below their maximum output capabilities on a daily basis. I still think we should test them all the way up (and remember that a 11025Hz sine wave is well, well below the Nyquist frequency, so hardly an 'illegal' signal)... but I can understand the logic here too.

Anyway, how about this, in the spirit of compromise:
Why not, as a first step, specifically test ISP behaviour on devices with no digital volume control, or no volume control at all?

In other words, CD players, stand-alone DACs, digital interfaces, or DACs with analogue volume controls. For these, the problem is neither intuitive nor easy to solve by the user. I think almost everyone uses their music player at maximum volume so it's a valid concern.

Not only would it be informative, but it would also expose manufacturers who make dubious claims about the superiority of analogue volume controls (and you know there are many), who do not follow the logical consequence of providing enough headroom for decoding. And it wouldn't do any harm to SOTA, digitally controlled devices.

 

[cyril_meysson]

Sounds like you're asking for ReplayGain scanning with loudgain.

Ah, thanks, I completely forgot ReplayGain was still a thing! I actually used it years ago, but I'm glad to see it's still being actively developed.
As you can see from the discussion, I'm not in favour of just pushing users to solve the problem themselves, but it's nice to have another solution at hand. And it shows that it's not entirely unreasonable to think about it.

If I (or we?) ever get around to writing an article about ISPs, I'll be sure to mention this!

 

[NTK]

[^Bold mine]

This is not correct. You changed sample time, ok. But you also changed input level (but have not updated the "Original" trace) so that the largest samples hit 0dBFS and guess what, then obviously the level of the reconstructed wave is different by exactly that amount.
Very embarrasing move IMHO, presenting fake news, not helping the reputation of the site.

The maximum level of the samples can only be 1. That's the whole point. Only the samples in the first plot didn't "catch" the clippings due to their timing. All the others clipped. Obviously if the original signal didn't go over 1, then there is no problem. The sampling theorem says after all the samples contain enough information for perfect reconstruction.

[Edit] If the original signal doesn't go over/under +/- 1, then we won't have intersample overs. If they do, what is the likelihood that we are so lucky that all the samples are taken when the original signal is within 1, as in the 1st plot? The plots in my post showed what will happen if we are not so lucky. Think also what would happen with real life signals (e.g. a 120 Hz tone that has >360 samples per period when sampled at 44.1 kHz).

 

[cyril_meysson]

Tell me this is not a tempest in a teapot.

Sorry, but your examples only show how the ISP test signals work, i.e. generate an fs/4 sine wave at 0dBFS and apply a 45° phase shift.
This is a perfectly legal signal to begin with, which sample values are encoded between -1 and 1. The problem is that the true, underlying waveform will peak 3dBFS higher than its PCM sample values. Exactly why the ITU-R BS 1770-4 specification exists, in order to clear up that fundamental metering issue that plagued the production/mixing/mastering stages for decades.

Only the samples in the first plot didn't "catch" the clippings due to their timing.

There's no clipping to catch. That sine wave is perfectly clean. The clipping only occurs when decoded with insufficient headroom.

Think also what would happen with real life signals

I'm sorry, but I think you misunderstand the problem. We're not talking about digitisation (where there can be no ISP because the system is band-limited by the anti-aliasing filters), we're talking about reconstructing already existing signals with sample points below 0dBFS at all times, but with real, interpolated values higher than that.
Furthermore, synthetized signals and digital processing are as real and valid as they can be.

 

[NTK]

Sorry, but your examples only show how the ISP test signals work, i.e. generate an fs/4 sine wave at 0dBFS and apply a 45° phase shift.
This is a perfectly legal signal to begin with, which sample values are encoded between -1 and 1. The problem is that the true, underlying waveform will peak 3dBFS higher than its PCM sample values. Exactly why the ITU-R BS 1770-4 specification exists, in order to clear up that fundamental metering issue that plagued the production/mixing/mastering stages for decades.

Just exactly what is the probability of encountering that particular signal in real life? You've got a number? As an engineer, I am perfectly happy with solutions that works almost everywhere.

There's no clipping to catch. That sine wave is perfectly clean. The clipping only occurs when decoded with insufficient headroom.

No. The clipping occur during the digitization.

I'm sorry, but I think you misunderstand the problem. We're not talking about digitisation (where there can be no ISP because the system is band-limited by the anti-aliasing filters), we're talking about reconstructing already existing signals with sample points below 0dBFS at all times, but with real, interpolated values higher than that.
Furthermore, synthetized signals and digital processing are as real and valid as they can be.

You are the one that doesn't understand. The "intersample overs demo" signal is fs/4, well within the Nyquist limit.

There is even less excuse with synthesized signals, since the maximum level is known in advance (before converting to digitized samples).

 

[popej]

Probably almost all current DACs do internal up-sampling. I think up-sampling ISP can lead to 3 results: true peak within headroom, hard clipping to 0dBFS or computing error like overflow. Do exist DACs, that belong to the last category?