Let's develop an ASR inter-sample test procedure for DACs!

[JIW]

Right. I'm focusing mainly on the time domain because of my limited understanding of the topic, but it's important to keep in mind that any change in the shape of a wave form in the time domain also means a change in harmonic content in the frequency domain. So even though you might see clipped peaks in both the NOS and OS outputs, it doesn't mean the reconstructed waves share the same harmonic characteristics or sound.

If the deformed waveforms are identical, their frequency components will be identical as well since the latter is the Fourier transform of the former. Of course, an OS DAC will only clip intersample overs while a zero-order hold will clip any peaks between any adjacent samples. The waveforms are really not comparable except for adjacent samples near digital full scale.

 

[JIW]

Yes... But isn't the whole point that we want those ISOs to be reproduced?

Obviously a filtered NOS DAC would be able do it but the output voltage would have to go above 1, so that's not ideal. An OS DAC without headroom either clips the wave or overloads the filter so that's arguably worse in terms of accuracy.

The only option left is to have headroom before the oversampling stage.

Yes.

What kind of overload is worse depends on many things. An op-amp becoming unstable might be terrible as well. Of course, adequately lowering the signal going into a NOS DAC also prevents overload of the following analog components.

 

[NTTY]

If it did have an analog filter that would bandlimit the output to below half the sample rate, that would be producing the intersample overs in the analog domain and might overload the following components.

No intersample-overs in analog if no oversampling filter. Only potential clip of the analog output stage (which I did not encounter in that case). But we probably don't speak about the same thing.

The ringing from the filters is entirely a function of the filtering not whether it is done digitally or using analog components due to the Gibbs phenomenon.

That is correct, but not sure of the relevance here. Again, we might not be talking about the same thing.

The only way for a NOS DAC to avoid intersample overs is to do zero-order hold without analog filter. Then it does not produce any peaks accurately except for the Goldilocks peaks that fall on the sample values. But for the latter there are no intersample overs of course.

There's no intersample-overs when you don't oversample

I thought my previous examples were clear though. Maybe clearer in time domain?

Here below the 11025kHz test which produces up to +3dBFS peaks. First with the Yamaha CD-1 (no clipping because NOS, with analog filtering and output stage not saturated):

Same with the Denon DCD-900NE, which clips as we saw before from the FFT:

 

[tccalvin]

No intersample-overs in analog if no oversampling filter. Only potential clip of the analog output stage (which I did not encounter in that case). But we probably don't speak about the same thing.

There's no intersample-overs when you don't oversample

What you mean is there are no clipped inter-sample overs without oversampling.

ISOs are just the part of the wave that goes above the samples.

So analog inter-sample overs are certainly recoverable when we run the output of a NOS DAC into a steep analog low-pass filter.

EDIT:

Of course, adequately lowering the signal going into a NOS DAC also prevents overload of the following analog components.

An interesting thing I was thinking is that overload could also be prevented by lowering the signal between the converted output of a NOS DAC and the analog low-pass filter, and this process would still allow for ISOs to be recovered.

But that wouldn't be the case with an OS DAC, would it?

 

[JIW]

No intersample-overs in analog if no oversampling filter. Only potential clip of the analog output stage (which I did not encounter in that case). But we probably don't speak about the same thing.


That is correct, but not sure of the relevance here. Again, we might not be talking about the same thing.


There's no intersample-overs when you don't oversample

I thought my previous examples were clear though. Maybe clearer in time domain?

Here below the 11025kHz test which produces up to +3dBFS peaks. First with the Yamaha CD-1 (no clipping because NOS, with analog filtering and output stage not saturated):

View attachment 406782

Same with the Denon DCD-900NE, which clips as we saw before from the FFT:

View attachment 406784

For the same filter applied either digitally or using analog components, the resulting waveform will be identical. This is entirely mathematical. For the same change of the Fourier transform, the resulting waveform will be the same and the peaks will exceed sample values in both cases or in neither case. This is what will overload the analog stage if it does not have enough headroom the same way it will overload the oversampling filter it that lacks headroom.

Your examples are not relevant to the general point. A NOS DAC with a steep analog filter at fs/2 and no analog headroom would produce severe distortion with the signal in question and an OS DAC with 10*log10(2) = 3.0103 dB headroom would not produce any additional distortion. What you show is that Yamaha engineers left enough analog headroom in their NOS DAC but not that this can be expected from NOS DACs in general.

 

[JIW]

What you mean is there are no clipped inter-sample overs without oversampling.

ISOs are just the part of the wave that goes above the samples.

So analog inter-sample overs are certainly recoverable when we run the output of a NOS DAC into a steep analog low-pass filter.

EDIT:



An interesting thing I was thinking is that overload could also be prevented by lowering the signal between the converted output of a NOS DAC and the analog low-pass filter, and this process would still allow for ISOs to be recovered.

But that wouldn't be the case with an OS DAC, would it?

Yes, that would work in the NOS case but not in the OS case. For the OS DAC, the overload is in the digital domain. The distortion from overloading the oversampling filter is fed to the converter and will thus be in the analog signal.

 

[NTTY]

What you mean is there are no clipped inter-sample overs without oversampling.

Yep

ISOs are just the part of the wave that goes above the samples.

They are just additional samples in between the original ones that need to go above the “initial” full scale of recorded digital signal as a consequence of oversampling and filtering in digital domain.

So analog inter-sample overs are certainly recoverable when we run the output of a NOS DAC into a steep analog low-pass filter.

Apéro-time has come for me, and it could explain why I don’t understand what you mean here You seem to be confused between digital and analog domain and the transition between the two.
Again, no ISO in a true NOS DAC*, by definition.

But of course, an analog low pass filter at Fs/2 will allow the sine to go above two samples that would be at 0dBFS in digital domain, as I showed before.

And allow me to quote @John_Siau here:

We didn't have this problem until oversampled DACs were invented. At worst, the analog output stage of a non-oversampled DAC will flat-top clip the peak. Most early DACs actually had the headroom to reproduce the peaks without clipping. None had DSP filters that could overload and spread the clip event before and after the actual peak. Reconstruction filters were analog.

*I say “true NOS DAC” because some simulate NOS by oversampling and filtering much later on purpose (like 4xFs).

 

[JIW]

They are just additional samples in between the original ones that need to go above the “initial” full scale of recorded digital signal as a consequence of oversampling and filtering in digital domain.

If that is how you define intersample overs, we are not talking about the same things.

I am talking about peaks above 0 dBFS that would be produced by bandlimiting the signal to fs/2 with an ideal brickwall filter. Oversampling reveals some of them but not all nor at the correct value due to lower bandwidth and slope of the oversampling filters as well as the finite sampling rate of the oversampled signal. For the analog filter, it is voltage peaks exceeding the voltage from the DAC corresponding to 0 dBFS.

By NOS, I mean zero-order hold for the original sample values with possibly an analog low pass filter. In this case, ISOs are only possible due to the low pass filter removing high frequency content that gives the DAC's output its stair-step shape.

John Siau is talking about the overloading behaviour of sample rate converters producing non-harmonic components before and after the intersample overs not whether intersample overs existed before oversampling. In fact he says they did but in the worst case were flat top clipped by the analog output stage. Partilarly, the problem he talks about seems to be audibility of ISOs not existence. See his full post and what he replies to.

 

[tccalvin]

Yep

They are just additional samples in between the original ones that need to go above the “initial” full scale of recorded digital signal as a consequence of oversampling and filtering in digital domain.


Apéro-time has come for me, and it could explain why I don’t understand what you mean here You seem to be confused between digital and analog domain and the transition between the two.
Again, no ISO in a true NOS DAC*, by definition.

But of course, an analog low pass filter at Fs/2 will allow the sine to go above two samples that would be at 0dBFS in digital domain, as I showed before.

And allow me to quote @John_Siau here:

We didn't have this problem until oversampled DACs were invented. At worst, the analog output stage of a non-oversampled DAC will flat-top clip the peak. Most early DACs actually had the headroom to reproduce the peaks without clipping. None had DSP filters that could overload and spread the clip event before and after the actual peak. Reconstruction filters were analog.

*I say “true NOS DAC” because some simulate NOS by oversampling and filtering much later on purpose (like 4xFs).

I'm sure we're talking about the same thing. The graphs you posted confirm this. We're just not using the same definition of "inter-sample over".

I'm defining ISOs as every peak that goes over the sampling points (I believe this is the mathematical definition).

"ISO" could refer to the peak of the original wave captured by the samples, or to the peak of a reconstructed analog wave form, or even to the peak introduced by the digital samples created by an interpolator/digital oversampling filter. This is true of every peak that exceeds the sampled values, at 0dBFS or at any other value, both in the analog and digital domain. There's no need to have samples in between the original samples to have an inter-sample peak. It's there by definition if the samples don't coincide exactly with the peak of the sampled wave.

I believe what @John_Siau was saying in the excerpt you quoted is not that ISOs didn't exist before the advent of OS DACs, but that they weren't a problem since there was no way for a signal to clip in the digital realm with traditional NOS DACs.

 

[NTTY]

I'm sure we're talking about the same thing. The graphs you posted confirm this. We're just not using the same definition of "inter-sample over".

I'm defining ISOs as every peak that goes over the sampling points (I believe this is the mathematical definition).

It’s not the definition. Inter-samples means inter-samples: more samples between original samples created in digital domain by oversampling. It’s the initial discussion here. There’s no reason for an analog output to clip when asked to produce a wave that is between two full scale digital sample points. It is as bad engineering as mastering at 0dBFS.

"ISO" could refer to the peak of the original wave captured by the samples, or to the peak of a reconstructed analog wave form, or even to the peak introduced by the digital samples created by an interpolator/digital oversampling filter. This is true of every peak that exceeds the sampled values, at 0dBFS or at any other value, both in the analog and digital domain. There's no need to have samples in between the original samples to have an inter-sample peak. It's there by definition if the samples don't coincide exactly with the peak of the sampled wave.

Well then you’re asking for headroom in initial recording, which was the recommendation of Philips/Sony more than 40 years ago to prevent folks to reach 0dBFS and other folks not to have analog headroom out of a DAC. Things have changed since that time.
This discussion goes nowhere.

I believe what @John_Siau was saying in the excerpt you quoted is not that ISOs didn't exist before the advent of OS DACs, but that they weren't a problem since there was no way for a signal to clip in the digital realm with traditional NOS DACs.

Well no, that’s not what John says or implies. What is unclear in “We didn't have this problem until oversampled DACs were invented.” ?

Seriously.

 

[TunaBug]

Thank you! The explanation is quite helpful to me.

Now, this sounds terrible—and it is—but it’s only true if we leave the analog, “stair-stepped” output of the conversion unfiltered or only minimally filtered. This unfiltered or lightly filtered output is often what we find in cheaper NOS DACs, which some people say sound more “analog” . In order to get a smooth, accurate representation of the original analog signal, you need to add a steep analog low-pass filter at the end of the chain. Ideally, this filter will remove most of the “stair-stepping” effect. (See post #509)

I'm having an "Aha!" moment here. The sample-and-hold followed by low pass filter is known to me as a reader of ASR, but I don't think I had quite internalized what the filter is doing in a NOS DAC until now. Going back to your figure in post 509: It's clear that the second waveform is the output of a dumb sample-and-hold. Are you saying that the third waveform is actually what comes out of the low pass filter when it's given #2 as input? I may be abusing terms here, but from a layman's(*) standpoint it appears then that the filter's job isn't so much "low pass filter" as "reconstruction". To be clear, I'm ignoring the +/-1.0dBFS limits in that graph and assuming, for this little tangent in the conversation, that those limits do not exist. And yes, I get that a square wave is an infinite series of... and all that, it's just that I hadn't really internalized that the filter might be shaping the waveform; I kept looking for some high frequency noise that needed to be removed (ala SACD) and not finding said noise.

*I loved doing digital hardware and software, but after all the imaginary numbers in freshman level electronics class I decided I preferred software (or digital, although skipping analog wasn't an option to me). Career-wise that was a good move, but I've always regretted that it left me feeling like a spectator in any discussion of audio electronics.

 

[VientoB]

No intersample-overs in analog if no oversampling filter. Only potential clip of the analog output stage (which I did not encounter in that case). But we probably don't speak about the same thing.


That is correct, but not sure of the relevance here. Again, we might not be talking about the same thing.


There's no intersample-overs when you don't oversample

I thought my previous examples were clear though. Maybe clearer in time domain?

Here below the 11025kHz test which produces up to +3dBFS peaks. First with the Yamaha CD-1 (no clipping because NOS, with analog filtering and output stage not saturated):

View attachment 406782

Same with the Denon DCD-900NE, which clips as we saw before from the FFT:

View attachment 406784

For someone like me with little understanding of electronics etc its hard to understand how the Yamaha appears to have sample points which are being clipped by the oversampling Denon. There shouldn't be any actual sample points being clipped, surely?

 

[NTTY]

For someone like me with little understanding of electronics etc its hard to understand how the Yamaha appears to have sample points which are being clipped by the oversampling Denon. There shouldn't be any actual sample points being clipped, surely?

I think I get what you mean by “points”. It is because, for that measurement, I sampled at the input of my ADC at 192kHz. So you see the sampling “points” of the ADC in use for the measurement.

That being said, the Yamaha CD-1 does not clip because 1) it uses a non-oversampling DAC which 2) has the required headroom in its analog output stage to reproduce an analog signal that goes above two digital samples at max amplitude, as it should.

 

[pjug]

You'd be surprised how many CD players in the past used digital level control prior to the OS filters and D/As. Yamaha, Pioneer etc. Many of course used post D/A level controls in the form of remote motorized volume controls on the variable outputs.

Dealing with this issue playing CDs can also be handled by using a CD transport and sending the digital signal to something that can create headroom. For example WiiM devices can do this if you use the volume limit (this is applied before it does any other DSP). What they call 95% is -3dB. 90% is -6dB.

 

[tccalvin]

It’s not the definition. Inter-samples means inter-samples: more samples between original samples created in digital domain by oversampling. It’s the initial discussion here. There’s no reason for an analog output to clip when asked to produce a wave that is between two full scale digital sample points. It is as bad as mastering at 0dBFS.


Well then you’re asking for headroom in initial recording, which was the recommendation of Philips/Sony more than 40 years ago to prevent folks to reach 0dBFS and other folks not to have analog headroom out of a DAC. Things have changed since that time.
This discussion goes nowhere.

Well no, that’s not what John says or implies. What is unclear in “We didn't have this problem until oversampled DACs were invented.” ?

Seriously.

Man. We were doing so well. Why do we have to fall apart like this?

It's "inter-sample" overs, and it means overshoots between samples.

Samples between samples would be inter-sample samples.

In the context of OS DACs, there is some overlap between inter-sample samples and inter-sample overs, but they're not quite the same thing.
Digitally filtered inter-sample samples are a digital representation of an inter-sample peak. That is, until they hit the 0dBFS threshold, at which point they can no longer represent the original inter-sample peak accurately and they will instead represent a clipped inter-sample peak.

You're right in saying that for the purpose of this thread we can forget about all the inter-sample peaks that don't exceed 0dBFS, but it is important to know what those look like, because we are trying to make sure that the ISPs above 0dBFS behave just like (or mostly like) the ones that are at or below that threshold.

Given the required analog headroom voltage-wise, a NOS DAC followed by an analog low-pass filter will do just that (you showed this yourself in the last couple of graphs you posted). It will reconstruct an inter-sample peak above or at 0dBFS just like it reconstructs an ISP at -1dBFS, at -2dBFS and at every other value it's able to hold. An OS DAC won't be able to do this.

"We didn't have this problem until oversampling DACs were invented" means exactly what it sounds like: NOS DACs didn't introduce digital clipping because they weren't oversampling.

And yes, I get that a square wave is an infinite series of... and all that, it's just that I hadn't really internalized that the filter might be shaping the waveform; I kept looking for some high frequency noise that needed to be removed (ala SACD) and not finding said noise.

You're getting there. A square wave is an infinite series of odd sine waves decaying in amplitude at a fixed rate.

In the time domain, if you keep adding those sine waves on top of each other you'll get a square wave. By extension, if you keep filtering out all of those odd sine waves (harmonics) from a square wave, you’ll get a sine wave (fundamental).

One crucial thing to understand is that changing the shape of the wave in the time domain also changes the harmonic content in the frequency domain and viceversa.

I'm sure you're also familiar with the spectrum of a square wave in the frequency domain. You'll get a series of vertical lines getting smaller and smaller representing the odd harmonics.

But what wave would we get in the time domain if instead of having those odd harmonics in the frequency domain we had that ultrasonic, high frequency noise you mentioned (specifically around the sampling frequency)?

Yep, we'd get a Zero-Order-Hold train.

If you think about it, a ZOH train is not exactly a square wave. If it's not exactly a square wave in the time domain, it won't be exactly a square wave in the frequency domain either. Instead of having odd harmonics we'll have what are called "images" of the original signal at multiples of FS, which are exactly the high frequency noise you want to filter out.

EDIT: Typos. Minor additions for added clarity.

 

[JIW]

Man. We were going so well. Why do we have to fall apart like this?

It's "inter-sample" overs, and it means overshoots between samples.

Samples between samples would be inter-sample samples.

In the context of OS DACs, there is some overlap between inter-sample samples and inter-sample overs, but they're not quite the same thing.
Inter-sample samples are a digital representation of an inter-sample peak. That is, until they hit the 0dBFS threshold, at which point they can no longer represent the original inter-sample peak accurately and they will instead represent a clipped inter-sample peak.

You're right in saying that for the purpose of this thread we can forget about all the inter-sample peaks that don't exceed 0dBFS, but it is important to know what those look like, because we are trying to make sure that the ISPs above 0dBFS behave just like (or mostly like) the ones that are below that threshold.

Given the required analog headroom voltage-wise, a NOS DAC followed by an analog low-pass filter will do just that (you showed this yourself in the last couple of graphs you posted). It will reconstruct an inter-sample peak at 0dBFS just like it reconstructs an ISP at -1dBFS, at -2dBFS and at every other value it's able to hold. An OS DAC won't be able to do this.

"We didn't have this problem until oversampling DACs were invented" means exactly what it sounds like: NOS DACs didn't introduce digital clipping because they weren't oversampling.



You're getting there. A square wave is an infinite series of odd sine waves decaying in amplitude at a fixed rate.

In the time domain, if you keep adding those sine waves on top of each other you'll get a square wave.

One crucial thing to understand is that changing the shape of the wave in the time domain also changes the harmonic content in the frequency domain.

I'm sure you're also familiar with the spectrum of a square wave in the frequency domain. You'll get a series of vertical lines getting smaller and smaller representing the odd harmonics.

But what wave would we get in the time domain if instead of having those odd harmonics in the frequency domain we had that ultrasonic, high frequency noise you mentioned (specifically around the sampling frequency)?

Yep, we'd get a Zero-Order-Hold train.

If you think about it, a ZOH train is not exactly a square wave. If it's not exactly a square wave in the time domain, it won't be exactly a square wave in the frequency domain either. Instead of having odd harmonics we'll have what are called "images" of the original signal at multiples of FS, which are exactly the high frequency noise you want to filter out.

The images are subtracted from multiples of fs from odd multiples of fs/2 to even multiples of fs/2 and are added to multiples of fs from even multiples of fs/2 to odd multiples of fs/2. For a signal of frequency f, the images have frequencies of n*fs-f between (2n-1)*fs/2 and 2n*fs/2 and n*fs+f between 2n*fs/2 and (2n+1)*fs/2 for n at least 1.

 

[tccalvin]

For someone like me with little understanding of electronics etc its hard to understand how the Yamaha appears to have sample points which are being clipped by the oversampling Denon. There shouldn't be any actual sample points being clipped, surely?

You can disregard the sample points you see in the last couple of graphs depicting the time domain. Ideally those two waves would have been shown on an analog oscilloscope, but who has that? So the analog output of the DAC goes right back into an ADC. A necessary compromise.

 

[Scytales]

Just to be precise and to prevent potential confusion from the readers:

• The above is ISO testing, generating a very different type of distortion, in this case, in audio band.
• What you quoted from me is not related, and was only to mention the out of band noise and distortion generated by a non-filtered digital nor analogue DAC. So that is a different story.

Cheers

Ah ! Yes. I misquoted you : I wanted to quote your post above the one I cited !

 

[danadam]

it appears then that the filter's job isn't so much "low pass filter" as "reconstruction".

The intent is reconstruction; what it actually does is low-pass filtering. If we start with a sampled signal:

A zero-order-hold DAC could produce an analog signal like this:

This process creates mirror images >20 kHz. Filtering them out reconstructs the signal (with the roll-off inherent to ZOH):
(Ignoring how feasible it is to have such steep analog filter in the real world)

And the principle is the same with oversampling too. At the risk of repeating my other post (linked somewhere above), oversampling adds zero samples:

This process creates mirror images >20 kHz. Filtering them out reconstructs the samples, although at lower level:

which can be easily brought back to the original one:

I kept looking for some high frequency noise that needed to be removed (ala SACD) and not finding said noise.

They are mirror images that need to be removed. With single/separate tones like above, those images will be single "spikes". With more noise-like signal (i.e. music), those images will look more like noise.

 

[NTTY]

Thanks @danadam, can’t be clearer.

And here below an example IRL, with a square signal digitally created at 0dBFS without filtering. When played from a CD player, it overloads the digital filter to a point of defeating it and so the output shows the typical images of the conversion with sin(x)/x envelope:

And for those interested, more available info in two tutorials from AD:
- MT-002 : What the Nyquist Criterion Means to Your Sampled Data System Design
- MT-017 : Oversampling Interpolating DACs

And of course the Nielsen / Lund AES paper that was recommended to me by @AnalogSteph and from which I reused the test protocol to measure ISO resistance with his help.