Quantization Noise 101: Where does SNR about 6N dB come from?

[RandomEar]

Thanks for the pdf file. The equations are a bit too low res on the forum post. We really need a build in LaTeX editor on the forum

There is a MathJax plugin for XenForo. Seems to be unmaintained, but may still be worth a try.

 

[DonH56]

Thanks for this. We are lucky to have you participating in this forum.

Thank you!

Here is a link where you can copy paste a lot of math symbols:

Math Symbols List (+,-,x,/,=,...)

List of all math symbols and meaning - equality, inequality, parentheses, plus, minus, times, division, power, square root, percent, per mille,...

www.rapidtables.com

You, Amir, and several others as well as some searching did lead me to a plethora of online ways to insert symbols into a web page. The problem is that I generate my articles, including the text with embedded equations and symbols, pictures, and all the various plots (using Excel, LTSpice, Mathcad, Matlab, Python, etc.) offline using Word (sometimes PowerPoint). I can edit and update as needed then upload, and it makes it easy(ier) to correct and update later whilst keeping a local copy. I have been through a couple of times a forum was wiped and I had to regenerate content, not fun (and not here, knock on wood). I do not generate the original content online, so copy and paste symbols from an online source does not work. I did try copying symbols from a couple of places, pasting into my Word file, then copying that here -- it did not work. I also tried saving as RTF and HTML and pasting from that, still NADA. As Amir said you'd need special tools plus a few more hoops to jump. I will probably dig into it again in the future.

On the contrary, I think this is a really good way of explaining quantization error. You know what they say, a picture speaks a thousand integrals

Although the images are very very good at explaining what quantization error is, they are a tiny bit inaccurate if I am not mistaken (not that you claimed they were 100% accurate). For example the sinusoidal that confirms to the samples around sample 40 would go over 1, and if the device can not go above 1, it would clip, if it can, the noise will be higher I think, no?. And around sample 120, the curve that passes through all the samples would have a high frequency component to it, and the quantization noise would be higher again, I think. Am I incorrect?


View attachment 300743

Bear in mind the examples don't have to worry about intersample overs so staying below full-scale means no clipping. Clipping would actually start about 1/2-lsb above or below full-scale. Finally, clipping generates distortion, not quantization noise per se, a different thing not really related to quantization noise.

As an aside, much of my career dealt with pulse waveforms where 6N did not necessarily apply, thus deriving the SNR got more "interesting". That is also why my notes go back to mathematical fundamentals, painful as it may be.

 

[DonH56]

Thanks for the pdf file. The equations are a bit too low res on the forum post. We really need a build in LaTeX editor on the forum

OK, I liked your post for the kind words, but honestly after having to deal with it for various papers in the primordial past, am not a huge LaTeX fan either. That said, some years back a friend introduced me to some pretty powerful LaTeX generators and editors that really streamlined the process.

 

[Multicore]

Thanks for the pdf file. The equations are a bit too low res on the forum post. We really need a build in LaTeX editor on the forum

MathML is quite well supported in browsers. Never used it myself.

 

[IAtaman]

Bear in mind the examples don't have to worry about intersample overs so staying below full-scale means no clipping. Clipping would actually start about 1/2-lsb above or below full-scale. Finally, clipping generates distortion, not quantization noise per se, a different thing not really related to quantization noise.

True. I didn't make that remark to point out potential clipping issues in the visual example, I just wanted to touch upon the fact that band limited DA convertion is better thought of as "fitting unique sinusoidals to samples" rather than "conversion of levels to voltages" if we were to stick to the precision of the math as you pointed out, and forgetting that (which is something I do more often than not I have to admit), creates the wrong mental image of how the process works, and the nature of the quantization noise.

 

[Multicore]

Underlying this discussion is the abstraction that we refer to by the word noise. Sometimes noise is a thing we can point to in the real word. Other times it's a well-known body of theory about random signals. And sometimes we make simplifying assumptions about theoretical or real systems by calling something noise so that we can use the theory of noise. If we are careful these can all be very enlightening investigations.

But underneath it is something that I find deeply mysterious and tremendously interesting for decades: randomness. My final year EE project in 86 was using random perturbations to the weights of adaptive FIR filters for communication channel equalizers. A year or so later working in industry I had to use pseudo random numbers in computer simulations. I was amazed to discover just a few years ago that a number of world renown PRNG experts were bragging about their algorithms using only statistical tests, which doesn't reveal all that we might want to know, e.g. it doesn't reveal entropy loss.

 

[Trell]

All my other derivations are longer and uglier, sorry... It took me long enough to type in this one, not used to equations in Word, then got blindsided because I couldn't copy and paste it here without losing the symbols.

LaTeX is your best friend.

 

[peng]

This post attempts to explain where the SNR = 6N dB approximation comes from. ASR does not have the symbol font available, and after an hour or so of trying to fix things, I ended up just pasting the whole thing as an image. Sorry about that! I did attach a PDF of the original.

View attachment 300660
View attachment 300652

Thank you Don for explaining the math. I basic question is, why is it necessary to express SNR in "bits" gain? What would be the issue for ASR members if we just use for example 96 dB, 102 dB? Is it just a preference, such as THD+N in % vs in -dB? Sorry if answered before, but I don't recall Amir explained this in his video, but I could have missed one or two.

 

[DonH56]

Thank you Don for explaining the math. I basic question is, why is it necessary to express SNR in "bits" gain? What would be the issue for ASR members if we just use for example 96 dB, 102 dB? Is it just a preference, such as THD+N in % vs in -dB? Sorry if answered before, but I don't recall Amir explained this in his video, but I could have missed one or two.

This is about quantization noise, which means the signal is quantized, i.e. converted to digital numbers, thus "bits" for dynamic range are correct. For a DAC (or ADC) it makes sense to express SNR in bits to clarify the measured performance of the DAC (ADC, data converter). The IEEE Standards actually use effective number of bits (ENOB), which is based upon SINAD (signal to noise and distortion ratio, which is equivalent to THD+N in many cases).

For an amplifier that does not include an ADC or DAC, expressing SNR in bits does not apply. However, these days most folk are familiar with bits (resolution), so it makes sense to provide a reference with which they are familiar. So, yeah, preference.

 

[Kaameelis]

How quantization noise figures are changed with dither and oversampling?

 

[DonH56]

How quantization noise figures are changed with dither and oversampling?

That is well beyond the scope of this article; those are complex subjects. Quantization noise itself is not affected by dither or oversampling in the sense of this article, which directly relates the error caused by quantizing a signal to the SNR. I suggest you look at some of the other articles, including the one on delta-sigma converters, for more insight. The quick answer for most situations is that dither reduces SNR and oversampling increases (improves) SNR. Here are answers (still hand-waving) with a little more detail:

In general dither decorrelates fixed samples to "spread" the noise, reducing fixed spurs (tones). When you quantize a signal, you create a bunch of little spurs (noise) related to the signal and sampling frequency. Combing all those spurs (in root-square-sum fashion, not a linear addition) to get the sum total of their energy and comparing that to the signal is how you determine SNR. A fixed signal sampled at a fixed rate may result in the same points sampled over and over, causing certain spur frequencies to be emphasized so they are higher. Adding noise (dither) moves the signal (or sampling instance) around so each sample is a little different. The fixed spurs go down, but because dither adds noise, SNR also goes down (is worse). You get a "cleaner" or "smoother" but higher noise floor for the same signal level.

Oversampling itself does not reduce noise, just spreads it over a wider bandwidth. But if you keep the same bandwidth by filtering away the "extra" bandwidth, then you gain SNR by throwing away out-of-band noise. If you double the sampling rate but keep the same signal bandwidth, you gain about 3 dB in SNR (from quantization noise), and will gain another 3 dB each time you double the sampling rate relative to the signal bandwidth. So a perfect converter at 44 kS/s having 100 dB SNR over 20 kHz bandwidth should be able to achieve 103 dB if you double the rate to 88 kS/s but maintain (through filtering) the same 20 kHz signal bandwidth.

Delta-sigma converters use oversampling and noise shaping in a design that "pushes" quantization noise above (in frequency) the signal band. That reduces the in-band noise and increases the noise at very high frequencies (above the audio band). That enables much greater noise reduction for each doubling in sampling rate. There is an introduction to delta-sigma converters in one of the technical articles linked in my signature.

HTH - Don

 

[danadam]

Although the images are very very good at explaining what quantization error is, they are a tiny bit inaccurate if I am not mistaken (not that you claimed they were 100% accurate).

FWIW all that is plotted there is taken from actual wav files, although admittedly with some shenanigans SoX doesn't want to create linear PCM with less than 8 bits, so the signal I generated was at about -30 dBFS, so that top bits are 0. Then I took the lower, non-zero bits and scaled them to [-1, 1] range for the graph.

Technically it is inaccurate, because it has 4 levels above 0 and this is neither 3 bits (3 levels) nor 4 bits (7 levels). Also my intention for X-axis was to count the samples 1 through 40, but it turned out it counts the samples from the continous waveform, which is the input upsampled 8x.

Here's more like I wanted them, 3-bits and 4-bits examples:

(still inaccurate, because I promptly ignored one additional level on the negative side )

But I don't think those have any impact on the main point.

And regarding the main point, because I don't think I spelled it out clear enough. When:

• you treat quantization as mixing the original signal with some additional signal
• and you use dither, which makes this additional signal a white noise

then it's easy to see that the only effect quantization/reducing bitdepth has on a signal is increasing noise floor. That's the point that some people try to put across, sometimes with some exasperation, whenever other people start discussing bitdepth.

For example the sinusoidal that confirms to the samples around sample 40 would go over 1, and if the device can not go above 1, it would clip, if it can, the noise will be higher I think, no?. And around sample 120, the curve that passes through all the samples would have a high frequency component to it, and the quantization noise would be higher again, I think. Am I incorrect?

If you mean intersample overs, then yes, I guess it's a possibility when your input hits at or near 0 dBFS. When I generate 32-bits sine, reduce it to 16-bits and then upsample both 8x:

sox -r44100 -n -b32 sin1.wav synth 10 sin 1k vol .98856218
sox -D sin1.wav -b16 tmp.wav
sox tmp.wav -b32 sin2.wav

sox sin1.wav -b32 sin1.up.wav rate 352800
sox sin2.wav -b32 sin2.up.wav rate 352800

then the max samples are:

   sin1.wav - 0x7e890000 2'122'907'648
   sin2.wav - 0x7e890000 2'122'907'648
sin1.up.wav - 0x7e89349b 2'122'921'115
sin2.up.wav - 0x7e895c91 2'122'931'345

The one that was reduced has a slightly higher level. That's one more reason to leave some headroom.

 

[Kaameelis]

..The quick answer for most situations is that dither reduces SNR and oversampling increases (improves) SNR. Here are answers (still hand-waving) with a little more detail:

Can dither be added to oversampled signal and then also pushed out of usable bandwidth with filtering? Or then it is no more called dither but noise shaping?

 

[xnor]

I thought undithered quantization produces distortion, not noise.

It does. Quantization noise that is correlated with the signal will produce distortion. Adding random noise before quantization (= dither) will decorrelate the quantization error.

 

[DonH56]

Can dither be added to oversampled signal and then also pushed out of usable bandwidth with filtering? Or then it is no more called dither but noise shaping?

Dither and noise shaping are two different things. Added dither increases the noise but reduces correlated spurs (from distortion or the sampling process). Noise shaping will act upon dither just like any other noise, reducing it in the passband and increasing it at higher frequencies. Dither will still slightly increase the in-band noise.

This thread shows how noise shaping works in a delta-sigma converter: https://www.audiosciencereview.com/...igma-delta-digital-audio-converters-dac.1928/

 

[DonH56]

I thought undithered quantization produces distortion, not noise.

Quantization errors described here (not discussing DNL, INL, and so forth) are treated as noise since the signal is in general uncorrelated with the sampling rate allow us to treat quantization errors as noise. The SNR calculation relies on that, and numerous measurements over the decades have provided empirical evidence supporting that analysis. The relation of specific quantization spurs to the signal and sampling rate is more complex, using various Bessel functions and fairly complicated integrals, and leads to the spurious-free dynamic range (SFDR) approaching 9N for N bits (also measured and verified empirically).

Dither, or noise decorrelation as I learned it and defined by the IEEE, randomly varies the sampling points relative to the signal, decorrelating the quantization spurs, which can reduce correlated spurs while raising the noise floor. I have written articles about dither but am not sure I discuss it here on ASR; my work was mostly not for audio applications.

Again, dither, noise shaping, and other data converter subjects are beyond the scope of this article and best topics of another thread(s).

 

[MusicNBeer]

For an intuitive description... Each bit of resolution adds ~6.02 dB SNR. 6.02 dB is 20*log10(2). 20*log10 is how signal magnitude is defined in dB. Each bit of added resolution doubles the possible quantization levels. Thus, on average, the quantization error is halved for each bit added.

 

[MRC01]

This post attempts to explain where the SNR = 6N dB approximation comes from. ASR does not have the symbol font available, and after an hour or so of trying to fix things, I ended up just pasting the whole thing as an image. Sorry about that! I did attach a PDF of the original.
...

As a mathematician, a different explanation seems more intuitive to me. This is just a different way of thinking about the same thing.

Each additional bit doubles the number of discrete amplitude values you can have. However many bits you already have, if you add 1 more bit, then you get all the prior combinations you had with the new bit set to 0, and again with the new bit set to 1.

In dB, dynamic range is 20 * log(R) where R is the ratio of two amplitudes. 20 * log(2) = 6.02, so each additional bit adds about 6 dB of dynamic range. Consequently, dynamic range is about 6*N when you have N bits.

 

[DonH56]

As a mathematician, a different explanation seems more intuitive to me. This is just a different way of thinking about the same thing.

Each additional bit doubles the number of discrete amplitude values you can have. However many bits you already have, if you add 1 more bit, then you get all the prior combinations you had with the new bit set to 0, and again with the new bit set to 1. This means each additional bit doubles the dynamic range.

In dB, amplitude is 20 * log(R) where R is the ratio of two amplitudes. 20 * log(2) = 6.02, so each additional bit adds about 6 dB of dynamic range. Consequently, dynamic range is about 6*N when you have N bits.

I treat dynamic range as SFDR which goes as 9N, perhaps just semantics... To me the math helps answer "why" 6 dB and why it is treated as noise, as well as explains the small discrepancy (offset) in the equation (why it is not exactly 6N dB).

I have a handful of derivations and used the one I knew best and felt easiest to explain at a fairly low level. Certainly there are other ways and no doubt you and others can explain it better, one reason I hesitated to put the time into this. I am an engineer, not a mathematician or physicist (I got the physicist's version working with JJ converters and qubits, mostly over my head).

 

[MRC01]

I treat dynamic range as SFDR which goes as 9N, perhaps just semantics... To me the math helps answer "why" 6 dB and why it is treated as noise, as well as explains the small discrepancy (offset) in the equation (why it is not exactly 6N dB).

I have a handful of derivations and used the one I knew best and felt easiest to explain at a fairly low level. Certainly there are other ways and no doubt you and others can explain it better, one reason I hesitated to put the time into this. I am an engineer, not a mathematician or physicist (I got the physicist's version working with JJ converters and qubits, mostly over my head).

Your explanation, while more complex, has the advantage of explaining the extra +1.761 dB factor.