Where does quantization noise come from.

[teapea]

OK - this may be due to lack of sleep or I'm just being incredibly stupid and missing the obvious - but I can't get my brain around where the noise comes from!

So this is an image from the famous Monty video - and basically everything he says makes perfect sense.

I appreciate the red line is the difference between the original and the quantized signal - but surely the DAC has no knowledge of the original signal?
In my head, the DAC is reading t1=4, t2=5, t3=6, t4=5, t5=2, t6=-1, ... and constructs the only continuous path to join the dots and creates the the yellow line.

So the analogue signal out the DAC is the yellow line. It doesn't know the green line at all - so where is the red line in the output!? In my head we just have 2 waveforms, the input and the output which are just very slightly different!

I think to think I'm pretty intelligent and have a pure maths degree, so I'm wondering why I'm finding this so hard to understand!!!

 

[staticV3]

Quantization noise happens when you try to quantize (=digitize) a waveform.

It happens because the original waveform may have a value of t1=4.43 but your digitizer is only capable of t1=4 or t1=5.

That error, happening thousands of times per second, results in quantization noise.

You don't have to have the original waveform to compare against, to objectively identify quantization noise as noise, because the noise is correlated to the input signal, and our brains are very good at picking out such noise.

We can mask that correlated noise by injecting uncorrelated noise, which is much easier for the brain to ignore.

An analogy would be aliasing artefacts in digital photography:


Super irritating, very identifiable even without a reference to compare to, and professionals get rid of it by injecting uncorrelated noise in form of an OLPF.

 

[teapea]

Nope - still not making the connection!

I understand the ADC quantizes from t1=4.43 to t1=4 - makes sense. The digital medium eg CD simply has this value. So the DAC then constructs the yellow line.
Where is the red line? Why do we get any noise from this? The yellow line is just another analogue signal?

If we take another (admittedly theoretical) example, where the input analogue signal just happens to not need any quantization. t1=4.0000, t2=5.0000 etc. Now the DAC reconstructs the exact same signal as before. So clearly no quantization, and can't be any noise.

But then isn't that what the yellow line in the original example is? A perfect signal with respect to not needing quantization to store it identically - and hence no quantization noise?

 

[solderdude]

The quantization error with 16 bits is already below any audible levels and is filtered out greatly by the reconstruction filter.
That is ... when one is using a proper one and not a 'slow' filter or 'NOS' filter.
It only starts to matter a bit when you do all digital volume control (so amp at max. gain) using a 16-bit resolution DAC.
Also the error can be 'masked' with dither thereby greatly improving the dynamic range for signals below -90dB which would otherwise be truncated if they would not have real world noise of mics and studio gear in the signal anyway which is bigger in amplitude than the quantization error anyway.

 

[teapea]

The quantization error with 16 bits is already below any audible levels and is filtered out greatly by the reconstruction filter.
That is ... when one is using a proper one and not a 'slow' filter or 'NOS' filter.
It only starts to matter a bit when you do all digital volume control (so amp at max. gain) using a 16-bit resolution DAC.
Also the error can be 'masked' with dither thereby greatly improving the dynamic range for signals below -90dB which would otherwise be truncated if they would not have real world noise of mics and studio gear in the signal anyway which is bigger in amplitude than the quantization error anyway.

Yup - appreciate all that. I'm not worried if it's audible or not, I'm talking theoretically here. I just don't understand why quantizing is creating "noise" when in my head it's just modifying the original signal. What is constructing the red noise line in Monty's chart? The only thing that knows about the quantization error is the ADC, so how does it ever appear in the DAC?

I did say there is a fundamental part of the process I'm not "getting"!

 

[staticV3]

I understand the ADC quantizes from t1=4.43 to t1=4 - makes sense. The digital medium eg CD simply has this value. So the DAC then constructs the yellow line.
Where is the red line? Why do we get any noise from this? The yellow line is just another analogue signal?

We don't get any noise from the DAC reconstructing the yellow line.

Instead, the yellow line already contains quantization noise due to a lack of dither during its creation - the DAC just reproduces the faulty signal.

If we take another (admittedly theoretical) example, where the input analogue signal just happens to not need any quantization. t1=4.0000, t2=5.0000 etc. Now the DAC reconstructs the exact same signal as before. So clearly no quantization, and can't be any noise.

Correct.
In the real world however, an analog audio signal that can be quantized perfectly is a statistical impossibility, and so we get quantization noise. Unless we apply dither.

But then isn't that what the yellow line in the original example is? A perfect signal with respect to not needing quantization to store it identically - and hence no quantization noise?

The red line is a perfect signal.
The yellow line is its poorly quantized counterpart, containing quantization noise.

 

[Multicore]

The question as asked in the thread title is interesting. Quantization residuals are fairly straightforward to explain. But why do we call a sequence of them "noise"? and where did that signal come from?

If I know your quantization scheme I can arrange an input signal so that one string quartet is optimally encoded and a different string quartet emerges in the quantization residuals.

 

[staticV3]

In other words, here's a poorly quantized picture of a suit (=yellow line):


It can be displayed, projected, or printed out without introducing any additional noise. A perfect reproduction.

But that doesn't make the picture any less flawed.

 

[teapea]

Instead, the yellow line already contains quantization noise ...

OK - I think this is the fundamental thing my head isn't getting...

How does the yellow line already contain the noise - isn't the noise the red line?

 

[staticV3]

OK - I think this is the fundamental thing my head isn't getting...

How does the yellow line already contain the noise - isn't the noise the red line?

The red line is what remains when you subtract the flawed quantized signal (=yellow) from the original signal (=green).

It's a representation of the noise that was added to the yellow line during to the quantization process.

 

[jhenderson0107]

Even if the original signal in green were noise-free, any real-world ADC used to digitize it would produce samples which imperfectly represent it. As StaticV3 said, the act of quantizing the analog signal into a fixed value is performed with finite resolution, resulting in lost information. This is the quantization "noise". Moreover, the conversion process within the ADC is imperfect and subject to variations from sample/hold circuitry limitations, voltage supply dependencies, thermal effects and parasitics. All of these conspire to create minute measurement variations (a combination of quantization and electrical noise) spread across samples which are characterized in ADC performance testing.

 

[JeremyFife]

My head hurts a little bit

What I'm taking from this is that there is only the yellow signal (quantized).
It is only when we compare that (externally to the audio process) to the original input (green) that we can measure the difference (red, noise). What we are calling 'noise' is a measurement artefact and also a way of describing why the output (yellow) differs from the input (green).

I think!

 

[antcollinet]

Forget the lines for a moment.

Noise is simply any unwanted signal added to your wanted signal. Take mains hum. If you have a 1Khz sine wave with 50hz hum, you will see the sine wave with a low level 50hz signal along it.

So with quantisation noise. At the sample time, the actual signal is at a level half way between two available digital numbers. The ADC picks one of the numbers which results in an error. That error is a change in the signal at that point - directly equivalent to a noise signal. It appears as noise - and if big enough to hear (low enough sample resolution) sounds like noise.

 

[Ulf]

Dunno if it helps, but the quantization process (with or without dithering) is nonlinear and the error (noise) therefore can have a much broader spectrum than the signal. StaticV3's suit picture is maybe quantized, but the biggest error in it looks like aliasing and not quantization noise.

And btw, it's true that we don't know the quantization noise at DAC stage, but give some information about the quantization scheme and some assumption of the input signal we can know the *spectrum* of the quantization noise.

 

[teapea]

OK I think the penny is dropping (albeit slowly!) here.
And it's actually to do with how I envisage "noise" to look on a signal.

To me, the yellow and green signal both look "clean" - they're both a nicely curved signal. One hits all the unquantized sample points, the other hits all the quantized sample points. But neither looks "noisy". I think I'm (wrongly of course) expecting to see "noise" in the signal representation somehow - and the fact that the noise has been represented by a totally different line makes it look like there isn't noise in the yellow signal but it's separate somehow.

 

[staticV3]

StaticV3's suit picture is maybe quantized, but the biggest error in it looks like aliasing and not quantization noise.

True. A more appropriate representation of quantization noise in video would be color banding.

 

[DVDdoug]

My take is - "Quantization noise is what low-resolution sounds like."

If you convert a file to 8-bits in Audacity you can hear it. (If you try it, make sure to configure Audacity for no dither.)

It's like a "fuzz" on top of the signal. It's similar to regular analog noise in-that it's most noticeable with quiet sounds, but unlike analog noise it goes-away completely with "dead digital silence" (which is minus infinity dB).

If you have 16-bit audio and the amplitude falls to -48dB (during a fade-out, etc.) you are only using 8-bits, but the quantization noise stays below audibility... You don't hear the loss of quality/resolution.... Unless you amplify that quiet part (by +48dB or so), then you can hear it. (With 24-bit audio, you've still got 16-bits of resolution at -48dB.)

 

[teapea]

OK I think the penny is dropping (albeit slowly!) here.
And it's actually to do with how I envisage "noise" to look on a signal.

To me, the yellow and green signal both look "clean" - they're both a nicely curved signal. One hits all the unquantized sample points, the other hits all the quantized sample points. But neither looks "noisy". I think I'm (wrongly of course) expecting to see "noise" in the signal representation somehow - and the fact that the noise has been represented by a totally different line makes it look like there isn't noise in the yellow signal but it's separate somehow.

Which I guess then begs the question, how do we know if the signal contains noise or is in fact what it was supposed to sound like in the first place?!

 

[teapea]

In other words, here's a poorly quantized picture of a suit (=yellow line):
View attachment 364680

It can be displayed, projected, or printed out without introducing any additional noise. A perfect reproduction.

But that doesn't make the picture any less flawed.

Yup - I think the thing is here, we know what the picture should look like, we don't actually know what the analogue signal "should" look like. (Or at least I don't look at amplitude/time line and read it like the matrix and know what it sounds like!)

Thanks @staticV3 really appreciate your time trying to help me

 

[renkitch]

The noise and the quantized signal are not drawn 'ideally' - they would have square type steps, not a smooth wave. This may cause confusion when trying to explain it. The noise is the difference between the input and the output. You can hear the noise clearly if it was 8-bit at an 8 kHz sample rate, but only if you know it's 8-bit... with a 16-bit 44 kHz DAC, would you know if it was conversion noise, or noise in the primary recording?