Master Thread: Are Measurements Everything or Nothing?

[rdenney]

But people have conducted tests with good-measuring but maybe not as fully wrought as Dan’s pro gear by re-digitizing through a DAC/ADC loop for 10 or 20 generations and not been able to detect any difference at all between the first and the tenth. I think good-measuring DACs and ADCs are highly unlikely to be the cause of any audible effects these days.

When we hear something our measurements don’t detect, there are two possibilities: one is that the measurements or wrong or incomplete. But the other is that the ears and brains that interpret them are inconsistent and unrepeatable because of uncontrolled biases and perceptual inaccuracies. Because those human effects have shown up time and time again in test after test, you really have to control for them before really knowing which cause pertains.

Rick “effects confidently observed seem to vanish in controlled testing” Denney

 

[Hapo]

Hello Hapo

What is it that you are not clear about?

I have to rely on a lot of measurements, and with many years of design I developed a lot of confidence that I am doing it right. But only an arrogant fool will forgo feedback from people that use the gear. (I will not forget, it was an ear person in the 70’s that made me aware of phase linearity issues). Sending gear to be tested in the field (in different environments) can solve a lot of headaches.

But my main point is that following the theory (as precisely as possible) yields good accurate conversion. The “digital sound” is the same as the analog, but with a different convenient format. You convert it back and the result is the SAME. Designing an AD is about precision an accuracy, which is an EE territory. I designed the Savitr when I was 72-75 years old with less than 10KHz hearing bandwidth. For me it was about electronic design. Needless to say, the ears had to confirm the measurements and the operation of the unit.

My early AD designs were not for audio. Testing the early MRI converters (100KHz max) began with feeding the AD an ultra linear analog ramp (constant current into a capacitor). The output was a “staircase”. A 14-bit AD has 16384 steps, but we focused the scope on one small section, 16 steps at a time (using the trigger threshold and feeding the LSB's to through a 4 bit DA ).

The goal was clear and visible. The step size (vertical) and time (horizontal) should be the same. It is called differential linearity. If the step size (vertical) changes over the whole range it is called integral non linearity (for audio it is the cause of most distortions).

It is not all that different for audio. We have better test tools that are more suitable for audio testing. It is a different language, but the cause of “audio distortions” and “integral non linearity” for MRI, instrumentation or a weighing scale are circuit design issues, aiming at accurate conversion involving voltage, current, electronic parts. Measurements are the tool; the ears are the “security guard rails”.

...the issue is not your coherence, but mine...thank you for posting this...

...I find coherence to be a transient state...

 

[Dan Lavry]

re-digitizing through a DAC/ADC loop for 10 or 20 generations and not been able to detect any difference at all between the first and the tenth.

The outcome of running ADC/DAC loop many times does not yield what people expect, and can be very misleading:

Lets take a simple example. Say each unit (AD and DA) has a noise floor level at -100dB.

The noise is mostly random, uncorrelated. So each path through the converters generate “different noise” across the frequency range. The combined noise does increase with each pass, but not linearly. Running 10 passes (20 units) raises the noise from -100dB each unit to -87dB. Even 100 loop passes (200 units) yields -76dB.

The more important point is that after the first path, the distortion of 10 or many more loop runs will be virtualy the same as that of a single AD/DA run.

Say the input for the first path is a perfect full scale analog 1KHz tone, but the converters first path generate -60dBFS harmonic distortion. This is the “new signal” for the next loop run. The converter sees a 1KHz full scale and a -60dBFS distortion. It is going to “apply” its distortion mechanism on such signal. So the 1KHz tone is going to add that -60dB distortion (as it did before). But what happens to the previous distortion? It was at -60dB, the converter will generate it at -120dB (60dB below -60dB distortion). The "distortion of the distortion" is too small, below the noise floor to matter at all.

I saw that “argument” years ago. It was used by a company to promote their stuff. The converters noise goes up some but they sound the same after many loops, even if they suck. The fact that they sound the same after 10 path does not mean that they don’t distort.

 

[Dan Lavry]

The outcome of running ADC/DAC loop many times does not yield what people expect, and can be very misleading:

My post was very simplistic. To be accurate, the distortions may pile up some but slowly, at times they may even cancel out, it is a bit complicated to explain.

The point is that the expectation of running 100 loops is to have unrecognizable audio, and the surprise that it is not as bad as expected. The ways signals and various noise combine is not a simple addition. Add to that the ear near logarithmic scale... The difference between 100 passes and 200 passes may be only 3dB.

Units should be tested separately. An AD – DA loop’s output THD+N results will be dictated by the weaker performer. Such test can hide a high performing unit, and there is no information about which of the unit is the limiting factor.

 

[Sernyl]

+1
Re-read Toole's book (just an example)

 

[Sal1950]

For me it was about electronic design. Needless to say, the ears had to confirm the measurements and the operation of the unit.

Was there ever any real doubt the "ears" would confirm a proper sounding component from a well designed one?

But people have conducted tests with good-measuring but maybe not as fully wrought as Dan’s pro gear by re-digitizing through a DAC/ADC loop for 10 or 20 generations and not been able to detect any difference at all between the first and the tenth. I think food-measuring DACs and ADCs are highly unlikely to be the cause of any audible effects these days.

Yes our member @Blumlein 88 had put up some test listening files doing just that, sadly they no longer are available.
In any case the results were as expected, no one was reliable able to identify the master from gen 20.

 

[Dan Lavry]

Was there ever any real doubt the "ears" would confirm a proper sounding component from a well designed one?


Yes our member @Blumlein 88 had put up some test listening files doing just that, sadly they no longer are available.
In any case the results were as expected, no one was reliable able to identify the master from gen 20.

I encountered people that say they don't care about specs. Some people tried to credit me with having a great ear for making good gear. It was not my ears. There was some designer guy that was listening to resistors for best sound. Well that is a bit extreme, but true.

 

[Dan Lavry]

Units should be tested separately. An AD – DA loop’s output THD+N results will be dictated by the weaker performer. Such test can hide a high performing unit, and there is no information about which of the unit is the limiting factor.

So here is a question: Say you loop AD in series with DA. The unit’s flatness response is perfectly flat bout it has a 0.1dB increase at and around 5KHz. The converter act like a very mild EQ boost.

So if you loop it 10 times, the 5KHz range gets amplified by 1dB. For +/-0.2dB boost you get “free” +/-2dB boost.

Listening to the outcome can give a good converter a bad name. I would think this flatness response issue would override the increase in noise and distortions. The flatness response deviation increases linearly with number of loops.

Distortions, noise, flatness all count. The question requires some way to quantify it. I would think that looping the converters to “uncontrolled” 1-2dB error is a way to force the process to magnify what can be very small error for real life use.

Comments are welcome.

 

[Hapo]

 

[danadam]

Yes our member @Blumlein 88 had put up some test listening files doing just that, sadly they no longer are available.

AFAICT they are still there:

Listen and choose the 8th generation digital copy.

Each of these zip files will open into three wav files. One is labeled Reference, one is A and one is B. These are 30 second snippets of full songs. The originals were from CD so all 44.1 khz and 16 bit. Either A or B has been thru 8 generations of DA to AD copying. The other is identical...

www.audiosciencereview.com

8th generation ADC/DAC loop vs the original.....can you hear it?

Download this zip file. It will unzip into a folder with three 30 second snippets of a Fiona Apple song. Two are the original digital file. One is an 8th generation loop thru DAC/ADC. Pick the odd file out on the poll...

www.audiosciencereview.com

I have also another one in my notes:

ZMIX Converter Tests

 

[Newman]

...food-measuring DACs and ADCs...

Is this some new feature that I am not aware of?

 

[Sal1950]

AFAICT they are still there:

Listen and choose the 8th generation digital copy.

Each of these zip files will open into three wav files. One is labeled Reference, one is A and one is B. These are 30 second snippets of full songs. The originals were from CD so all 44.1 khz and 16 bit. Either A or B has been thru 8 generations of DA to AD copying. The other is identical...

www.audiosciencereview.com

8th generation ADC/DAC loop vs the original.....can you hear it?

Download this zip file. It will unzip into a folder with three 30 second snippets of a Fiona Apple song. Two are the original digital file. One is an 8th generation loop thru DAC/ADC. Pick the odd file out on the poll...

www.audiosciencereview.com

I have also another one in my notes:

ZMIX Converter Tests

Ah, good catch! I just assumed that after 7 years the files wouldn't be there and was too lazy to start searching. You know what they say about "assume".
Thanks !

 

[antcollinet]

But the concept of using resistors for analog attenuation is solid. Digital attenuation means ending up with less bits (quantization). Take a dynamic range of 96dBFS (16-bit, CD quality) and attenuate digitally by 36dB. The resulting DR is 60dB or 10 bits audio. Analog attenuation offers quantization free attenuation.

Turning that around a little.

The quantisation noise out of the DAC is fixed at the (in your example) -96 dbFS level. DR is lost because the signal is reduced. But if you can't hear the noise in silence between the tracks at normal listening level - then nor can you hear it when the volume is reduced.

Then consider that there are almost no 16 bit DACS. 24 is the minimum and then quantisation noise is already below the analogue noise of the output electronics (typically limited to no better than 22 bits.) I'm not aware of any situation where noise at -132dB FS is ever going to be audible, except in the "user being an idiot" case of digital volume turned down low and post DAC gain turned up to ludicrous levels to compensate the DAC low output level.

Digital volume control done at the 24 bit or higher level - as far as I can determine, and baring user stupidity - is audibly perfect.


(Nervously aware that I am debating here with someone who designs the damn things - usually a situation where I end up looking like a bloody idiot. Rushing in where angels fear to tread regardless )

 

[antcollinet]

The outcome of running ADC/DAC loop many times does not yield what people expect, and can be very misleading:

Lets take a simple example. Say each unit (AD and DA) has a noise floor level at -100dB.

The noise is mostly random, uncorrelated. So each path through the converters generate “different noise” across the frequency range. The combined noise does increase with each pass, but not linearly. Running 10 passes (20 units) raises the noise from -100dB each unit to -87dB. Even 100 loop passes (200 units) yields -76dB.

The more important point is that after the first path, the distortion of 10 or many more loop runs will be virtualy the same as that of a single AD/DA run.

Say the input for the first path is a perfect full scale analog 1KHz tone, but the converters first path generate -60dBFS harmonic distortion. This is the “new signal” for the next loop run. The converter sees a 1KHz full scale and a -60dBFS distortion. It is going to “apply” its distortion mechanism on such signal. So the 1KHz tone is going to add that -60dB distortion (as it did before). But what happens to the previous distortion? It was at -60dB, the converter will generate it at -120dB (60dB below -60dB distortion). The "distortion of the distortion" is too small, below the noise floor to matter at all.

I saw that “argument” years ago. It was used by a company to promote their stuff. The converters noise goes up some but they sound the same after many loops, even if they suck. The fact that they sound the same after 10 path does not mean that they don’t distort.

Here I am rushing in again.

But in the second pass, the -60dB distortion tone from the 1kHz tone will be added to that created in the first pass. So double voltage to -54dB. After two more passes an additional 6db. Four passes later, another 6dB. So after 8 passes the distortion tone is at -42dB

What am I missing?

 

[Dan Lavry]

Turning that around a little.

The quantisation noise out of the DAC is fixed at the (in your example) -96 dbFS level. DR is lost because the signal is reduced. But if you can't hear the noise in silence between the tracks at normal listening level - then nor can you hear it when the volume is reduced.

Then consider that there are almost no 16 bit DACS. 24 is the minimum and then quantisation noise is already below the analogue noise of the output electronics (typically limited to no better than 22 bits.) I'm not aware of any situation where noise at -132dB FS is ever going to be audible, except in the "user being an idiot" case of digital volume turned down low and post DAC gain turned up to ludicrous levels to compensate the DAC low output level.

Digital volume control done at the 24 bit or higher level - as far as I can determine, and baring user stupidity - is audibly perfect.


(Nervously aware that I am debating here with someone who designs the damn things - usually a situation where I end up looking like a bloody idiot. Rushing in where angels fear to tread regardless )

 

[Dan Lavry]

I am talking about feeding a DA a digital attenuated signal, thus less bits. Also it does not matter that the DA is 24 bits data. The question is where the DA noise floor is relatively to the attenusted signal.
I agree that little digital attenuation is not a practical issue. But if the DA has a fixed noise floor and you feed it less bits the signal to noise is reduced.
You are correct that at 24 bits quantization is not much of an issue. But what if you play 16 bits music attenuated by say 36dB? The DA sees the bottom 10 bits. The top 6 bits are 0. That is not the original 16 bits conversion. What am I missing?

 

[Dan Lavry]

Turning that around a little.

The quantisation noise out of the DAC is fixed at the (in your example) -96 dbFS level. DR is lost because the signal is reduced. But if you can't hear the noise in silence between the tracks at normal listening level - then nor can you hear it when the volume is reduced.

Then consider that there are almost no 16 bit DACS. 24 is the minimum and then quantisation noise is already below the analogue noise of the output electronics (typically limited to no better than 22 bits.) I'm not aware of any situation where noise at -132dB FS is ever going to be audible, except in the "user being an idiot" case of digital volume turned down low and post DAC gain turned up to ludicrous levels to compensate the DAC low output level.

Digital volume control done at the 24 bit or higher level - as far as I can determine, and baring user stupidity - is audibly perfect.


(Nervously aware that I am debating here with someone who designs the damn things - usually a situation where I end up looking like a bloody idiot. Rushing in where angels fear to tread regardless )

I chose analog attenuation reduces both the signal and the noise, thus maintaining the dynamic range.

Digital attenuation operates on the data before it is fed to the DA. I was not going to that. Say a DA provides 18dBu un-balanced. You want to attenuate to line level 4dBu and consumer level around -8dBu. It amounts to 14dB attenuation for line and 26dB for consumer. That is over 2 bits and 4 bits respectively.

The DA is generating much smaller signals. But the noise floor of the DA stays the same, it is electronic noise. So shifting the most significant bits and loss of dynamic range is only one issue. The least significant bits are also shifted and their contribution to the fine detail of the waveform is buried under the noise.

A DA with full scale analog output of 10V p-p. analog output noise floor at -100dB (100uV noise). You ask it to feed a consumer level 0.316V. The converter will “try” to construct the samples accurately but the level of the fine detail is pushed under the noise.

Take a very extreme example, attenuation by say 84dB. The signal is only 16dB above the noise floor. Only 4 bits generate the signal, the other details in the lower bits are gone.

The software designers of DAW have a different job to do and they are restricted by the need to do it in the digital world. I am a hardware designer, so I could do it with resistors in hardware (or software). Hardware is a pure approach, the converter noise is attenuated with the signal and the waveform detail stays intact. Attenuating the digital value feeds the DA noise without attenuation. A power amplifier receives less DA noise.
.
It may not be an issue to most, but at high attenuation one loses bits. I am not talking about what one hears. I am staying in my lane talking technical stuff.

 

[danadam]

But what if you play 16 bits music attenuated by say 36dB? The DA sees the bottom 10 bits. The top 6 bits are 0. That is not the original 16 bits conversion. What am I missing?

I think @antcollinet point is that if the attenuation of such signal is done in 24-bits and the DAC accepts 24-bits then it will still see all the bits, only shifted down. The top 6 bits will be 0 and bits 7 to 22 will contain the original 16-bit data. Of course some of those bottom (17-22) bits won't be properly reconstructed because that's below the analogue noise of the converter, but some of them (17th, 18th) might. So the reduction in SNR won't be 6-bits but something smaller.

Here I generated 44.1 kHz, 16-bit, full scale 999.91 Hz signal without dither (with these parameters it is self-dithered). Played through JCally JM20 and captured with E1DA ADCiso we get SNR 98.2 dB and ENOB 16-bits:

Then I attenuated it by 36 dB in 24-bits and played again. We get SNR 81.5 dB and ENOB 13.2-bits:

 

[antcollinet]

I am talking about feeding a DA a digital attenuated signal, thus less bits. Also it does not matter that the DA is 24 bits data. The question is where the DA noise floor is relatively to the attenusted signal.
I agree that little digital attenuation is not a practical issue. But if the DA has a fixed noise floor and you feed it less bits the signal to noise is reduced.
You are correct that at 24 bits quantization is not much of an issue. But what if you play 16 bits music attenuated by say 36dB? The DA sees the bottom 10 bits. The top 6 bits are 0. That is not the original 16 bits conversion. What am I missing?

But isin't it the case that in a 24 bit or higher dac, the 16 bit data is padded out to 24bits before passing to the actual DAC function. As long as digital volume control is done after padding then we are dealing with a 24 bit noise floor. In fact the 16 bit noise floor is part of the padded signal, and being 48dB above the 24bit digital noise floor, is itself turned down without SNR loss for that first 48dB.

Yes - if digital volume is carried out at the 16 bit domain perhaps there is an issue - but I don't think it is in typical DAC with preamp function, even when the source is 16 bits.


Edited to add:

I am not talking about what one hears. I am staying in my lane talking technical stuff.

I fully agree that if one is only looking at the numbers/measurements, and ignoring audibility - then analogue attenuation is going to beat digital for most metrics - possibly excluding channel matching depending on the analogue implementation.

 

[Dan Lavry]

I think @antcollinet point is that if the attenuation of such signal is done in 24-bits and the DAC accepts 24-bits then it will still see all the bits, only shifted down. The top 6 bits will be 0 and bits 7 to 22 will contain the original 16-bit data. Of course some of those bottom (17-22) bits won't be properly reconstructed because that's below the analogue noise of the converter, but some of them (17th, 18th) might. So the reduction in SNR won't be 6-bits but something smaller.

Here I generated 44.1 kHz, 16-bit, full scale 999.91 Hz signal without dither (with these parameters it is self-dithered). Played through JCally JM20 and captured with E1DA ADCiso we get SNR 98.2 dB and ENOB 16-bits:

View attachment 536018

Then I attenuated it by 36 dB in 24-bits and played again. We get SNR 81.5 dB and ENOB 13.2-bits:

View attachment 536019

I think @antcollinet point is that if the attenuation of such signal is done in 24-bits and the DAC accepts 24-bits then it will still see all the bits, only shifted down. The top 6 bits will be 0 and bits 7 to 22 will contain the original 16-bit data. Of course some of those bottom (17-22) bits won't be properly reconstructed because that's below the analogue noise of the converter, but some of them (17th, 18th) might. So the reduction in SNR won't be 6-bits but something smaller.

Here I generated 44.1 kHz, 16-bit, full scale 999.91 Hz signal without dither (with these parameters it is self-dithered). Played through JCally JM20 and captured with E1DA ADCiso we get SNR 98.2 dB and ENOB 16-bits:

View attachment 536018

Then I attenuated it by 36 dB in 24-bits and played again. We get SNR 81.5 dB and ENOB 13.2-bits:

View attachment 536019

Nice work. Now we need to expand the argument. So far, we talked about a full scale 1KHz sine wave, and what happens when you effectively truncate it. You decrease the signal to noise and generate some distortions. It still "looks like" a sine wave with little alterations.

But take a real instrument, where the character of the sound is defined by harmonics. Low level harmonics (with small amplitude) require lower order bits. If digital “push down” some lower bits signals under the noise floor, it may eliminate the fine details that define the instrument sound.

For volume control, I chose analog attenuation. The main reason (if I recall) was the noise issue. Say feeding an 18dBu signal directly to a 4dBu line level requires 14dB attenuation. With digital attenuation, the noise of the DA (even with no digital signal) is fed directly to the power amplifier input. With analog attenuation the noise is reduced by 14dB.

Can anyone hear 14dB more noise, some added distortions and some loss of details (harmonics)? It depends on the music, the setup and of course the listener. Driving consumer level is 26dB more noise. My view as a designer is to avoid degradation if it can be avoided (and be ready for worse case scenario). It calls for providing some MARGIN for the best ears under the most critical conditions (say a great mastering room).

Analog attenuation is nearly perfect. Digital attenuation causes some alteration of the data. Doing it in digital would be much cheaper and easier, and it is “not noticeable” to almost all the customers. I still want to avoid all possible issues with large margins, beyond human hearing capabilities.