• Welcome to ASR. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Master Thread: Are Measurements Everything or Nothing?

Many of our AD and DA converters are used by the companies that make the vinyl. Much of the music passes through converters before and during the manufacturing of vinyl. I think it should put an end to the argument that vinyl is better because it is all analog.

Yeah, I've been pointing out since the 1990's that Redbook CD can capture LP sound without any trouble, but the other way around just doesn't work.
To paraphrase another line from an old movie "Evidence means little to their kind". (Sorry, Brad.)
 
What caught my eye first on the thread was SINAD. I preferred the dB scale from day one. SINAD is the same results we test for without a (-). For DA we use dBr relative to 24dBu analog, for AD we use dBFS relative to full scale (the largest code).

Measuring analog THD+N in % starts with the noise (no signal), but the peak is not well defined. It is what we had in the old days. With digital we have the noise floor and a well-defined peak (the maximum DA output, the highest AD number). Digital gives the measurement a definite and precise reference.

Worse yet using % scale is linear. We use logarithmic scale (dB) to approximate the ear. The SINAD and my THD+N are logarithmic. They are also easier to display. I use 0, -1dB, -2dB, -3dB… LEDS as far as I can. Both audio sound and visual level meter offer better connection with the logarithmic scale.
 
What caught my eye first on the thread was SINAD. I preferred the dB scale from day one. SINAD is the same results we test for without a (-). For DA we use dBr relative to 24dBu analog, for AD we use dBFS relative to full scale (the largest code).

Measuring analog THD+N in % starts with the noise (no signal), but the peak is not well defined. It is what we had in the old days. With digital we have the noise floor and a well-defined peak (the maximum DA output, the highest AD number). Digital gives the measurement a definite and precise reference.

Worse yet using % scale is linear. We use logarithmic scale (dB) to approximate the ear. The SINAD and my THD+N are logarithmic. They are also easier to display. I use 0, -1dB, -2dB, -3dB… LEDS as far as I can. Both audio sound and visual level meter offer better connection with the logarithmic scale.
In the old days, we measured distortion by comparing the true RMS voltage of everything but the test signal with the true RMS voltage of the incoming test signal. At least that's how the classic HP distortion analyzers do it. My 8903b analyzer reads either in distortion-level actual voltage, percent, or dB. (My older HP 339A requires one to set the voltage of the test signal, and then it reads the distortion and noise in dB.) Given that it's measuring everything that isn't signal, it's measuring both distortion and noise, so the dB measurement is SINAD.

To measure noise by itself, we just measured the true RMS voltage of the device with no input signal. Any lab-grade RMS voltmeter with, say, at least 100K of bandwidth will do that. I've used both an HP3456 and an HP34401 for those measurements that didn't use the analyzers.

I still would rather do quick distortion measurements that way than run software analysis, though I do have a Quant Asylum QA403 Analyzer Interface on my wish list, and truly nothing beats spectrum analysis with a -100 dB noise floor or better.

Rick "nowhere on the planet of paying for an AP" Denney
 
He has indeed, and has been reviled almost as much as I have for standing up for facts and mathematics in the face of greed.

Seems like so many review threads get challenged with:

1. Measurements are not everything.

2. You all never listen.

3. I trust my ears, not graphs.

4. I don't listen to graphs. I listen to music.

5. You all must not listen to music at all.

6. Why don't you all buy the best SINAD gear?

7. I have heard your best SINAD gear and they sound terrible. I don't like any of this Chinese stuff.

8. You don't trust your ears. I/we do.

9. All these reviewers/youtubers/audophiles say these amps, DACs, etc. sound different and you say they don't. They can't all be wrong.

10. Surely designers have created certain house sound for each equipment which your measurements don't show.

11. Your measurements are only at one frequency. You need to also measure X, Y and Z like impulse response, slew rate, etc., etc.

12. You guys run a cult here where you only go by measurements and no one is allowed to disagree.

On and on...

I have had to answer these so many times that I thought it is time to stop having them go into every review as they are not product specific. From here on, any such questions should be posted here. Answers will be given in this thread and simply referenced in future challenges in other threads.

Thanks. You all are free to discuss this topic, provide answers, argue, whatever, in this thread. :)
About measurements:

As a manufacture, we test each converter extensively. But I hear complaints that “SINAD is not a perceptual model”. Here is my take:

Perceptual model? It opens a can of worms. People will argue about kind of music, the proper drum kick sound and all that is needed or ideal test…

So the point is to be able to measure how the converter can handle ANY POSSIBLE AUDIO SIGNAL. How can we possibly do that?

It is difficult to verify performance in the time domain. In theory, you can feed a digital full scale random signal file (audio bandwidth) to a DA, convert back with the AD and compare the files. The approach has problems, a small DC offset and/or a small amplitude difference, cause mismatch. Also, time delays will need to be corrected and aligned to great accuracy. The approach is not practical. Random source is good for providing statistics. A good match of the file is good news, but random signal is as unmusical as I can think of. It tells you nothing about harmonics, it is random after all…

But it is possible to verify performance in the frequency domain. It takes some math and signal theory to understand how, but extensive testing of how the device operate across the hearing range provides a view of the performance. We can’t test for all frequencies between 20 and 20KHz. But we can have the range broken to say 50 or 100 frequencies. Similarly we can step the amplitude by .1dB, 1dB or anything. It is about taking enough of the important data to get a good picture of how future signals (the music) will be handled by the circuits.

I think of testing as exercising the converter. My Pilates instructor would be good at it. We do the core, the legs, arms, inner thigh, outer. Leave nothing unstretched.

One can argue that such a test is still not done with real music. It is true. Single and multiple sine waves are not music. I welcome some good violin and clarinet players (I play accordion). We can play and record, have fun and listen. But this is not a good test, it does not test the lower octaves. It is not practical approach. Proper testing overcomes such limitations. Having control over the test signals range enables testing how the unit will handle music across the whole hearing range and beyond.
 
About measurements:

As a manufacture, we test each converter extensively. But I hear complaints that “SINAD is not a perceptual model”. Here is my take:

Perceptual model? It opens a can of worms. People will argue about kind of music, the proper drum kick sound and all that is needed or ideal test…

So the point is to be able to measure how the converter can handle ANY POSSIBLE AUDIO SIGNAL. How can we possibly do that?

It is difficult to verify performance in the time domain. In theory, you can feed a digital full scale random signal file (audio bandwidth) to a DA, convert back with the AD and compare the files. The approach has problems, a small DC offset and/or a small amplitude difference, cause mismatch. Also, time delays will need to be corrected and aligned to great accuracy. The approach is not practical. Random source is good for providing statistics. A good match of the file is good news, but random signal is as unmusical as I can think of. It tells you nothing about harmonics, it is random after all…

But it is possible to verify performance in the frequency domain. It takes some math and signal theory to understand how, but extensive testing of how the device operate across the hearing range provides a view of the performance. We can’t test for all frequencies between 20 and 20KHz. But we can have the range broken to say 50 or 100 frequencies. Similarly we can step the amplitude by .1dB, 1dB or anything. It is about taking enough of the important data to get a good picture of how future signals (the music) will be handled by the circuits.

I think of testing as exercising the converter. My Pilates instructor would be good at it. We do the core, the legs, arms, inner thigh, outer. Leave nothing unstretched.

One can argue that such a test is still not done with real music. It is true. Single and multiple sine waves are not music. I welcome some good violin and clarinet players (I play accordion). We can play and record, have fun and listen. But this is not a good test, it does not test the lower octaves. It is not practical approach. Proper testing overcomes such limitations. Having control over the test signals range enables testing how the unit will handle music across the whole hearing range and beyond.

I want to point out that measurements are the only tool we have for manufacturing. The listening and feedback happens during the development of prototypes. Production units have to measure up against the approved prototype.

I sent the last prototypes to high end mastering, and some to recording and mixing facilities. The sound is no longer the issue, I am confident about that. Indeed, it was not an issue for anyone. I did get some feedback, to increase the AD adjustable analog gain range by 2dB. Also, to double the edit state timeout to 20 seconds.

I think some lack of trust in measurements and specifications is due to very little published specifications. A complete set of tests is long, and not well understood by the users (audio people). It is “too technical” for many. But publishing the minimum (often only for 1KHz tone test) or a couple of other figures next to product dimensions and weight is inadequate description. It leaves room for speculations and misinformation…
 
About measurements:

As a manufacture, we test each converter extensively. But I hear complaints that “SINAD is not a perceptual model”. Here is my take:
Hi Dan, nice to meet you.
For the uninitiated of us would you name the company you represent here?
TIA
 
A 924 always wanted one, never quite managed it.
Keith
 
Since the absolute sensitivity of human hearing is pretty well established, especially if we're talking about sampling rates over 64k, it's pretty simple to know when you're clear of any "perception".

20 bits at 96k is overkill, for instance, if it's close to real. In most homes, 16 bits is plenty. With consumer (as opposed to pro, not talking "hifi" here) 16 bits is probably overkill.

Now, the only relevant issue is "spectrum of error signal". But at 20 bits it's just not necessary to care, at 16, just barely, and only if the error signal is very peaky and in the middle of the spectrum.

And, of course, measurements are how one tests ADC and DAC. Running 100 trials in ABC/hr for each device isn't happening, and isn't necessary.
 
Hi Dan, nice to meet you.
For the uninitiated of us would you name the company you represent here?
TIA
I made sure to not mention the company or specific products. I am speaking as an EE with tons of experience with converters, way before digital audio. I am not going to change my last name.
I hope I am not breaking the forum rules.
 
I stopped looking at forms 20 years ago, after 2+ years as a moderator at Pro Sound Web. I needed all my time for design work. I took a little time to catch up, viewing a few audio forums. I looked at the ASR thread "Are Measurements Everything or Nothing". I liked what I saw. It looks like a good thread with interesting comments. Good work.

Excellent to have you here! I still throw your white papers on sample rates at people who insist we need 196 kHz or whatever.
 
You cannot decipher this without a conversion.

01010111 01101000 01100101 01101110 00100000 01001001 00100000 01110010 01100101 01100001 01100100 00100000 01110100 01101000 01101001 01110011 00100000 01100110 01101111 01110010 01110101 01101101 00101100 00100000 01100101 01110110 01100101 01101110 00100000 01110100 01101000 01100101 00100000 01101100 01100101 01110100 01110100 01100101 01110010 01110011 00100000 01100001 01110010 01100101 00100000 01100100 01101001 01100111 01101001 01110100 01100001 01101100 01101100 01111001 00100000 01100101 01101110 01100011 01101111 01100100 01100101 01100100 00101100 00100000 01100001 01101110 01100100 00100000 01111001 01100101 01110100 00100000 01001001 00100000 01100011 01100001 01101110 00100000 01110011 01110100 01101001 01101100 01101100 00100000 01100100 01100101 01100011 01101001 01110000 01101000 01100101 01110010 00100000 01110100 01101000 01100101 01101101 00100000 01110111 01101001 01110100 01101000 01101111 01110101 01110100 00100000 01100001 01101110 01111001 00100000 01110000 01110010 01101111 01100010 01101100 01100101 01101101 01110011 00101100 00100000 01100001 01110011 00100000 01100011 01100001 01101110 00100000 01100101 01110110 01100101 01110010 01111001 01101111 01101110 01100101 00100000 01100101 01101100 01110011 01100101 00101110
These guys probably can (circa 1:27)

 
I made sure to not mention the company or specific products. I am speaking as an EE with tons of experience with converters, way before digital audio. I am not going to change my last name.
I hope I am not breaking the forum rules.
As far as I understand the rules there is no issue talking about your company anywhere. But if you want to promote it or your products then those posts should be in the Audio Industry sub forum.

So "My company looks like this and makes these" Or "when we did this we do it that way" (as you did upthread) no problem when it is used as part of a discussion. So for example we have some manufacturers who when asked, give factual information about their products in the products review thread, and that is fine.

But "We have a great new Doodad - you should buy it" goes to the Audio Industry sub forum.
 
Since the absolute sensitivity of human hearing is pretty well established, especially if we're talking about sampling rates over 64k, it's pretty simple to know when you're clear of any "perception".

20 bits at 96k is overkill, for instance, if it's close to real. In most homes, 16 bits is plenty. With consumer (as opposed to pro, not talking "hifi" here) 16 bits is probably overkill.

Now, the only relevant issue is "spectrum of error signal". But at 20 bits it's just not necessary to care, at 16, just barely, and only if the error signal is very peaky and in the middle of the spectrum.

And these facts have not changed since what, the late 90s... maybe earlier?

You'd never know it from the river of obfuscatory blather emitted by the audiophile contingent since then.
 
About measurements:

As a manufacture, we test each converter extensively. But I hear complaints that “SINAD is not a perceptual model”. Here is my take:

I hope I am not breaking the forum rules.

As far as I understand the rules there is no issue talking about your company anywhere. But if you want to promote it or your products then those posts should be in the Audio Industry sub forum.
Yes, he talking about his company and manufacturing processes, etc, has Audio Company in his banner but nothing in his Account - About
I don't see it as matter of rules, but if your going this far why not identify yourself to those members here that don't already recognize the name?
Why the "insider" secret ?

Heh. It's kinda right there in front of you.
No it's not, or I wouldn't have asked, Your evidence please
 
And these facts have not changed since what, the late 90s... maybe earlier?
Well, maybe early 2000's when delta-sigma took over, and the "problems around zero" ceased to exist.

Regardless, the noise at your eardrum from molecules bouncing off it is not going to go away.
You'd never know it from the river of obfuscatory blather emitted by the audiophile contingent since then.

If it was only blather it would be ok. It's the threats, etc, that really need to stop.
 
Yes, he talking about his company and manufacturing processes, etc, has Audio Company in his banner but nothing in his Account - About
I don't see it as matter of rules, but if your going this far why not identify yourself to those members here that don't already recognize the name?
Why the "insider" secret ?


No it's not, or I wouldn't have asked, Your evidence please
No inside secretes. I knew some people here, I did not put much thought into presenting myself to others. I wanted to talk about audio.

Someone posted Lavry Engineering, so now you know my company name. Some people don’t know about us, it is understandable, we don’t advertise at all.

Why was I carefull to keep the company and product names? I was posting about audio and electronics gear. The facts are presented for you to accept or reject. I tend to think that if I were to include statement or information about how great my products are, it would take focus away from the conversation. You could say “the guy is just trying to sell his stuff”. So I stay on topic and I stay technical, my area.

I used the same approach when I was a moderator for a tech forum 20 years ago.
 
Back
Top Bottom