How to ensure the exhaustiveness of measurements?

[JustAnAudioLover]

As to amplifiers or electronics generally, there are only three measurements that affect what we hear. Distortion, noise and frequency response. Do those at various power levels and permitted loads, and that will fully characterise the amplifier.

Loudspeakers will need additional measurements, polar response and pair-matching for stereo for example, but again, these measurements will characterise what a loudspeaker can do.

Audio has been measured for 100 years, so the body of knowledge is pretty substantial.

There's no evidence that any measurements are lacking that would affect what we hear.

S

Thanks, that's a very clear answer

 

[Lambda]

How can we be sure that measurements are exhaustive?

Or, put another way, how can we be sure that all the measurements performed on, say, a DAC, like SINAD, jitter and so on are complete enough to say that a device is basically transparent to the ear?

They are not.
(The typical measurements here on ASR.)

And I think we should be more upfront about this and not pretend that measurements are the "be-all and end-all."

Don't get me wrong—measurements can be the definitive "be-all and end-all." But that would require a vast number of measurements, which would be difficult to interpret.

You can come up with many ways a DAC, for example, can affect overall performance in ways that wouldn’t show up in standard measurements, or where the extent of the issue is not proportional to the output.

For example: THD+N​

Two DACs can have the same THD+N but perform totally differently.

• How much it is the noise
• What type of noise?
• What is the spectral distribution of that noise?
• How much of the measurement is THD?
• What type of distortion is present? What order

And that’s only at 1kHz and for a specific signal level. What happens at different frequencies and levels?

Now we need a THD+N vs. frequency and level graph—already a complex 3D diagram. You can see how things get complicated quickly.

Or what if the DAC applies some strange DSP processing, like stereo image expansion or other "enhancements"? Standard tests wouldn't reveal this, but you could hear it.

An "easy" way: The Null Test​

A null test involves subtracting the output of a DAC or amp from a known "ideal" signal.

Conceptually, this is simple—but in practice, the test setup is difficult. There’s a lot of room for errors: phase shifts, timing mismatches, and level mismatches can make it hard to properly null the signal.

If it nulls, it's a straightforward proof that the output is "ideal."
But it's hard to extract meaningful, quantitative data from such a test, and many users here seem to prefer the simple "more is better" SINAD ranking.

The Problem with SINAD Rankings​

I would say it’s kind of pointless and misleading to rank the best DACs and amps purely by SINAD.


SINAD/THD+N can prove without a doubt if a DAC or amp is bad.
But it alone cannot prove whether a DAC is truly "transparent."
It’s just a proxy for engineering excellence—we can assume that if SINAD is high, it was properly engineered.

 

[Cbdb2]

and you didn't know better because you weren't aware that you needed to measure something else to get the full picture.

The crust of the biscuit. We are aware of the limitations of the measurements. For audio electronics there are none. Speaker measurements are close behind.

 

[JustAnAudioLover]

The crust of the biscuit. We are aware of the limitations of the measurements. For audio electronics there are none. Speaker measurements are close behind.

How do we know that measurements of electronics have no limitations? (especially since several people in this thread said the opposite)
Proving there isn't missing piece to a scientific measurement seems pretty impossible to me?

 

[kemmler3D]

It might have been said already but exhaustive measurements are usually not useful.

That said, for electronics, there is a shortcut to "exhaustive" measurements. You can take two recordings, (The original and the output of the device) and compare them with Deltawave. This shows you everything that's changed. There is still a need to interpret results, but in an A-B=C operation you can be sure everything you want to know is somehow captured in C. You can even play back C and listen to it.

Also, for electronics, you can capture 100% of the output of any given device simply by recording it. There isn't some parallel signal path that escapes the recorder but reaches your ears. As such, there isn't anything an amp or DAC can do that isn't amenable to quantitative analysis. 100% of a novel can be found in between the covers of the book, audio recordings are no different.

 

[JustAnAudioLover]

It might have been said already but exhaustive measurements are usually not useful.

That said, for electronics, there is a shortcut to "exhaustive" measurements. You can take two recordings, (The original and the output of the device) and compare them with Deltawave. This shows you everything that's changed. There is still a need to interpret results, but in an A-B=C operation you can be sure everything you want to know is somehow captured in C. You can even play back C and listen to it.

Also, for electronics, you can capture 100% of the output of any given device simply by recording it. There isn't some parallel signal path that escapes the recorder but reaches your ears. As such, there isn't anything an amp or DAC can do that isn't amenable to quantitative analysis. 100% of a novel can be found in between the covers of the book, audio recordings are no different.

Very interesting, thanks!

 

[antcollinet]

I'm wondering on a more "high-level" point of view, how to ensure that measurements are exhaustive enough that we can say, from the results we've got, that a DAC or amp is indeed acoustically transparent to the human ear.

And you've already been answered.

When someone can demonstrate (in a scientifically valid way - ie properly controlled and blind) that they (or someone) can hear something that the current measurements do not show. Then we can evaluate that and see what mesurements need to be put in place to cover that phenomena.

But we've been waiting a long time for someone to actually do this. So far the only thing audible has been...... crickets.