Test and Measurement Procedures and Standards

[Blumlein 88]

Seems like the discussion has died down a bit.
I'd like to contribute as well in measuring DAC's or headphone amps, however I do agree we need to standardize our results.
As I understand from the recent discussion, allow me to summarize and correct me if i'm wrong.

Testing items is listed below:

1. Frequency response - 44, 48, 88 & 96 kHz replay bandwidth. Extend acquisition bandwidth beyond replay to show filter roll off and out of band noise (is 15Hz -> 24kHz enough?, is 88.1 really necessary? is 61 points slow sweep enough?)
2. No signal noise floor (FFT's at 32K, low average,)
3. 1kHz tone @ 0dB, -1dB, -60dB FS. THD + Noise (1 graph with overlapped results?)
4. 48kHz J test. USB and spdif input (if available) (FFT's at 32K, low average)
5. Small signal time waveform 16 bit and 24 bit @ -90 something dB (is 24 bit necessary?)

Could we standardize all tests to either 16bit 44.1kHz or 24bit 96kHz?
Based on Amir's test , can we also standardize the graphs to dBv vs freq? however we would need a reference voltage @ 0dB

Amir wants to keep the test threads streamlined so they will be attractive to more people. Probably a good idea. So I would suggest we do fairly comprehensive testing. Yet much of the results we need only check and actually show in detail if something is unexpected. Like frequency response. Pretty unusual for that not to be very good. So sure check it, but unless something odd shows up, just say it was fine. Then move on.

About your #3 above, yes, but I don't think that will work for one graph. All the results would be right on top of each other. Even with color coding it will get hard to read.

I also think lots of this testing will be to show exemplary performance need not involve megabuck gear. So what does lower priced gear skimp on and sometimes fall short? Power supplies, and low noise seem the two most common areas they fall short. And output signal level too. So maybe you show a couple areas a given bit of gear over-achieves and a couple areas it is less than stellar. You still have to do comprehensive testing to find those places.

It need not be that hard. Myself, I made up a short file with all the various test signals in it. Few seconds of silence in between. Made the tests at different sample rates. So I set up the file, run it to set levels on the DAC and ADC. Then run it again. Record the result and analyse it after the fact. Not terribly burdensome or time consuming that way.

 

[Wombat]

Some insight into the Underwriters Laboratory(UL Mark) wrt audio:

https://www.ul.com/aboutul/standards-of-business-conduct/

https://www.ul.com/inside-ul/ul-introduces-new-audio-quality-verification-mark/

https://standardscatalog.ul.com/ enter 'audio' in search.

 

[amirm]

Interesting that UL is getting into audio certification. Wonder if anyone has applied for it.

 

[RayDunzl]

Wonder if anyone has applied for it.

Database of companies/products with various "verified" marks...

https://verify.ul.com/?post_type=mark&s=

 

[amirm]

Did you find HD audio there?

 

[RayDunzl]

No.

 

[captain paranoia]

I find these sort of spectrograms useful from Audacity.

Our RF design guys use that sort of plot, both simulated and measured, for investigating RF generation schemes (when you have a lot of digital and RF mixing going on). They're a good way of finding the mix products you forgot about...

 

[captain paranoia]

My suggestion on another thread:

The proper engineering answer to this question of 'is a DAC sesnitive to noise on the USB' is to do an objective test.

It's what the standard EMC compliance tests do, only we may want to look a bit harder.

We need to start with a clean supply input, and use a LISN to deliberately inject an interfering source into the input, and see how well the UUT eliminates that interferer from its output.

This test can be done with power input (mains or DC), with USB input (power and signal), and with SPDIF input (yes; deliberately add the dreaded jitter).

The magnitude of the interfering signal ought to be representative of real-world environments.

With known clean inputs, and quantifiable interfering sources, we can then be confident that any unwanted signal appearing at the ouput is caused either by generation within the UUT itself (with no interferer), or by the inability of the UUT to be resistant to interference on its power and signal inputs. We can even start quoting rejection figures...

 

[astr0b0y]

A few marketing spiels for some DACs these days talk about whether they are design for good PCM or DSD playback or sometime they claim to have separate circuitry for both data inputs.
Is there any need (or even any way) to test the maximum rate of PCM or DSD that the device supports?
For example, would it be reasonable to assume that the waveform reproduction test would show better results for DSD256 input than PCM 48kHz?

 

[stunta]

Feel free to delete this post because here comes another stupid question from me -

I have a Benchmark ADC which IIRC, measures quite good. I know how to run programs on a computer. With this immense power and knowledge, is there any way I can contribute with useful measurements?

I will go hide now...

 

[amirm]

You very well could. The trick is to figure out how to correlate what you are measuring to what I am measuring. You would also have to invest in some software to turn it into a measurement system. Rightmark is free but very limited.

 

[stunta]

If I get the ADC to spit something out to my computer and send it over to you, would that help? Its easier than shipping physical DACs over.

This is the one I have: https://www.stereophile.com/content/benchmark-adc1-usb-ad-converter-measurements

 

[stunta]

In other words, is there any real-time analysis in the analog domain that software cannot save to file?

 

[amirm]

If I get the ADC to spit something out to my computer and send it over to you, would that help? Its easier than shipping physical DACs over.

This is the one I have: https://www.stereophile.com/content/benchmark-adc1-usb-ad-converter-measurements

We would both need to measure the same thing as BE718 and I plan to do.

Without it, it is very hard to correlate the results.

 

[stunta]

We would both need to measure the same thing as BE718 and I plan to do.

Without it, it is very hard to correlate the results.

I have the Topping D30 and the DX7s. So do you. Is that good enough or do you think we need the same exact unit?

 

[dc655321]

I don't know if it has been mentioned already (search turned up nothing), but I would be interested to see "crosstalk" measured where applicable.

Something like putting a 1 kHz, 0 dBFS signal in channel A, a null signal in channel B, and measuring (+ graphing) the signal on channel B. And vice versa.
It may be appropriate to measure not just at 0 dbFS, but also at lower output values. Not sure of appropriate values for the latter. Maybe -30 dB?

This came to me as I was looking at this little nugget and was concerned about some of its listed specs...

 

[captain paranoia]

I was looking at this little nugget and was concerned about some of its listed specs

I'm a bit concerned about the claimed PCM resolution...

"PCM up to 384Khz/64bit"

Really? 64-bit? That's 217 zV (zeptovolts, 10e-21) LSB with a 2V FSD.

32-bits is bad enough, at 930 pV LSB with 2V FSD.

 

[amirm]

I don't know if it has been mentioned already (search turned up nothing), but I would be interested to see "crosstalk" measured where applicable.

Yes, I keep wanting to measure this but forget. It is very simply over S/PDIF and analog inputs of amps. More work over USB.

 

[RayDunzl]

I'm a bit concerned about the claimed PCM resolution...

"PCM up to 384Khz/64bit"

I'd view it merely as a data format which could be successfully processed by the gear, not its resolution.

 

[dc655321]

"PCM up to 384Khz/64bit"

Really? 64-bit? That's 217 zV (zeptovolts, 10e-21) LSB with a 2V FSD.

Yeah, I noticed that too, but chalked it up to a misprint.
Or marketing voodoo.

I suppose the difference is really just intent.