Multi-tone audio testing?

[pkane]

It's often stated that single tone sine-wave tests that are typically used by audio engineers are insufficient to characterize a device under test. The intuitive claim made is that music consists of many tones combined, and so more complex test signals must be used to determine how a device will behave with real music.

Multi-tone frequency response charts are frequently posted, but usually without any quantitative analysis, i.e., measurements. The noise+distortion between the tones is pointed to as something to avoid, but often, it is hard to compare two different devices, unless the noise and distortion are high enough to become very obvious. Some examples from Amir:




AP analyzers and similar measurement devices, often include multi-tone test capability. This is frequently done to get a quick reading, often measuring TD+N (total distortion + noise) or similar characteristic. Because multi-tone signal can be quite short, such a test can be done in a few seconds, unlike the much longer frequency sweeps and other analysis tools normally needed to properly measure a device.

So, my question is.... How useful are the results of a multi-tone test? Does anyone actually post numeric measurements using these? Are such tests in any way better (or, for that matter, different) than single- or two-tone ones? Does the result change with the number of tones? Do we need 3 tones? 32? 64? 100? 1000? And why isn't everyone doing this type of testing, resorting, instead to mostly a single or possibly two-tone test signals? What other useful measurements can be derived from such a multi-tone test? SINAD? THD? IMD? others?

I plan to explore this topic in more detail by running and posting a few tests for discussion, but wanted to hear your experience, thoughts and opinions, first.

 

[MRC01]

... Are such tests in any way better (or, for that matter, different) than single- or two-tone ones? Does the result change with the number of tones? Do we need 3 tones? 32? 64? 100? 1000? ...

I think multi-tone testing is useful because the more tones you have playing simultaneously, the more harmonic and intermodulation artifacts you can get. Each tone has its own harmonics, which creates more difference tones, each of which can have harmonics, etc. When you add all that up you see a busier/messier distortion profile than you'd get with a single tone.

 

[Matias]

I think it is very useful visually although no single number metric is given. But always interesting to see how the noise floor and spikes show up between tones.

 

[RayDunzl]

REW lets you do some multitone.

Example:

Top, in room with microphone

Bottom - signal in loopback at Focusrite Clarett 4 Pre USB

There are choices for what tone to generate.

The one above is "ND - Tones spaced to avoid interharmonic distortion"

 

[Blumlein 88]

I think twin tone IMD is the best test to see how a piece of gear performs. I don't think you need single tone THD or SINAD type tests other than they are very simple and everyone understands how they work. If you use say 18.5 and 19.5 khz you even parse out which are from even and odd harmonics. The evens are at 1000 hz and multiples while those at 1 khz intervals around the higher tones are odds.

This is the most stressful test for a device. The multi-tones are just to convince those complaining music is more complex. They need to get educated about how their idea is misleading. With more tones you have to reduce the level of each tone and it becomes an easier test. You don't get IMD with only one tone so two is the best. The multi-tones also are messy to compare the results from one device vs another unless their is a large disparity in performance. IMD twin tone will do that just fine.

 

[NTK]

IMO I think the fundamental problem with quantifying non-linear distortions into numbers is that the distortions are non-linear. Superposition doesn't work. The THD number at 1 V output at 1 kHz tells us very little about what the THD will be at 1 V at 5 kHz or 5 V at 1 kHz. In this case, I think the picture of the multi-tone test spectrum plot is worth a thousand numbers

Don't know if you are familiar with this old paper.
https://www.researchgate.net/public...n_of_Nonlinear_Distortion_Measurement_Methods

 

[dc655321]

Could one just pick off the SFDR from the multi-tone and call it a day?

Asking for a friend...

 

[Matias]

I think twin tone IMD is the best test to see how a piece of gear performs. I don't think you need single tone THD or SINAD type tests other than they are very simple and everyone understands how they work. If you use say 18.5 and 19.5 khz you even parse out which are from even and odd harmonics. The evens are at 1000 hz and multiples while those at 1 khz intervals around the higher tones are odds.

This is the most stressful test for a device. The multi-tones are just to convince those complaining music is more complex. They need to get educated about how their idea is misleading. With more tones you have to reduce the level of each tone and it becomes an easier test. You don't get IMD with only one tone so two is the best. The multi-tones also are messy to compare the results from one device vs another unless their is a large disparity in performance. IMD twin tone will do that just fine.

IMD 18+19kHz may be more stressful but it is further from reality of music. Music signal is often -30 to -40dB in that frequency range anyway, so what stressing the DUT with this?
Multitone is closer to reality. And if the DUT is not stressed enough and results are similar to another DUT, well, then reality is that playing music with both DUTs will also sound similar, which is the end goal, right?
Please correct me if I am missing something.

 

[JohnYang1997]

People didn't use multitone is because the noise floor is limiting the measurements resolution. And it's not easy(genuinely difficult) to implement in analogue.
If you don't have AP, it's likely you get less than 120db of resolution sometimes 100db if you run real time.
If you do have AP. The DAC output is just not comparable to the oscillator output. Distortion is -110db vs -140db. The multitone generator limits what can be measured. As opamps nowadays can reach down to -160db -170db distortion. Using multitone isn't helpful.

Multitone is supplementary measurement to all other existing measurements. Like Sanskrit, M300 mkii have horrible multitone measurements but very good SINAD. But E30 has almost identical SINAD but almost perfect multitone graph.

I think THD no N vs Power and vs Frequency is also very useful. They can reveal what's masked by the noise. Also frequency spectrum is also very important whenever something weird happens like imd hump, weird thd+n vs frequency etc.

Have them all man. Have them all.

 

[RayDunzl]

462 tones (Focusrite loopback) maybe one dB below clipping the source:

Don't see a problem...

 

[RayDunzl]

462 tones from speakers:

 

[SIY]

It's often stated that single tone sine-wave tests that are typically used by audio engineers are insufficient to characterize a device under test. The intuitive claim made is that music consists of many tones combined, and so more complex test signals must be used to determine how a device will behave with real music.

Multi-tone frequency response charts are frequently posted, but usually without any quantitative analysis, i.e., measurements. The noise+distortion between the tones is pointed to as something to avoid, but often, it is hard to compare two different devices, unless the noise and distortion are high enough to become very obvious.

The AP has a delightful option- you can null out the test tones. You can also generate test tones that are centered in the FT bins so there's no windowing. Here's a couple examples:

 

[Blumlein 88]

IMO I think the fundamental problem with quantifying non-linear distortions into numbers is that the distortions are non-linear. Superposition doesn't work. The THD number at 1 V output at 1 kHz tells us very little about what the THD will be at 1 V at 5 kHz or 5 V at 1 kHz. In this case, I think the picture of the multi-tone test spectrum plot is worth a thousand numbers

Don't know if you are familiar with this old paper.
https://www.researchgate.net/public...n_of_Nonlinear_Distortion_Measurement_Methods

I disagree.

Non-linearity usually doesn't change all that much. So most often THD at 1 khz will be about the same as 5 khz. Some devices have it creep up a little at higher frequencies.

Here is a single tone and twin tone sweep of a couple DACs. You can see there is little change with frequency. With level we know, higher is higher distortion lower is less distortion. If highest levels are low enough in distortion, we don't need to worry about lower levels.

In this spectrogram, the background goes gray at -120 dbFS. You can see in the twin tone sweep both the harmonics and the difference signals.

 

[Blumlein 88]

IMD 18+19kHz may be more stressful but it is further from reality of music. Music signal is often -30 to -40dB in that frequency range anyway, so what stressing the DUT with this?
Multitone is closer to reality. And if the DUT is not stressed enough and results are similar to another DUT, well, then reality is that playing music with both DUTs will also sound similar, which is the end goal, right?
Please correct me if I am missing something.

This is back to the old conumdrum of whether we are testing for actual audibility or not. If a device is clean when stressed it is clean. If it is clean when lightly stressed it may be sometimes dirty and it may be sometimes audible. I hold the multi-tone testing is both less stressful and more messy to compare quality of two devices. I don't see the any advantage of it.

Is there something multi-tone tells me that twin tone testing doesn't?

 

[Matias]

This is back to the old conumdrum of whether we are testing for actual audibility or not. If a device is clean when stressed it is clean. If it is clean when lightly stressed it may be sometimes dirty and it may be sometimes audible. I hold the multi-tone testing is both less stressful and more messy to compare quality of two devices. I don't see the any advantage of it.

Is there something multi-tone tells me that twin tone testing doesn't?

I think the whole idea of measurements and testing is to help correlate it with better sounding music, higher fidelity.

As extreme examples, differentiating 2 devices by testing their noise floor 140 versus 150 db or THD of 50kHz tone are pointless as this is not even remotely on the recording and even less heard.

 

[pkane]

The AP has a delightful option- you can null out the test tones. You can also generate test tones that are centered in the FT bins so there's no windowing. Here's a couple examples:

View attachment 60376View attachment 60383

Perfect! That's pretty much how I thought about the problem: generate a multi-tone test signal centered perfectly on FFT bins, capture enough to process the FFT without averaging, then remove the signal leaving just the noise and distortions. I've started to run some tests using just this type of a software tool. I'll start posting the results a bit later. What I was really curious about was whether multi-tone test signal would reveal anything beyond a two-tone one.

 

[Blumlein 88]

I think the whole idea of measurements and testing is to help correlate it with better sounding music, higher fidelity.

As extreme examples, differentiating 2 devices by testing their noise floor 140 versus 150 db or THD of 50kHz tone are pointless as this is not even remotely on the recording and even less heard.

True enough, but what if we are comparing -140 db vs -70 db? Is 70 db enough (maybe, probably not quite). What about -140 db and -85 db? Is 85 db enough (well we are definitely closer)?

 

[pkane]

Still work in progress. I wanted to get some feedback on whether it might be useful and what other features might be interesting to add to it. For now, this looks like this:

The software allows you to chose the desired test signal, and then run a loop-back test and process the result. Test signals supported include single-tone, two-tone, and multi-tone signals. You can change the frequency and tone count numbers in the Test Signal selector, so the choice of test signals is infinite

The result is shown as blue (captured waveform) and white (original signal removed, leaving noise and distortions). TD+N is computed, as well as phase and frequency response.

So, now, I can run some tests to compare various test signals, with different loop-back devices and see what difference they make.

 

[scott wurcer]

I disagree.

I hope you don't disagree that in a non-linear system superposition does not apply.

 

[pkane]

A few test results using my preferred measuring device, Apogee Element24 in the loop-back configuration (96k/24bit).

1kHz single tone (note that TD+N in this case is the same as THD+N):
TD+N = -115dB

11kHz single tone, and a zoomed-in version around the 11k tone to show the distortions there (jitter?).
TD+N = -108dB

CCIF 18.5k/19.5k two-tone signal:
TD+N = -110dB

32-tone test signal:
TD+N = -114dB

500-tone test signal:
TD+N = -115dB

And for good measure, a 20,000 tone test signal
TD+N = -114dB

With more tones, it's also easy to extract the frequency and phase response of the DUT, so here's what it looks like with the 20k-tone signal (all captured in 11 seconds):

Phase:


Frequency response: