• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

How good is your PCM sine wave test tone? Niven's theorem is your friend.

Speedskater

Major Contributor
Joined
Mar 5, 2016
Messages
1,610
Likes
1,333
Location
Cleveland, Ohio USA
Would not this discussion have been more meaningful in 1983 than in 2023? Back than commercial digital technology (including ADC & DAC) was struggling to do 16x44.1.
 
OP
S

scottd

Member
Joined
Jul 10, 2022
Messages
36
Likes
16
Location
Texas
Yes, and that is great for number-crunching and theoretical analyses, but in the real world is far too limiting for frequency choice and means distortion and noise will be set by the ADC/DAC and analog signal chain, as always. That sets the practical spur floor irrespective of whether every point in the sine wave is rational or not.

Note an ideal N-bit converter has an actual spur floor (SFDR) of about 9N dB; the SNR set by quantization noise (and nothing else) is about 6.02N+1.76 dB.


Again, this is great for a theoretical analysis, getting mathematically "perfect", but in practice REW must operate on the input from a real-world ADC chain, and output through a real-world DAC, which will add noise and distortion of its own far surpassing the 53-bit level (for your IEEE FP example; some (many?) signal-processing programs use 64-bit integers internally so have more resolution).

Typo: the 53-bit (52 plus sign) IEEE 64-bit FP standard yields about 53*6.02+1.76 ~ 320 dB -- dB, not bits.

Your last point is key to me: this allows you to see the limitations of your digital processing chain. That can be very useful, but to use that DSP chain you're probably going to want to sweep more than just a few signals to look for things like filter overloads and limit cycles etc.


The point was that other means are used to generate a (broader) range of test signals that avoid spectral leakage. Again, the focus of most of the counter arguments (mine included) is on component and system testing, not just the bits inside the machine.

It is really cool to see the noise floor go to (essentially) 0 in the model or simulation! But of limited practical use, alas. Prime or relatively-prime signals are more useful for broader testing IME.

FWIWFM - Don
True. But I see so much discussion of technical fine points here that I am surprised to see the dismal state of test tones people are using. They are frightening! What's funny is the ones claiming high quality are usually the worst. What good are 24-bit samples if they contain errors that are clearly visible in the spectrum?

So now let's forget about this 3:1, 4:1, 6:1 special case business. Start with something simply, such as a 1000 Hz sine wave. 997 Hz is fine too. They are not really much different (at 44100, 1000 Hz has 220 harmonics and 997 Hz has 11025 harmonics). I chose 1000 Hz because I wanted to download and compare the type of test tones people here are using. I don't
Would not this discussion have been more meaningful in 1983 than in 2023? Back than commercial digital technology (including ADC & DAC) was struggling to do 16x44.1.
The other day I was reading a discussion of PCM for voice audio time division multiplexing. Bit depths of 7 or 8 were being considered. Not a bad choice, especially given it was written in 1948! So surely 16 bit samples are enough for any kind of audio. The point I am trying to make is that some of the discussion here assumes any well-known sine generator is good enough. That may be true for the real world hardware stuff being tested here. But as a software developer, it seems wrong that test tone files aren't bit perfect. So I am exaggerating to say the test tones in use here are horrible. But some, especially at bit depths of 32 and higher, have flaws that are visible in a spectrum view when analyzed directly and not passed through hardware. That doesn't make them unusable for the testing done here. But when validating a spectrum view, it's so annoying when the display is randomly bouncing around, or has more noise than expected.
 

DonH56

Master Contributor
Technical Expert
Forum Donor
Joined
Mar 15, 2016
Messages
7,834
Likes
16,496
Location
Monument, CO
True. But I see so much discussion of technical fine points here that I am surprised to see the dismal state of test tones people are using. They are frightening! What's funny is the ones claiming high quality are usually the worst. What good are 24-bit samples if they contain errors that are clearly visible in the spectrum?
IME this biggest offenders are the ones not following the relatively-prime conditions to make it better for testing. I do not know exactly what errors you are finding, but frequency choices causing spectral leakage is what I usually find, and that is often a numerical (FFT) problem rather than a real-world problem. Unfortunately the real world has all frequencies and does not insist on "nice" FFTs without windowing...
So now let's forget about this 3:1, 4:1, 6:1 special case business. Start with something simply, such as a 1000 Hz sine wave. 997 Hz is fine too. They are not really much different (at 44100, 1000 Hz has 220 harmonics and 997 Hz has 11025 harmonics). I chose 1000 Hz because I wanted to download and compare the type of test tones people here are using. I don't
Seems like a typo there... At 44.1 kS/s, 1000 Hz can have 22 tones (fundamental plus harmonics) and still meet Nyquist; 997 Hz still only gets you 22 tones (fundamental included) meeting Nyquist at 44.1 kS/s. There will be more terms (exceeding Nyquist) due to numerical roundoff and such but by that point they should be small and filtered before applying to an ADC or DAC.
The other day I was reading a discussion of PCM for voice audio time division multiplexing. Bit depths of 7 or 8 were being considered. Not a bad choice, especially given it was written in 1948! So surely 16 bit samples are enough for any kind of audio. The point I am trying to make is that some of the discussion here assumes any well-known sine generator is good enough. That may be true for the real world hardware stuff being tested here. But as a software developer, it seems wrong that test tone files aren't bit perfect. So I am exaggerating to say the test tones in use here are horrible. But some, especially at bit depths of 32 and higher, have flaws that are visible in a spectrum view when analyzed directly and not passed through hardware. That doesn't make them unusable for the testing done here. But when validating a spectrum view, it's so annoying when the display is randomly bouncing around, or has more noise than expected.
Being an analog guy, I am sort of used to noise being noise and not always known and stable. :) Given there are no 32-bit audio DACs (at least none with anywhere near ideal 32-bit performance) I am not surprised you see spurs at that level (let alone 64-bit). And yeah probably plenty of test files not meeting standards or pushing the precision of the processors. It would be good to list the good ones; there are likely a lot of 16-bit'ish files around not suitable for higher-resolution DACs. But for audio, at the end of the day performance is probably going to be set by the HW and not the DSP's word size. I have had to argue more than once with a system guy who did not understand why ADCs and DACs did not scale with processors. For a software-defined radio one PM spec'd a 32-bit, 10 GS/s ADC, and I had to try to explain why there was no such thing. Not sure he was ever convinced, and continued to issue RFPs for one... I suspect I'll be long retired before he gets one. Not sure you can even get a clock source clean enough let alone all the other issues.

For software, I have had no real issues achieving essentially ideal waveforms up to 24 bits or so, about as great as I have gone, using built-in functions to create the data array (ADA, C/C++, Fortran, Mathcad, Matlab, and Python -- the BASIC I used ages ago was limited to 16-bit precision). I don't doubt there are bad ones out there but I've usually just generated the signal myself as needed. For me the biggest issue was making sure the tones did not induce spectral leakage in the FFT.
 

JohnPM

Senior Member
Technical Expert
Joined
Apr 9, 2018
Messages
340
Likes
901
Location
UK
I don't believe we yet have any devices that manage 24-bit performance in the analog domain, the current best efforts are around 130 dB dynamic range and THD+N around -124 dB. Almost every device and driver has a 24-bit sample width restriction somewhere along its data path, even those claiming to be 32-bit.
 
OP
S

scottd

Member
Joined
Jul 10, 2022
Messages
36
Likes
16
Location
Texas
A while back, Amir posted this sample test file from AP that is presumably high quality. Yesterday I decided to try to rebuild it with increased bit depth. It's a multitone file containing 32 steady sine waves. Their frequencies are spread roughly log scale. The first thing I checked for is any repetition in the wav file using this free utility. The wav data is made of a 0x300000 byte pattern that repeats 22 times. The 0x300000 bytes are 0x10000 24-bit samples. Choosing 22 repetitions gets the play time as close as possible to 2 minutes.

I used command line FFT to dump the frequencies and phases using 8 significant digits and ignoring harmonics with power less than -100 dB. The '_single' added to the file name is because I used a single repetition of the pattern for efficiency:

>sv fft.8,-100 "D:\spectrumViewer\spectrumViewer-107\demo\multitone\wavefile\APx555 Multitone 32 192 khz 24 bit_single.wav"
Code:
Harmonic     Frequency           Power           angle
    (0-524288)          (Hz)            (dB)           (deg)
            80       14.6484    -24.71874421      0.06159566
           112       20.5078    -24.71874421      0.60781184
           144       26.3672    -24.71874417      0.75861226
           176       32.2266    -24.71874422      0.87396498
           224       41.0156    -24.71874419      0.70443636
           272       49.8047    -24.71874419      0.95871987
           352       64.4531    -24.71874419      0.61743527
           432       79.1016    -24.71874416      0.27648399
           544       99.6094    -24.71874416      0.21705898
           688       125.977    -24.71874417      0.12747603
           880       161.133    -24.71874420      0.14058409
          1088       199.219    -24.71874420      0.37903255
          1360       249.023    -24.71874425      0.42272619
          1728       316.406    -24.71874420      0.92870460
          2192       401.367    -24.71874421      0.14235732
          2736       500.977    -24.71874422      0.88125051
          3440       629.883    -24.71874417      0.91201400
          4368       799.805    -24.71874422      0.33120633
          5456       999.023    -24.71874422      0.81250651
          6832       1250.98    -24.71874420      0.27409667
          8736       1599.61    -24.71874419      0.23856877
         10928       2000.98    -24.71874418      0.32774481
         13648       2499.02    -24.71874416      0.47019342
         17200       3149.41    -24.71874421      0.21303107
         21840       3999.02    -24.71874419      0.75482235
         27312       5000.98    -24.71874422      0.72061276
         34400       6298.83    -24.71874421      0.91744560
         43696       8000.98    -24.71874419      0.04343103
         54608       9999.02    -24.71874421      0.90461145
         68272         12501    -24.71874420      0.27080336
         87376         15999    -24.71874418      0.54698210
        109232         20001    -24.71874420      0.29017821
32 harmonics found

Next, I modified SpectrumViewer to do multitone generation in addition to the existing single tone. The code adjusts each frequency so that the file plays a complete number of sine cycles. The level is normalized to max 1.0 then written to a wav file. I wrote only one section and not all 22. They resulting wav files have 1048576 samples. No doubt AP chose that number for FFT-friendliness.

I tested two ways: First, using the standard sine function. The angle handling is double precision. For pi loss, the angle is reset to its starting value each time the sample falls on an exact cycle start. The second way is using the MPFR library for both angle and sine calculation. I am pretty sure the MPFR versions are bit perfect because using 128 bit precision makes the same wav files as using 256 bit precision.

Comparing the original and new FTP dumps shows at least 7 digits match everywhere.

The FFT results speak for themselves:
Original AP file:
APx555 Multitone 32 192 khz 24 bit.png


Rebuilt file using MPFR math, saved as 64-bit fp:

make32_f64.png


The wav files are here (APx555_Multitone_32_192_24_test). Give them a try.
 

fpitas

Master Contributor
Forum Donor
Joined
Jul 7, 2022
Messages
9,885
Likes
14,191
Location
Northern Virginia, USA
Would not this discussion have been more meaningful in 1983 than in 2023? Back than commercial digital technology (including ADC & DAC) was struggling to do 16x44.1.
Sure. But it's still a good review, especially for those of us not familiar with the details of what's available now.
 

earlevel

Addicted to Fun and Learning
Joined
Nov 18, 2020
Messages
545
Likes
776
I used command line FFT to dump the frequencies and phases using 8 significant digits and ignoring harmonics with power less than -100 dB.
I probably have more comments on the post, but we still haven't determined what you mean by "harmonics". You didn't answer my question about it on the first page, and DonH56 asked about it again. It seems you're using a different definition of "harmonics" than the standard meaning, making it difficult to understand some of your points. You've mentioned things like 11025 harmonics for 1 kHz signal, which would require megahertz sampling, but you referenced 44.1 kHz. Hard to figure what that means.

(BTW, just a minor point, but there is no "power" involved here, I think you mean magnitude referenced to the fundamental.)
 

DonH56

Master Contributor
Technical Expert
Forum Donor
Joined
Mar 15, 2016
Messages
7,834
Likes
16,496
Location
Monument, CO
The spur floor (SFDR) of an ideal 24-bit signal is about -9*24 = -216 dB. That is from quantization noise only, no other noise or distortion.
 
OP
S

scottd

Member
Joined
Jul 10, 2022
Messages
36
Likes
16
Location
Texas
Bug fix: Double precision floating point as used in 64-bit wav files are good for 53 bits of precision. Yet Multitone Analyzer reports ENOB of only 52.7. I added a new set of wave files (APx555_Multitone_32_192_24_test_v2.7z) that corrects the problem, which affects only the 64-bit version. The problem was that while each of the 32 component tones was generated using 128 bit precision, the final summation was done using double precision math. The new version sums the 32 individual tones using MPFR 128-bit extended precision before converting to (53-bit) double precision.
Another change is that the phase angles from the FFT dump of the original AP file have been imported at a higher precision. Now the FFT dump of the new file matches that of the Audio Precision file to at least 14 digits.
The archive includes 24-bit and 32-bit signed integer versions, as well as 64-bit floating point.
I think I have proved my point that even the Audio Precision MT32 file is not bit perfect and can be improved, even within the 24-bit sample size limit. Take a look at what Multitone analyzer reports. All the stats are the same for the 24-bit file except for the DC level, and the DC level is improved (from -137.9 dB to -190 dB). Multitone Analyzer reports some stats not seen every day such as ENOB of 53 for the 64-bit file. Same with TD+N of -317.4 dB.
All my work is free and open source. To rebuild these high precision versions of the AP 32 tone file, just extract the archive, then go to the demo directory. From there, go to the multitone directory. Click on a batch file to build the corresponding wav file. Then use your favorite utility to check the quality. Hopefully these examples will help others improve their test files. But I did it as a personal challenge more than anything.
 
OP
S

scottd

Member
Joined
Jul 10, 2022
Messages
36
Likes
16
Location
Texas
Since the noise floor of a real system is limited by resolution (number of bits) and the analog signal chain, I can live with truncating the sine wave to 1/2^N. More important in the test and measurement world is the ability to choose frequencies that bin perfectly in the FFT, a function of sampling rate and FFT length. The means to choose such frequencies, and thus avoid windowing in the FFT, are described in various IEEE Standards e.g. 1057 (waveform recorders, the grandfather), 1241 (ADCs), 1658 (DACs) etc. as well as numerous other references various manufacturers use. I have lost track of the IEEE numbers and revs since it's been a decade or so since I was involved with them.
I think what you are saying is equivalent to using a whole number of sine cycles in the test pattern. At first, I was accidentally truncating the tones at an arbitrary point, such as 10 seconds. This doesn't affect high frequencies much. If a high frequency tone of 200,000 cycles comes up short by half a cycle, it's not so noticeable. But the low frequencies are greatly affected. If a low frequency tone of 200 cycles is truncated by half a cycle, thew difference is bigger.
The multitone patterns included in Multitone Analyzer seem to have this problem.
 
OP
S

scottd

Member
Joined
Jul 10, 2022
Messages
36
Likes
16
Location
Texas
With 24-bit samples, you have 144 dB of dynamic range. Lose 3 dB due to dither noise is not important as even state of the art equipment has far more noise. And for FFTs, we can use FFT gain to go to any level of noise floor we want.
I agree. I made the error-free files as a personal challenge. The easiest way to get to the top of Mt Everest is helicopter. But some people climb it for the challenge.
 

restorer-john

Grand Contributor
Joined
Mar 1, 2018
Messages
12,579
Likes
38,273
Location
Gold Coast, Queensland, Australia
The easiest way to get to the top of Mt Everest is helicopter.

Apparently not. :) It's only been done a few times, with specially stripped down helicopters of enormous power. Not an easy or safe job.

"Except for Didier DelSalle who flew to the top of Mount Everest, helicopters won’t land any higher than 21,000ft/6,400m. This is because this is the location of Camp 2, which has flat terrain and designated landing areas for helicopters."
 
OP
S

scottd

Member
Joined
Jul 10, 2022
Messages
36
Likes
16
Location
Texas
FFT window is normally applied to any signal that has discontinuities, i.e., one that isn't infinitely periodic (which includes all real signals). FFT windows cause leakage, or spreading of energy, to the neighboring bins. If one wants to eliminate such leakage, test tone frequency must be exactly periodic with the length of the FFT which is easily accomplished by centering the test tone frequency precisely in the middle of an FFT bin.
I agree. Isn't this the same as saying the test pattern should contain whole sine wave cycles, and not truncate a sine wave mid cycle? Of the Multitone Analyzer pre-built test files, I looked at 'Multitone 20 44100 64bits'. Viewing a the spectrum in real time shows a jump every 5 seconds. The file itself has one unique section of 131,072 samples. It's followed by the same pattern, but truncated to 89,424 samples. That doesn't seem right.
 
OP
S

scottd

Member
Joined
Jul 10, 2022
Messages
36
Likes
16
Location
Texas
I probably have more comments on the post, but we still haven't determined what you mean by "harmonics". You didn't answer my question about it on the first page, and DonH56 asked about it again. It seems you're using a different definition of "harmonics" than the standard meaning, making it difficult to understand some of your points. You've mentioned things like 11025 harmonics for 1 kHz signal, which would require megahertz sampling, but you referenced 44.1 kHz. Hard to figure what that means.

(BTW, just a minor point, but there is no "power" involved here, I think you mean magnitude referenced to the fundamental.)

I probably have more comments on the post, but we still haven't determined what you mean by "harmonics". You didn't answer my question about it on the first page, and DonH56 asked about it again. It seems you're using a different definition of "harmonics" than the standard meaning, making it difficult to understand some of your points. You've mentioned things like 11025 harmonics for 1 kHz signal, which would require megahertz sampling, but you referenced 44.1 kHz. Hard to figure what that means.

(BTW, just a minor point, but there is no "power" involved here, I think you mean magnitude referenced to the fundamental.)
I mis-spoke. What I am talking about is better described here. A 1000 Hz tone encoded at 44100 samples per second, with no partial cycles has 220 harmonics. That's because 441 samples encodes 10 complete cycles before the pattern repeats. 441 / 2 gives the potential number of harmonics (or whatever you want to call the component sine waves). So there are 220 component sine waves. Because of the odd / even relation of the two numbers, 220 is the final number. A big list of the number of component sine waves in a PCM encoded sine wave is here. It is interesting to note that I cannot find anywhere in my DSP text books where this simple calculation is described.
 

dualazmak

Major Contributor
Forum Donor
Joined
Feb 29, 2020
Messages
2,817
Likes
2,949
Location
Ichihara City, Chiba Prefecture, Japan
Not only the sine waves but also (for example) white noise and pink noise, I (we) need very accurate and QC-ed test signals for measurement and validation in our audio project.

In my multichannel PC-DSP-based multi-driver multi-amplifier fully active stereo project, I have been using "Sony Super Audio Check CD" (i.e. 16 bit 44.1 kHz, 22.05 kHz in L & R) all the way through.

I definitely need strictly accurate and QC-ed test tone signals in my project, and "Sony Super Audio Check CD" is the best fit (and excellent enough) at least for my project. Attached please find the CD booklet (liner notes) of English translation by myself, and the spectrum of all the tracks analyzed (by myself) using Adobe Audition 3.0.1 and Music Scope 2.0.1 can be found in post here.

In the liner notes they (Sony) described "The recorded original signals were prepared mainly by computer simulation in ultra-high precision of (99.999999999) or higher precision, test noises which satisfy over 3σ (sigma, standard deviation) Gaussian distribution,, and high-precision analog oscillator and a high-quality music source are used."

Just for one typical example case, I have developed my own rather primitive but reliable/reproducible "cumulative white noise averaging method" for Fq response measurements in my project; in this case the use of very strictly prepared and QC-ed/validated white noise (20 Hz - 20 kHz) should be indispensable (please refer to my summary post here and here).

In any way, we should be always very much careful about the accuracy/degree-of-precision/QC of the test signals/tones in our audio measurements, and our own "validation and proof" of such signals/tones would be frequently needed.

Furthermore, I believe that we should not over-trust any black-box type measurement software and its test tones/signals unless we could fully understand it and objectively "validate" the processing and the tones/signals.

In case if you would be seriously interested in the contents of "Sony Super Audio Check CD", please simply PM me writing your wish.
 

Attachments

  • SONY Super Audio Check CD_ Booklet_English by dualazmak_rev03.pdf
    2.3 MB · Views: 55
Last edited:
Top Bottom