How good is your PCM sine wave test tone? Niven's theorem is your friend.

[dualazmak]

All the highest precision industry standard test CDs (16bit) I have and use, deliberately ensure the spot frequency is prime to the sample rate. ie, not 1000Hz, it is 997Hz. Not 16kHz but 16001Hz. The square wave is not 1kHz, it is 1002.27Hz.

Yes, I assume your point is related also to the two descriptions in liner notes of "Sony Super Audio Check CD"; they wrote as follows.

 

[pkane]

I agree. Isn't this the same as saying the test pattern should contain whole sine wave cycles, and not truncate a sine wave mid cycle? Of the Multitone Analyzer pre-built test files, I looked at 'Multitone 20 44100 64bits'. Viewing a the spectrum in real time shows a jump every 5 seconds. The file itself has one unique section of 131,072 samples. It's followed by the same pattern, but truncated to 89,424 samples. That doesn't seem right.

There are no pre-built files that are included with Multitone, everything is generated on the fly. Here's a 20 tone multitone with 10 averages generated and analyzed by MT.

SFDR of 336 dB. Spectrum displayed with a rectangular window:

And here's a 555 multitone

 

[DonH56]

I think what you are saying is equivalent to using a whole number of sine cycles in the test pattern. At first, I was accidentally truncating the tones at an arbitrary point, such as 10 seconds. This doesn't affect high frequencies much. If a high frequency tone of 200,000 cycles comes up short by half a cycle, it's not so noticeable. But the low frequencies are greatly affected. If a low frequency tone of 200 cycles is truncated by half a cycle, thew difference is bigger.
The multitone patterns included in Multitone Analyzer seem to have this problem.

I defer to the IEEE Standards; I believe the AES uses something similar. But yes you must ensure the sine wave(s) are an integer number of complete cycles within the number of samples you acquire for the FFT. The idea is to avoid any discontinuities that require windowing in the FFT. In addition you want them all relatively prime to each other and the sampling rate to avoid multiple tones in each FFT bin or spectral leakage spanning multiple bins. This has been known for many years (many decades) but not always implemented correctly, sometimes through ignorance or mistakes, sometimes through SW/HW limitations. And sometimes test requirements specify frequencies that do not meet those criteria (at least in the RF world).

As for Multitone, @pkane has discussed his scheme many times on ASR, and his current program correctly calculates multiple tones with noise floor at the numerical limit. I think he has several ways to generate them, or you may be using an older version, but I know he has shown results equivalent to yours. Most any program should do that. Real HW and various SW implementations use fewer bits, often integers, for efficiency in processing and memory use. A lot of DSP chips are integer, not floating-point, for example.

In deference to @dualazmak's post, creating a good noise source numerically has long been a somewhat challenging problem. In the late 1970's I built a small pink noise generator from a kit schematic and found it exhibited tones and was not large (long) enough (not enough bits in the shift register) to produce a decent 10 Hz lower end. I ended up learning enough to roll my own and built it using a bunch of SSI/MSI TTL chips on a board about the size of a piece of paper (8.5" x 11"). These days a little chip offers better performance and much more flexibility, but you still have to watch out for the sort of bugs delineated in this thread.

Edit: Paul beat me to it, natch...

 

[earlevel]

I mis-spoke. What I am talking about is better described here. A 1000 Hz tone encoded at 44100 samples per second, with no partial cycles has 220 harmonics. That's because 441 samples encodes 10 complete cycles before the pattern repeats. 441 / 2 gives the potential number of harmonics (or whatever you want to call the component sine waves). So there are 220 component sine waves. Because of the odd / even relation of the two numbers, 220 is the final number. A big list of the number of component sine waves in a PCM encoded sine wave is here. It is interesting to note that I cannot find anywhere in my DSP text books where this simple calculation is described.

OK, thanks for clearing that up.

That also clears up what you said here:

When the sine frequency doesn't divide the sample rate, there will sampleRate/4 harmonics, assuming the sample rate is even and the sine frequency is integer. So a 1KHz sine at 44100 has 11025 harmonics. One fundamental plus 11024 due to quantization noise. That gives a self-dithering effect. That is why the coprime sample rate/frequency pairs are used. With that much self dithering, added dithering hardly changes the spectrum. So it makes the dithered/undithered forms essentially the same.
So why not take it to an extreme and choose a frequency such as 997.001 Hz? That gives 11025000 harmonics for super smooth self dithering. Of course the problem with that is the resulting wav file is 1000 seconds long (or integer multiples of that).

Except that's I'd say the problems is that you have no control over the PDF of the noise, and if your point is to dither to a target word length, you don't have control over the magnitude of the dither. And, the sine calculations are almost certainly being done in floating point, so the noise level is rising and falling with the sine itself, making the error completely unrelated to dither.

I hope you don't read that as taking you to task about a trivial side point. I recognize your point was a side observation, I'm just making a side observation that it really doesn't work out that way.

 

[Digital Mastering System]

All the highest precision industry standard test CDs (16bit) I have and use, deliberately ensure the spot frequency is prime to the sample rate. ie, not 1000Hz, it is 997Hz. Not 16kHz but 16001Hz. The square wave is not 1kHz, it is 1002.27Hz.

Back when we made test tones for the 3M Digital Mastering System, we would make a EPROM with the sine wave in it. We would try to make each sin different so eventually we would test nearly all the numbers in 16 bits. For example, if we had a 16K Byte (16384) EPROM and we wanted about 1K Hz at 50k samples/sec, we would put a prime number of sin waves in that sucker. In this case, with about 50 samples per 1k period, for exactly 1K Hz we could get 327.68 periods in that EPROM. But fractions aren't allowed, because the EPROM just repeats - so we chose to put exactly 327 periods in. 327 is a prime number, so no two 1K ish periods will have the same set of 50 or so numbers. The actual test frequency would be in this case 997.9248 Hz. Then on top of that we added normally distributed dither.
Longer memory is better as you can get closer to the desired frequency and more and more unique sin wave periods - assuming you use the prime rule.

 

[scottd]

OK, thanks for clearing that up.

That also clears up what you said here:

Except that's I'd say the problems is that you have no control over the PDF of the noise, and if your point is to dither to a target word length, you don't have control over the magnitude of the dither. And, the sine calculations are almost certainly being done in floating point, so the noise level is rising and falling with the sine itself, making the error completely unrelated to dither.

I hope you don't read that as taking you to task about a trivial side point. I recognize your point was a side observation, I'm just making a side observation that it really doesn't work out that way.

It's really just a software challenge to me. I'm retired and getting old. At first I wanted all garage hobbies for retirement, like fabrication and machining. But my blurry eye sight takes some of the fun out of those things. So now I spend part of the day playing with the computer, with the help of a big monitor and reading glasses. Since starting full time coding work in 1982, I'v had some side coding project or another going on just for fun.

 

[restorer-john]

Back when we made test tones for the 3M Digital Mastering System, we would make a EPROM with the sine wave in it. We would try to make each sin different so eventually we would test nearly all the numbers in 16 bits. For example, if we had a 16K Byte (16384) EPROM and we wanted about 1K Hz at 50k samples/sec, we would put a prime number of sin waves in that sucker. In this case, with about 50 samples per 1k period, for exactly 1K Hz we could get 327.68 periods in that EPROM. But fractions aren't allowed, because the EPROM just repeats - so we chose to put exactly 327 periods in. 327 is a prime number, so no two 1K ish periods will have the same set of 50 or so numbers. The actual test frequency would be in this case 997.9248 Hz. Then on top of that we added normally distributed dither.
Longer memory is better as you can get closer to the desired frequency and more and more unique sin wave periods - assuming you use the prime rule.

So basically a sine lookup table in EPROM?

The CBS CD-1 standard test disc states the following in relation to the frequencies chosen:

 

[scottd]

I defer to the IEEE Standards; I believe the AES uses something similar. But yes you must ensure the sine wave(s) are an integer number of complete cycles within the number of samples you acquire for the FFT. The idea is to avoid any discontinuities that require windowing in the FFT. In addition you want them all relatively prime to each other and the sampling rate to avoid multiple tones in each FFT bin or spectral leakage spanning multiple bins. This has been known for many years (many decades) but not always implemented correctly, sometimes through ignorance or mistakes, sometimes through SW/HW limitations. And sometimes test requirements specify frequencies that do not meet those criteria (at least in the RF world).

As for Multitone, @pkane has discussed his scheme many times on ASR, and his current program correctly calculates multiple tones with noise floor at the numerical limit. I think he has several ways to generate them, or you may be using an older version, but I know he has shown results equivalent to yours. Most any program should do that. Real HW and various SW implementations use fewer bits, often integers, for efficiency in processing and memory use. A lot of DSP chips are integer, not floating-point, for example.

In deference to @dualazmak's post, creating a good noise source numerically has long been a somewhat challenging problem. In the late 1970's I built a small pink noise generator from a kit schematic and found it exhibited tones and was not large (long) enough (not enough bits in the shift register) to produce a decent 10 Hz lower end. I ended up learning enough to roll my own and built it using a bunch of SSI/MSI TTL chips on a board about the size of a piece of paper (8.5" x 11"). These days a little chip offers better performance and much more flexibility, but you still have to watch out for the sort of bugs delineated in this thread.

Edit: Paul beat me to it, natch...

I have yet to see a sample wave file here that is at the numerical limit of accuracy for 24-bit or better, as measured by command line FFT (plotting FFT rounds the values to fit the nearest pixel, loosing detail). An example is the Audio Precision test tone linked earlier. It is very good, though it's 24-bit integer. The compiler sine and angle limitations don't show up until 32-bit, in my experience. But even so, the AP sample file can't be called perfect. When I rebuild it, FFT shows the 32 generated harmonics are absolutely identical. The largest quantization harmonic is essentially the same, with the rebuilt file fractionally better. The DC offset is the most significant difference. The AP file has DC offset -147.5392248 dB. The rebuild has DC offset -199.6439952 dB. I know these differences are harmless, but it's incorrect to say the AP file is at the limit of 24-bit numerical accuracy. As for REW and the others, I am sure they are good enough for their intended use. But they are no where near as good as the AP example, despite the AP example's imperfections. Command line FFT is a good thing because the results are black and white and easy to interpret.

 

[scottd]

If people here are asked to evaluate the quality of a steady (periodic) test tone, no doubt FFT is going be the tool of choice. It's a natural fit because because in the case of a periodic signal and no overlapping windows, no windowing functions, and no averaging, FFT extracts enough information to rebuild the signal. So what is the natural way to generate the signal in the first place? I am pretty sure most every test tone in widespread use here is generated by adding sines of various phase and amplitudes, using a compiler sine function and a pi value rounded to double precision. Otherwise, the quality would be better. If FFT is how you analyze it, isn't the natural way to generate it Inverse FFT?

Well I've been a busy beaver testing the idea of multitone generation using IFFT. It works well. Generating a test tone using IFFT is much faster than MPFR extended precision sine when more than a couple of tones are being combined. the inverse (complex to real) function in the FFTW library avoids the compiler sine inaccuracy problems, and avoids the pi inaccuracy problem. Well for the most part. The exception is when a non-zero phase is needed. The sine/cosine of that phase angle is needed for preparing the IFFT input. In my test, I avoid double precision for that part so it doesn't matter. I am using a double precision build of FFTW, so it can't be expected to match MPFR extended precision in every case (I am using 128 bit for the MPFR precision). But even so, the IFFTW results are amazing. They match the MPFR 128-bit results to 53 out of 54 possible bits.

If someone wants to generate high quality test tones without the slow performance of MPFR extended precision library calls, consider IFFTW. I put all my code in a stand-alone utility along with batch files for generating many examples. The utility still defaults to MPFR 128 bit, but a command line option switches it to IFFTW. It is here: PerfectSineWaves. Clickbait name I know.

 

[pkane]

I have yet to see a sample wave file here that is at the numerical limit of accuracy for 24-bit or better,
...
As for REW and the others, I am sure they are good enough for their intended use. But they are no where near as good as the AP example, despite the AP example's imperfections. Command line FFT is a good thing because the results are black and white and easy to interpret.

So, you're saying this is not good enough for 24 bits? This file contains exactly 64k samples, so no averaging or windowing is done to display this FFT:

 

[pkane]

If people here are asked to evaluate the quality of a steady (periodic) test tone, no doubt FFT is going be the tool of choice. It's a natural fit because because in the case of a periodic signal and no overlapping windows, no windowing functions, and no averaging, FFT extracts enough information to rebuild the signal. So what is the natural way to generate the signal in the first place? I am pretty sure most every test tone in widespread use here is generated by adding sines of various phase and amplitudes, using a compiler sine function and a pi value rounded to double precision. Otherwise, the quality would be better. If FFT is how you analyze it, isn't the natural way to generate it Inverse FFT?

Well I've been a busy beaver testing the idea of multitone generation using IFFT. It works well. Generating a test tone using IFFT is much faster than MPFR extended precision sine when more than a couple of tones are being combined. the inverse (complex to real) function in the FFTW library avoids the compiler sine inaccuracy problems, and avoids the pi inaccuracy problem. Well for the most part. The exception is then a non-zero phase is needed. The sine/cosine of that phase angle is needed for preparing the IFFT input. In my test, I avoid double precision for that part so it doesn't matter. I am using a double precision build of FFTW, so it can't be expected to match MPFR extended precision in every case (I am using 128 bit for the MPFR precision). But even so, the IFFTW results are amazing. They match the MPFR 128-bit results to 53 out of 54 possible bits.

If someone wants to generate high quality test tones without the slow performance of MPFR extended precision library calls, consider IFFTW. I put all my code in a stand-alone utility along with batch files for generating many examples. The utility still defaults to MPFR 128 bit, but a command line option switches it to IFFTW. If is here: PerfectSineWaves. Clickbait name I know.

I'm glad you're discovering the power of FFT test tone generation, but what puzzles me is that you think that nobody is doing this.

 

[DonH56]

I have yet to see a sample wave file here that is at the numerical limit of accuracy for 24-bit or better, as measured by command line FFT (plotting FFT rounds the values to fit the nearest pixel, loosing detail). An example is the Audio Precision test tone linked earlier. It is very good, though it's 24-bit integer. The compiler sine and angle limitations don't show up until 32-bit, in my experience. But even so, the AP sample file can't be called perfect. When I rebuild it, FFT shows the 32 generated harmonics are absolutely identical. The largest quantization harmonic is essentially the same, with the rebuilt file fractionally better. The DC offset is the most significant difference. The AP file has DC offset -147.5392248 dB. The rebuild has DC offset -199.6439952 dB. I know these differences are harmless, but it's incorrect to say the AP file is at the limit of 24-bit numerical accuracy. As for REW and the others, I am sure they are good enough for their intended use. But they are no where near as good as the AP example, despite the AP example's imperfections. Command line FFT is a good thing because the results are black and white and easy to interpret.

I don't have an AP and the type of work I do is not audio (so no .wav files). I certainly never said the AP is at the limit of 24-bit accuracy as I do not know. As for the rest, tones generated at the limit of numerical accuracy have been around for decades, whether or not implemented by audio test gear. Of course, when I started playing this game, it was 6-bit, 1 GS/s ADCs and numerical accuracy was not all that hard. But again I defer to IEEE and using their method have created various test signals at numerical limits of accuracy. They were usually generated on the fly and for much wider bandwidth and center frequencies than audio. But we were always limited by the processor's (DSP or something like it) and data converters' word lengths in the real world. The largest dynamic range in any system I helped design was only around 160 dB, a far cry from 320 dB or so. The test signals might (and did) hit 320 dB but we'd never see it.

AFAIK Paul @pkane also generates signals on the fly. A HW system may use a ROM but these days even they tend to calculate and load the signal in memory on the fly. I know some (many, most, whatever) high-speed RF systems (digital RF memories, DRFM, phased array radar systems, and such) still use hardwired versions for speed given the systems operate well into the GS/s range.

What you are doing is great stuff but I have not really been following this thread. Have you documented the algorithm/methodology for generating your test signals? Are you planning to publish it anywhere? So far, again bearing in mind I check in just now and then, it seems to be mostly "you are all wrong, I have a better way" posts. I need to wade back at some point to understand your method, but it will have to wait for a break in my day job. Being here is just for "fun".

 

[restorer-john]

So far, again bearing in mind I check in just now and then, it seems to be mostly "you are all wrong, I have a better way" posts.

@DonH56 , you're so wrong about that.

 

[KSTR]

-147.5392248 dB [...] -199.6439952 dB

Stating those values with 10(!) significant digits seem to indicate a slight obsession with precision for precision's sake?
I'm not belittling your efforts but I don't think it has a place in current audio engineering and science.

 

[KSTR]

I am pretty sure most every test tone in widespread use here is generated by adding sines of various phase and amplitudes, using a compiler sine function and a pi value rounded to double precision

For single sines and for small number of sines in a multitone the simple sin() or cos() direct time domain generators are good enough.
But agreed, for dense multitones, up to the point of periodic noise with gaps (areas of unpopulated bins) iFFT will be better.

 

[KSTR]

It is here: PerfectSineWaves.

I just downloaded the archive... the .WAV files in the "perfect" demo folder are completely useless:

• All the 3_xyz.wav (for example) files are actually the same data with sample rate fields just patched to give the nominal frequency and play duration of 2minutes when played at those arbitrary (non-standard) sampe rates.
• The data is not scaled down properly and will give inter-sample overs from hell. You simply cannot use full scale values for this, rather you have to allow headroom so that a perfect sinc() filtered reconstruction does not clip.

Not sure what your intention was with these files...

 

[pkane]

For single sines and for small number of sines in a multitone the simple sin() or cos() direct time domain generators are good enough.
But agreed, for dense multitones, up to the point of periodic noise with gaps (areas of unpopulated bins) iFFT will be better.

Don't know why @scottd thinks that large multitones are generated with sines. In most cases, they are not (certainly not in Multitone), and the main reason is not the accuracy of the sine computation (or even the speed), but the need to phase-optimize these to produce the lowest possible crest factor for measurements.

 

[earlevel]

If people here are asked to evaluate the quality of a steady (periodic) test tone, no doubt FFT is going be the tool of choice. It's a natural fit because because in the case of a periodic signal and no overlapping windows, no windowing functions, and no averaging, FFT extracts enough information to rebuild the signal. So what is the natural way to generate the signal in the first place? I am pretty sure most every test tone in widespread use here is generated by adding sines of various phase and amplitudes, using a compiler sine function and a pi value rounded to double precision. Otherwise, the quality would be better. If FFT is how you analyze it, isn't the natural way to generate it Inverse FFT?

I don't know, I think the FFT is probably fairly popular for this use, when multiple harmonics are needed. For single tones, there isn't much symmetry to exploit in an FFT (and you only need generate a quarter wave, if cycles are more important than code simplicity). For multiple harmonics, of course the FFT is the way to go, and I'm sure that's pretty common. Here's some of my code to generate arbitrary harmonics, for instance (this is optimized for "real-time" synthesis at arbitrary frequency, for my wavetable oscillators, not test tone precision): WaveUtils updated

 

[earlevel]

It is here: PerfectSineWaves.

I just downloaded the archive... the .WAV files in the "perfect" demo folder are completely useless:

• All the 3_xyz.wav (for example) files are actually the same data with sample rate fields just patched to give the nominal frequency and play duration of 2minutes when played at those arbitrary (non-standard) sampe rates.
• The data is not scaled down properly and will give inter-sample overs from hell. You simply cannot use full scale values for this, rather you have to allow headroom so that a perfect sinc() filtered reconstruction does not clip.

Not sure what your intention was with these files...

For me (Mac), none of the files in the "perfect" folder have a proper RIFF header, won't open in any wave editor (RX 10, Audacity), I looked at a small one (3_1_20Hz.wav), it's all zeros. Looking at all the wav files in the distribution, some will play, most aren't recognized as wav files, indicating lack of a proper header.

PS—OK, didn't notice it was a 7z file, not zip. Mac OS (via its Archive Utility) didn't complain and unpacked it with a double click, but obviously not well. I used Unarchiver and it did the job correctly.

 

[danadam]

I looked at a small one (3_1_20Hz.wav), it's all zeros.

Maybe corrupted download? Seems fine here:

]$ hexdump -Cn128 3_1_20Hz.wav
00000000  52 49 46 46 64 38 00 00  57 41 56 45 66 6d 74 20  |RIFFd8..WAVEfmt |
00000010  10 00 00 00 01 00 01 00  3c 00 00 00 78 00 00 00  |........<...x...|
00000020  02 00 10 00 64 61 74 61  40 38 00 00 00 00 ff 7f  |....data@8......|
00000030  01 80 00 00 ff 7f 01 80  00 00 ff 7f 01 80 00 00  |................|
00000040  ff 7f 01 80 00 00 ff 7f  01 80 00 00 ff 7f 01 80  |................|
00000050  00 00 ff 7f 01 80 00 00  ff 7f 01 80 00 00 ff 7f  |................|
00000060  01 80 00 00 ff 7f 01 80  00 00 ff 7f 01 80 00 00  |................|
00000070  ff 7f 01 80 00 00 ff 7f  01 80 00 00 ff 7f 01 80  |................|
00000080

]$ sndfile-info 3_1_20Hz.wav
========================================
File : 3_1_20Hz.wav
Length : 14444
RIFF : 14436
WAVE
fmt  : 16
  Format        : 0x1 => WAVE_FORMAT_PCM
  Channels      : 1
  Sample Rate   : 60
  Block Align   : 2
  Bit Width     : 16
  Bytes/sec     : 120
data : 14400
End

----------------------------------------
Sample Rate : 60
Frames      : 7200
Channels    : 1
Format      : 0x00010002
Sections    : 1
Seekable    : TRUE
Duration    : 00:02:00.000
Signal Max  : 32767 (-0.00 dB)