FFT Size impact on Noise level in Virtins Multi Instrument 3.8 ? - FIXED in v3.9

[Rja4000]

Hi
I thought I'd create a new thread about this topic.

I originaly posted my question here
Detailed analysis will be in next post

Here-under is the original question:

Yes. Here is the amount of "FFT gain" (measurement noise reduction) for different number of audio samples used:

32K = 42 dB
256K = 51 dB
1 million = 57 dB

The actual dB is a few dB different since it also includes the effect of the FFT Window. But is not material in grand scheme of things.

Hi

This "gain" is the level drop for noise in the display (or for the distortion analysis).
But it's not impacting SNR or SINAD, or does it ?

Actually, I see some impact of FFT size on SNR, but that's nowhere near linear.
And I have no clue what's causing that.
Example of a Loopback FFT Size/SNR

And this looks quite random, on top of it: I've seen 128k being higher than 32k for other configurations, as an example.

Does anyone know why ?
At lower FFT size, it looks like there is high low frequency noise, due to the fuzzy frequency resolution.
But why do we see variation above 16k points ?

Notes: sampling frequency in this example was 48kHz
I get the same figures for the same config anytime
I also read the exact same values from Multi Instrument and from REW

 

[Rja4000]

I've performed more analysis

First, I realised that the exact frequency in the signal generator should be selected as a function of sampling rate AND of the FFT Size, to avoid "spectral leak"

Here is an example of a test with a FFT size of 2048 points
First with a fixed 1000Hz sine

The result is horrible, isn't it ?
But that is most probably due to the 23.4375Hz "Resolution" for that low FFT size.
(See this value in bottom left of the spectrum window)

By properly selecting a frequency close to 1kHz, but in function of both Sampling rate and FFT Size, we get the following

Much more in line with what I expected !
Frequency is now 1007.8125Hz

Now, I repeated the same test for each Window size up to 1024k points:

FFT Size 4k points

FFT Size 8k points

FFT Size 16k points

FFT Size 32k points

FFT Size 64k points
(note that it's more than sampling frequency)

FFT Size 128k points

FFT Size 256k points

FFT Size 512k points

FFT Size 1024k points

We see the noise graph becoming lower and lower in the Spectrum display.
This was to be expected.

What was NOT expected for me are the changes in measured Noise Level and SNR values.
(Also, at 1024k points, everything crashes. That may be linked to the PC CPU Load.)

I've recorded all those values in a table (rounding to integers, to take into account the test repeatability)

(Note H2-H5 are the values shown as 'fxRMS' above minus the level, to have their relative value vs signal, as in THD)

THD is pretty stable (as from an FFT Size of 8k),
but SNR and NL (Noise Level) vary quite a lot !
And so does SINAD, of course, since it's mainly linked to noise here.


Tests with REW

I repeated the same tests with REW
Same configuration exactly.

Note that REW, per default, adapts the frequency in the Sine generator to the FFT Size if you check the "Lock Frequency to RTA FFT" checkbox.
(Multi Instrument is also doing that if you want)

I have all sizes from 8k to 1024k, but I'll post just 3 here:

REW FFT Size 8k points

REW FFT Size 64k points

REW FFT Size 1024k points

(Note the frequencies: they were selected automatically by REW. They are the same I computed for MI)

REW gives much more stable values
THD values are pretty much the same than Multi Instrument
But noise varies much less.

Bottom line (for now)

There is probably a problem in the way Virtins Multi Instrument (3.8) computes the Noise values when FFT Window Size changes.
The problem is visible here for FFT Window size >64k points.
Fortunately, for standard FFT points number around Sampling Frequency value, the results seem OK and are comparable to REW.

I like Virtins Multi Instrument a lot.
The Value of the 'pro' version is excellent, in my opinion. Automation is also very handy. (A must for me, actually !)
So having it measuring accurate values is key for me.

@VIRTINS: Any comment ?
Is there something I missed ?


Test conditions
RME ADI-2 Pro fs in Loopback
Output: Balanced phone mode in Lo power mode (+13dBu), level set to -2.5dBFS
Input: Both XLR Analog inputs, +13dBu range
Analog input "M/S proc" on (average of both analog inputs on left channel)
Left channel ("M") measured.
Spectrum band-filtered to 20Hz-20kHz, 4 averages
No weighting
@MC_RME: The ADI-2 Pro is a beast !

 

[DonH56]

It looks like the noise floor of your system is around 110~113 dB so increasing FFT size does not help below that.

Processing gain is logarithmic, not linear, at roughly 10log(N/2) for N samples. Lots of online references, here is one: https://www.designnews.com/aerospace/where-does-fft-process-gain-come/100022666833951

 

[Rja4000]

It looks like the noise floor of your system is around 110~113 dB so increasing FFT size does not help below that.

Processing gain is logarithmic, not linear, at roughly 10log(N/2) for N samples. Lots of online references, here is one: https://www.designnews.com/aerospace/where-does-fft-process-gain-come/100022666833951

Well, in the measurements from the second post, I think the noise is rather around -115dB~-116dB below signal here (which is -2.4dBFS, so "Noise Level" should be -117~-119dBFS).

Other than that, I'm aware of the processing gain... which we see in the plots: The noise seems to get lower as the FFT size increases.
But, as long as the level remains the same, the SNR figure should NOT change, as far as I know. And it does, in Multi Instrument.

 

[Rja4000]

One good thing with Virtins is that their support is answering questions, usually with a lot of explanations.

Here, they started to answer:
https://www.virtins.com/forum/viewtopic.php?f=7&t=1382#p2068
Which explains why we have that hump in measurements on low frequencies at 2k points, when frequency is not aligned with FFT boxes.
And how to avoid it.

 

[Rja4000]

They've actually posted more details.
Well worth reading.

I still don't have a definitive explanation why we see that step in Noise Level and SNR between a FFT window size of 32768 and 131072 though.

I'll perform more tests when possible.

@SIY: As a Multi Instrument user, did you ever noticed that behaviour ?

 

[Rja4000]

Virtins support actually found some error:

We have (...) done a detailed investigation and found that these strange behaviours occur only when the signal level (S) is roughly 165 dB or more above the "apparent" noise level. This condition will be satisfied only when FFT size is very large, the noise level of the test is extremely low, and the signal level is close to the maxium.
The cause of the issue has something to do with how to maintain the required precision during numerical computation.
We have improved the code in our internal version and it will be released soon in MI 3.9.

https://www.virtins.com/forum/viewtopic.php?f=7&t=1382&p=2108#p2100

Thanks to them !

Not everyone is willing to admit that their own product may require improvement. And be ready to improve it.

My favorite measurement software may now become even better

 

[Rja4000]

Release 3.9 was posted on April 23rd 2020.
It now fixes this bug.

 

[audio_tony]

I appreciate this topic is over 2 years old now, but @Rja4000 this AP article might be of interest to you.