I managed to get around the problems related to the players and finally carry out the test I wanted.
I compared the Windows resampler to the venerable iZotope RX 10 with DeltaWave, using full band white noise signal sampled at 44.1/16 and upsampled to 192/24.
IZotope resampler & dither was left with the default settings.
Here are the results (stats are from iZotope RX 10 analyzer):
STATS OF ORIGINAL TEST SIGNAL (RANDOM WHITE NOISE FULL BAND MONO 44.1kHz 16bit DITHERED, GENERATED WITH REW)
STATS OF SIGNAL UPSAMPLED AND DITHERED WITH IZOTOPE RX 10
STATS OF SIGNAL LOOPED BACK FROM WINDOWS MIXER @ 192/24 WITH WASAPI SH. DRIVER (VB CABLE HIFI ASIO BRIDGE)
RESULTS OF DELTAWAVE (BLU IS WINDOWS LOOPBACK, WHITE IZOTOPE RX 10)
As can be seen, the big difference in the audible (almost) band lies in the LPF filter of the two resamplers, which causes differences in the order of -60dB towards the cut off frequency.
iZotope's resampler has a less steep rolloff, but starts at exactly 20kHz and has virtually zero residual noise (-160dB).
Windows has some leakage up to 21/22kHz and then attenuates more quickly to almost -130dB.
Tiny phase shifts also appear, I assume in Windows. But they are ridiculous.
Also, Windows produces about 0.5dB more of true peaks level.
Let me say I cannot figure out why iZotope predicted true peaks differ so much, especially compared to the original signal... but this is another matter.
Obviously changing the iZotope resampling settings could vary its results, for better or for worse. I didn't want to experiment too much in this sense because I was mostly interested in the difference in the udible band, not beyond the cutoff nor in the transition band.
And in fact practically in the whole audio band it can be seen that the delta is around -128dBr.
I think PK Metric is -111dBr due to LPF differences.
I'm not sure why they are expressed as dBFS, they should be dBr, but maybe I'm missing something.
EDIT. I tried to apply LPF @ 16kHz with DeltaWave filter function to see if PK Metric improve, but strangely it only goes to -113dBr, although visually the corrected spectrum is around -128dBr.
There will be something in the metric that I don't remember... or its LPF is hitting... maybe
@pkane could explain.
Not bad anyway, the difference remains inaudible.
With this test I can say that I consider the Windows resampler to be perfectly functional and completely inaudible.
I can leave Windows at 192/24 and not care about sampling the music I hear from various sources because resampling has no impact anyway.
The only thing to do is disable the limiter or put a few dB attenuation via Eq APO to prevent distortion.