The second spectrum is definitely a measurement error. It's very distinctive. Windowing was my first guess (I don't use REW so don't know how reliable the window setting is), other possibilities include a signal interruption during acquisition. You should re-run that.
Yes, measurement error.
Multitone in REW requires sample-synchronous record-while-playback, the FFT size set equal to the generator sequence length, and rectangular (==no) window.
Looks like the first condition wasn't met, hence the leakage skirts on the bottom of the needles.
Both devices may operate at the same nominal sample rate, but the clocks are not phase-locked (drifting apart by some amount) and can't be locked, with both devices being USB-driven with no way to sync them.
Workaround: use a way larger FFT size with the Dolph-Chebychev-200 window. Note that this still will probably favor the RME because it is sample-synced loopback,.
Yes, my mistake sorry, REW set to Rectangular window on multitone, not Flat Top as I mentioned earlier. I forgot.
Next time I tried your advice with larger FFT size and Dolph-Chebychev-200 window. Thanks for the tips!
It seems that the implementation still a bit buggy and not consistent.
For the same device, once I got this:
On another measurement I got this:
I did several other measurements and most of the time I got the second graph which is more consistent with rectangular window.
So I tested both devices, RME Babyface Pro and the cheap Tempotec USB C DAC with QuantAsylum QA401, this time I match the level, and I got this with Hann Window (all at 24bit - 192kHz with 32 ohm load):
RME Babyface Pro (192kHz):
Tempotec Sonata HD:
The Tempotec FR doesn't go beyond 20kHz despite operating at 24/192k. Seems same low pass filter for all sampling rate.