"tweak everything as you please to get whatever results you want"
Well
First, you need to properly calibrate the output.
If you want to compare figures together, you need to define a (somehow arbitraty) level you want to compare figures at.
Amir selected 4Vrms for balanced (2Vrms for unbalanced)
In REW, to measure a DAC, you need to set the output level to this level.
One of the benefits of the AP hardware is that it includes a Digital Multi Meter with the Analyzer.
That's not the case of your ADC, of course, and given its low Input impedance, voltage may vary, so you'll have to measure using a (true RMS) DMM.
Then you calibrate level in REW
As can be seen below, here I calibrated REW for 0dBFS = 6,799Vrms for my RME ADI-2 Pro fs R @19dBu output range, balanced.
After that, I may directly dial in the Voltage I want to test at (Here 4Vrms)
And then the resulting measurement
Note that I usually select dB for distortion display, rather than %
Also note important settings: Limit strictly frequency from 20Hz to 20kHz
Window function is here set to Dolph-Chebyshev, which is supposed to be closest to AP's proprietary window function.
But other functions, like Blackmann-Harris 7 or Kaiser 8, work too.
Good news is that now I can reproduce the same measurement with other softwares:
ARTA
or, my favorite,
Virtins Multi-Instrument Pro
(It's my favorite because it's widely controlable through a .Net application, allowing similar functionalities than the APx 500 scripting available with Audio Precision, for a fraction (<1%) of the price)