After doing the trim you recommended I can indeed get the files working at the native sample rate
Though, whilst (with the 20khz filter to limit to the audible band) there is an 82dB null, the RMS difference is only -44dB
I'm not sure I fully understand why two recordings from the same dac (a well performing one at that) could be so different?
The RMS difference is heavily weighted towards larger errors (like all RMS metrics). So, in effect, it's designed to help bring out the effect of larger errors, even if they are few and far in between. It's really an engineering metric rather than audibility metric, since it doesn't weigh the differences based on their perceptual significance.
Could you advise on what the best settings to use might be, or if I could perhaps alter my recording method in some way for an improvement? Im not sure what the best option would be in terms of sample rate when recording, vs what I should/should not be resampling to etc etc.
Ideally, you want to test the equipment at the rate you'll be using it. Unless you're trying to measure something well outside the audible range, 24/96kHz should be plenty to capture every tiny audible nuance in a recording or playback. A 705kHz sampling rate isn't needed unless you are trying to detect and measure ultrasonics.
Essentially, what I am currently trying to do is see how similar a result two dacs produce when playing music.
For this, you'd want to use the same music track, at the same sampling rate, recorded with the same ADC but with different DACs.
Two dacs can have the same SINAD, but handle transients differently, different filter designs, etc etc, and so I'm trying to do a very basic "How similar do dacs A and B sound overall" measurement.
Well, that's not quite for the null metric will tell you. As I said, this is more of an engineering quality metric. dBA numbers as reported by DeltaWave will give A weighting to the result, making it a little more related to audibility. For example, your two files show this:
8dB improvement in overall RMS difference and about the same in Correlated Null when looking at A-weighted values.
I've found that dacs that are similar, say, the SU8 and Motu M2/M4, which are both well measuring ESS dacs, correlate very heavily, but then compare the SU8 to the ADI-2, Holo May, or UD501 etc, all of which are different topologies, and you suddenly get a way lower correlation (and they sound different to my ear too, some to quite a degree) even though they all measure well.
The differences that I find are often the cause of large discrepancies in otherwise excellent measurements are the timing/phase errors. One obvious one is caused by different filter designs and implementations that can result in a variable group delay, and sometimes ringing/oscillations. Others have to do with clock rate differences. The others are caused by jitter. While I tried to deal with as many of these timing issues in DW as I could, they are not all easy to measure and/or to correct.
Would you be able to recommend a method/config to do this in a way that Deltawave would be able to perform best?
I've been struggling because obviously things such as long term clock drift are not really important to what a human will hear. A track playing at 99.8% speed will not be noticeable (though jitter could), but for this test, they do need to be aligned so that deltawave can make a proper comparison.
Constant clock drift is detected, measured and corrected by DW, so in most cases, it'll not have an appreciable effect on the result.
Jitter, on the other hand, is not correctable in a general way, especially jitter caused by random noise sources.
Variable group delay is something DeltaWave can measure and correct for, but it requires a larger track (a couple of minutes) to get a better, cleaner result out of it. Getting a few minutes at high sampling rates can eat up all available memory on the computer. Another reason not use high sampling rates.
At lower sampling rates, you can easily use the Non-linear EQ options to compute and correct for the effects of variable group delay. Here are some settings I usually try when I want to see and correct for it:
Remember the example from Gearslutz website I posted earlier that showed -44.5dB RMS difference? That same track processed in DeltaWave with the non-linear EQ options enabled, produces this result:
Nearly doubling all the null and difference values. Eliminating a predictable phase error, such as can be caused by a poor or non-linear phase filter, can help improve the null significantly.
I've been getting inconsistent results correlating a given dac against itself, which means that I can't really make any sort of comparisons between dacs.
surely a -44dB null is somewhat woefully poor?
I've been able to get nulls of -110dB on some configs, but I'm not sure what the optimal configuration I should be using for this situation would be. As its hard to tell what is a genuine difference between recordings and what is simply my settings being odd.
Yes, a -44dB null is not great when considering that the same device is used. There are a number of things you need to consider, and DeltaWave gives you the graphs to find the source of the error. Here's one possible problem: the track you're capturing appears to be very clipped, so you may be using the wrong gain settings or not adjusting the volume properly for the recording:
Thank you so much for all your help, you've created some truly fantastic software and I really appreciate all the assistance you've given me and others here
Appreciate the kind words! And I hope I didn't confuse you!
DeltaWave was designed more as a toolkit to help run multiple types of analysis and to give some insight into what a device is doing. To me, the plots and the comparison spectrograms is where the value is. A single number, like the RMS difference or a correlated null are good for quick look, but they can't fully describe the audible differences between devices. Don't forget, that you can also listen to the difference track to see what the difference sounds like. If there's obvious music components, or just the high notes or noise in it. You can also compare difference tracks recorded from two different DACs to see which one is quieter, and has less music or is mostly noise.
