Beta-test: DeltaWave Null Comparison software

[111MilesToGo]

So I did post my watermarking analysis here on ASR: https://www.audiosciencereview.com/...watermarking-what-do-we-know.4871/post-573370.

Hope this serves as a good use case for DeltaWave.

 

[pkane]

Version 1.0.56 of DeltaWave is now available with a couple of bug fixes:

• Fix: Drift correction could result in a decrease in higher frequencies in the Spectrum plot
• Fix: Turning off drift correction would not clear the previously computed data in the Clock Drift plot

 

[KSTR]

Hi Paul,
when I save the comparison or delta file, is that before or after any corrections applied? I assume after.
Background: I'm using strictly aligned loop-back analog cable recordings (sample-sync'd), so no drift comp necessary, but due to different analog low-pass effects (from different cables used) there will be a minor sub-sample offset (phase shift) that needs to be corrected before. The associated very small magnitude change at high frequencies will typically be below noise floor and ususally be swamped in slight gain drifts from take to take, even with the global gain correction applied. As I will use time-domain pre-averaged input data to lower the noise and the effect of drifts, it still might be relevant and needs correction, too, to obtain relevant differences apart from the simple linear transfer function differences.
Further, is there a way to save the linear correction parameters obtained from previous matching and apply them 'stupidly' on a new set of files? Idea is to use signals with dense spectra and high energy (eg pink/white noise) to establish the best possible correction first and then apply this to music tracks with much sparser spectra.

 

[pkane]

Hi Paul,
when I save the comparison or delta file, is that before or after any corrections applied? I assume after.
Background: I'm using strictly aligned loop-back analog cable recordings (sample-sync'd), so no drift comp necessary, but due to different analog low-pass effects (from different cables used) there will be a minor sub-sample offset (phase shift) that needs to be corrected before. The associated very small magnitude change at high frequencies will typically be below noise floor and ususally be swamped in slight gain drifts from take to take, even with the global gain correction applied. As I will use time-domain pre-averaged input data to lower the noise and the effect of drifts, it still might be relevant and needs correction, too, to obtain relevant differences apart from the simple linear transfer function differences.
Further, is there a way to save the linear correction parameters obtained from previous matching and apply them 'stupidly' on a new set of files? Idea is to use signals with dense spectra and high energy (eg pink/white noise) to establish the best possible correction first and then apply this to music tracks with much sparser spectra.

Delta file is saved based on the latest processing you requested. If you just load the original and comparison file using the "Show" button and then save delta, the result will be the difference without the corrections. If you click Match before saving, the delta will include all corrections applied during that match.

You can apply arbitrary corrections to the delta file manually. This can be done through Process->Manual Adjustments window. But here, you'll need to enter the parameters or copy/paste them from somewhere. The simplest way is to go to the Results tab, and copy the values reported there after the match. Or you can see them in the Manual Corrections window, also. The Results tab raw correction values should look like this, near the bottom of the report:

gn=0.996473974264749, dc=2.62711891571871E-06, dr=4.59605723356772E-06, of=1.2822988224

gn: gain
dc: dc offset
dr: clock drift
of: phase offset

You can enter these values directly into here:

and then hit Apply! to correct. Note that the clock drift value needs to be converted to ppm (multiplied by 1e6).

A save/load button on this screen might be useful...

Correction parameters are also saved with File->Save Settings option. The next time the settings are loaded, you just need to change ref/comp file names, then go to Manual Corrections screen and press Apply! to correct using these settings.

 

[KSTR]

Paul, this is perfect, thanks. "Manual Corrections" is the key.
Best Regards, Klaus

 

[KSTR]

First quick test looks very promising using this approach, diff'ing 1.5m vs 4m cable in loopback.

While the automated full match didn't work out as expected with white noise (no subsample offset found and corrected for, probably user error as I'm still new to the program), letting the optimizer run for rms null via adjusting offset provided a good starting point that I could fine tune manually, arriving at way less than 0.1° phase diff, with the phase plot being very clean up to fs/2. Very deep audible null when monitoring the diff, just a few samples "jumping around" (as indicated by very grainy low-bit depth sound of the residual).

Applying that offset to a music file, the delta now seems to be dominated by the remaining gain drift error (seen in the corresponing plot and confirmed by listening, still a lot of LF content in the diff). There is still a lot of HF in the diff in short time sections where the static gain correction happens to compensate the dynamic gain drift delta during these moments, so it is not clear if that is coming from remaing offset error or the (microscopic) magnitude difference at HF starting to kick in... have to do some SPICE'ing first to check plausibilty of this.

Since I can measure the transfer functions up to 300kHz++ precisely in REW and might be able to derive a convolution kernel from it to undo the systematic transfer function mismatch, I hope to be able to correct the systematic magnitude (and phase) error in a preprocessing step, only fine-tuning then offset in DeltaWave. Tackling the gain drift then would be the only thing left, I have working provisions for this (the mentioned averaging over many takes, and more importantly, interleaving the takes for the cables in short snippets (relay-switched) so that much of the gain drift averages out. Lot's of work ahead...

Kudos for that stunning software!

 

[Blumlein 88]

First quick test looks very promising using this approach, diff'ing 1.5m vs 4m cable in loopback.

While the automated full match didn't work out as expected with white noise (no subsample offset found and corrected for, probably user error as I'm still new to the program), letting the optimizer run for rms null via adjusting offset provided a good starting point that I could fine tune manually, arriving at way less than 0.1° phase diff, with the phase plot being very clean up to fs/2. Very deep audible null when monitoring the diff, just a few samples "jumping around" (as indicated by very grainy low-bit depth sound of the residual).

Applying that offset to a music file, the delta now seems to be dominated by the remaining gain drift error (seen in the corresponing plot and confirmed by listening, still a lot of LF content in the diff). There is still a lot of HF in the diff in short time sections where the static gain correction happens to compensate the dynamic gain drift delta during these moments, so it is not clear if that is coming from remaing offset error or the (microscopic) magnitude difference at HF starting to kick in... have to do some SPICE'ing first to check plausibilty of this.

Since I can measure the transfer functions up to 300kHz++ precisely in REW and might be able to derive a convolution kernel from it to undo the systematic transfer function mismatch, I hope to be able to correct the systematic magnitude (and phase) error in a preprocessing step, only fine-tuning then offset in DeltaWave. Tackling the gain drift then would be the only thing left, I have working provisions for this (the mentioned averaging over many takes, and more importantly, interleaving the takes for the cables in short snippets (relay-switched) so that much of the gain drift averages out. Lot's of work ahead...

Kudos for that stunning software!

Yes Paul's software is something. I don't have your understanding of what all is involved. I have done some nulling of cables and the software is good enough sometimes manually adjusting parameters seems to show the difference in say a meter and 10 meters of cabling. Which is impressive. I've seen the same thing the residual gain drift error seems to be what prevents near perfect nulling into the noise floor.

As for averaging many takes, do you think clock variability from run to run is why you might need such averaging?

 

[pkane]

I've seen the same thing the residual gain drift error seems to be what prevents near perfect nulling into the noise floor.

Certain types of gain modulation (not drift) can be corrected by the use of non-linear EQ settings in DeltaWave. If you have any examples of files that have gain drift, I'd be interested in seeing them, as I can imagine certain kinds of drift should also be correctable, or at least, measurable.

 

[Blumlein 88]

Certain types of gain modulation (not drift) can be corrected by the use of non-linear EQ settings in DeltaWave. If you have any examples of files that have gain drift, I'd be interested in seeing them, as I can imagine certain kinds of drift should also be correctable, or at least, measurable.

I'll have to see if I have any I kept or maybe make some new files. And yes those that looked like varying gain were mostly corrected in the additional non-linear EQ settings.

 

[KSTR]

Certain types of gain modulation (not drift) can be corrected by the use of non-linear EQ settings in DeltaWave. If you have any examples of files that have gain drift, I'd be interested in seeing them, as I can imagine certain kinds of drift should also be correctable, or at least, measurable.

Hhm, isn't drift just another word for modulation?
In loopback tests, gain drift I would assume come from the reference voltages of the DAC and ADC chips drifting, from their intrinisic ULF 1/f noise, from general supply drift feedthrough, etc.

You explicitly advise against non-linear gain EQ (where I don't understand what is non-linear there) for anything else than gross differences, so I haven't tried that. I would think from the gain drift error recorded (where I don't understand why this isn't a smooth curve coming from a sliding window to calculate some short-term RMS ratios) can be used to create a smooth gain riding curve, a very slowly changing correction factor applied on a sample-per-sample basis.
You see, I still have a lot of white spots wrt the inner workings of your software.

I will upload some test files with gain drifts (and no other effects), which is basically just doing two identical recordings in a row.

 

[pkane]

Hhm, isn't drift just another word for modulation?
In loopback tests, gain drift I would assume come from the reference voltages of the DAC and ADC chips drifting, from their intrinisic ULF 1/f noise, from general supply drift feedthrough, etc.

You explicitly advise against non-linear gain EQ (where I don't understand what is non-linear there) for anything else than gross differences, so I haven't tried that. I would think from the gain drift error recorded (where I don't understand why this isn't a smooth curve coming from a sliding window to calculate some short-term RMS ratios) can be used to create a smooth gain riding curve, a very slowly changing correction factor applied on a sample-per-sample basis.
You see, I still have a lot of white spots wrt the inner workings of your software.

I will upload some test files with gain drifts (and no other effects), which is basically just doing two identical recordings in a row.

Non-linear EQ is a correction of phase and/or level of signal relative to frequency. For example, variable group delay or uneven gain response across the spectrum (for example, a filter). A random, periodic or even linear gain drift is a function of time and not frequency, so non-linear EQ can't correct for it.

The reason I caution against using non-linear EQ is that when not used properly, it can make unwanted corrections. It requires plenty of data to analyze, since it's deriving the corrections in the frequency domain from a complex spectrum average. If not enough samples are provided, the data can contain noise or even some signal which will make the result worse.

It may not be hard to correct for a time-dependent modulation of gain. But I'd want to see some files where this is a real problem.

By the way, the non-linear drift correction option corrects for any random, periodic, or other phase errors as a function of time already.

 

[KSTR]

Some observationas and ideas:

I've now compared two loopback recordings of two slightly different cables of about the same length (thus with slightly different capacitance, resistance and probably different velocity factors, resulting in a different systematic gain, different low-pass effect and associated phase differences plus unknown amounts of simple time-of-fligh differences).

Not using the mentioned higher quality phase baseline measurement by using white noise (as that isn't always an option) I still could arrive at a decent phase null with a lot of optical/mental smoothing and trend-lining as the phase plot is very ragged at high frequencies. I think I'm eyeballing still some falling slope at the very top end but it is really hard to judge this way).

As an improvement, I would suggest to plot the thin 0° grid-line on top of the data, not below it (marking Y-axis zero would apply to other plots as well) and of course some sort of (even adjustable) smoothing (overlay graph or otherwise) would be extremely helpful to judge the remaining offset in a manual optimization cycle. Further, this smoothed trend line might be useful for the automatic optimizer as a new target "phase null" (via varying only offset, of course).

Looking at the delta of spectra, a somewhat similar picture:

What I think I'm seing here is the different low-pass effect kicking in, resulting in a tiny level mismatch as governed by the systematic transfer function differences. Besides the bouncy wideband residual from the gain drift, listening to the residual I feel it is dominated by this effect, exposing high frequencies.
Again, smoothing would be helpful for the eye. And the obtained smoothed data might also be used for an automatic or manual linear transfer function matching in the "manual" adjustments page.

I'm mainly interested in the nonlinear / chaotic / stochastic residuals which is hard to peel out as long as simple amendable linear transfer function differences still dominate the diff (under the assumption the overly dominant gain drift issue can be settled / mitigated first), so some expert features to reduce these systematic linear differences as much as possible would seem worthwile.

And as a general feature, smoothing where it makes sense, additional to the linearity plot where it is present already.

 

[pkane]

Some observationas and ideas:

I've now compared two loopback recordings of two slightly different cables of about the same length (thus with slightly different capacitance, resistance and probably different velocity factors, resulting in a different systematic gain, different low-pass effect and associated phase differences plus unknown amounts of simple time-of-fligh differences).

Not using the mentioned higher quality phase baseline measurement by using white noise (as that isn't always an option) I still could arrive at a decent phase null with a lot of optical/mental smoothing and trend-lining as the phase plot is very ragged at high frequencies. I think I'm eyeballing still some falling slope at the very top end but it is really hard to judge this way).
View attachment 108115
As an improvement, I would suggest to plot the thin 0° grid-line on top of the data, not below it (marking Y-axis zero would apply to other plots as well) and of course some sort of (even adjustable) smoothing (overlay graph or otherwise) would be extremely helpful to judge the remaining offset in a manual optimization cycle. Further, this smoothed trend line might be useful for the automatic optimizer as a new target "phase null" (via varying only offset, of course).

Looking at the delta of spectra, a somewhat similar picture:
View attachment 108116
What I think I'm seing here is the different low-pass effect kicking in, resulting in a tiny level mismatch as governed by the systematic transfer function differences. Besides the bouncy wideband residual from the gain drift, listening to the residual I feel it is dominated by this effect, exposing high frequencies.
Again, smoothing would be helpful for the eye. And the obtained smoothed data might also be used for an automatic or manual linear transfer function matching in the "manual" adjustments page.

I'm mainly interested in the nonlinear / chaotic / stochastic residuals which is hard to peel out as long as simple amendable linear transfer function differences still dominate the diff (under the assumption the overly dominant gain drift issue can be settled / mitigated first), so some expert features to reduce these systematic linear differences as much as possible would seem worthwile.

And as a general feature, smoothing where it makes sense, additional to the linearity plot where it is present already.

Just to make sure, you know you that can change the zoom level for each of the axis independently (or together)? If you zoom in to 0.01 degree or 0.01dB level, any signal will look a bit noisy. The automatic scale that's used on initial display will always zoom in as far as possible to show all the available data, which may be at the level of an electron microscope

 

[KSTR]

Yes, I'm aware of that.
Comparision of the initial phase plot (top trace) with no offset correction deemed neccessary (probably because it is already quite good in absolute terms) and the manually corrected one (lower trace), hopefully showing better what I meant:

At this moderate zoom level the trend line of course can easily spotted but if one wants to dig in deeper for the best possible phase null a smoothed curve would be helpful.
And yes, striving for the ultimate match sure is nerd stuff, I fully agree. But it is neccessary for the topic im investigating ;-)

 

[pkane]

Yes, I'm aware of that.
Comparision of the initial phase plot (top trace) with no offset correction deemed neccessary (probably because it is already quite good in absolute terms) and the manually corrected one (lower trace), hopefully showing better what I meant:
View attachment 108168
At this moderate zoom level the trend line of course can easily spotted but if one wants to dig in deeper for the best possible phase null a smoothed curve would be helpful.
And yes, striving for the ultimate match sure is nerd stuff, I fully agree. But it is neccessary for the topic im investigating ;-)

If you turn on nonlinear EQ for phase, you’ll see a trend line through the original phase error plot. I can add it to the uncorrected plot, as well, not a problem.

 

[KSTR]

I've made a computed test file to better illustrate the specific nerd issues I'm trying to solve. Origin was a mono pink noise, left and right channels filtered by 1st-order lowpasses at around 300kHz, with a 10% difference between them (like a 10% difference in cable capacitance would produce), all other effects like static gain difference, gain drift, propagation delay etc neglected.

Setup file is attached.

Manual best fit for phase null was acheived with 0.00225 samples shift (with L:reference, R:compare), with the test file being noise the trend-lining issue was mitigated:

But of course the tiny magnitude difference isn't yet corrected, we see the (in this case) rising slope:

(the general offset comes from the computed gain correction factor of 0.999982674755387)

In the spectrum of the residual we see comb-filtering and that's also exactly how it sounds:

All of this is expected as the constant offset (==constant group delay) correction is only an approximation of the not truly constant group delay delta, and in general we are not undoing the general transfer function mismatch, with the magnitude part missing.
Quick shot with LTspice :

Looks like a reasonable match (0.00225@ 48kHz is 47ns vs 62ns, and the magnitude factors @20kHz are also close, both being ~5mdB).


Basically, I would think all the data needed for a transfer function matching is available, using the (smoothed) gain and phase delta curves to create an inverse filter undoing it. When the input data is "good enough" (high energy / low noise, dense full spectrum) and ref and comp being already really close (sample-sync'd / no clock drift, low gain drift) to provide meaningful raw data to obtain reasonable smoothing, I would think it is possible to do such a "linear transfer function matching" to fully expose the part of the residual which isn't explained by simple static transfer function delta.

PS: If it is of any help, I could upload a test file with artifical known gain drift, also supplying the modulator file (1.0 + some heavily low-passed 1/f noise for something like 0.1%/0.01dB range of deviation).

 

[pkane]

I've made a computed test file to better illustrate the specific nerd issues I'm trying to solve. Origin was a mono pink noise, left and right channels filtered by 1st-order lowpasses at around 300kHz, with a 10% difference between them (like a 10% difference in cable capacitance would produce), all other effects like static gain difference, gain drift, propagation delay etc neglected.

Setup file is attached.

Manual best fit for phase null was acheived with 0.00225 samples shift (with L:reference, R:compare), with the test file being noise the trend-lining issue was mitigated:
View attachment 108292

But of course the tiny magnitude difference isn't yet corrected, we see the (in this case) rising slope:
View attachment 108294
(the general offset comes from the computed gain correction factor of 0.999982674755387)

In the spectrum of the residual we see comb-filtering and that's also exactly how it sounds:
View attachment 108298

All of this is expected as the constant offset (==constant group delay) correction is only an approximation of the not truly constant group delay delta, and in general we are not undoing the general transfer function mismatch, with the magnitude part missing.
Quick shot with LTspice :
View attachment 108300
Looks like a reasonable match (0.00225@ 48kHz is 47ns vs 62ns, and the magnitude factors @20kHz are also close, both being ~5mdB).


Basically, I would think all the data needed for a transfer function matching is available, using the (smoothed) gain and phase delta curves to create an inverse filter undoing it. When the input data is "good enough" (high energy / low noise, dense full spectrum) and ref and comp being already really close (sample-sync'd / no clock drift, low gain drift) to provide meaningful raw data to obtain reasonable smoothing, I would think it is possible to do such a "linear transfer function matching" to fully expose the part of the residual which isn't explained by simple static transfer function delta.

PS: If it is of any help, I could upload a test file with artifical known gain drift, also supplying the modulator file (1.0 + some heavily low-passed 1/f noise for something like 0.1%/0.01dB range of deviation).

Hi Klaus,

Yes, please share the test and reference files.

 

[KSTR]

Ahm, there should be a working link under the "test file" keywords at the beginning of my post.

btw, turning on both Level EQ and Phase EQ in the nonlinear calibration does not seem to help my problem. While the above plots look good, with the phase and the mag differences nulled, the residual is just one funny sounding artifact, a modulated noise pattern. I would have expected a more steady-state residual noise pattern but of course I don't really know if this guess is any correct.

 

[pkane]

Ahm, there should be a working link under the "test file" keywords at the beginning of my post.

btw, turning on both Level EQ and Phase EQ in the nonlinear calibration does not seem to help my problem. While the above plots look good, with the phase and the mag differences nulled, the residual is just one funny sounding artifact, a modulated noise pattern. I would have expected a more steady-state residual noise pattern but of course I don't really know if this guess is any correct.
View attachment 108313

Ha! That's what I get for reading on a smart phone Thanks for sharing it. I'll take a look.

 

[KSTR]

Ah, sorry, gain error test file to come... (if that was the message).
EDIT: It wasnt ;-) x-posting micro-timing issue, haha