• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required as is 20 years of participation in forums (not all true). There are daily reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

PK Error Metric discussion and beta-test

DDF

Addicted to Fun and Learning
Joined
Dec 31, 2018
Messages
608
Likes
1,221
Is it to be expected that adding a 20Hz highpass filter spoils rms PKEM big time?

Ya, if it change the envelope and the signal has appreciable energy near cut off, due to the filter phase response (see here and here). If comparing two time files (envelopes), it'll create deltas that aren't audible.

That's a danger with time comparisons alone. Transfer functions that introduce all pass phase response anomalies can distort the envelope but not be audible.
 
Last edited:

KSTR

Major Contributor
Joined
Sep 6, 2018
Messages
2,024
Likes
4,431
Location
Berlin, Germany
Ya, if it change the envelope and the signal has appreciable energy near cut off, due to the filter phase response (see here and here). If comparing two time files (envelopes), it'll create deltas that aren't audible.

That's a danger with time comparisons alone. Transfer functions that introduce all pass phase response anomalies can distort the envelope but not be audible.
That's what occured to me as well when thinking about it. Especially since the filters seem to be very steep --> must ring terribly when exited. Yet post-filtering seems to fix it...
 
OP
pkane

pkane

Major Contributor
Forum Donor
Joined
Aug 18, 2017
Messages
3,201
Likes
5,649
Location
North-East
That's what occured to me as well when thinking about it. Especially since the filters seem to be very steep --> must ring terribly when exited. Yet post-filtering seems to fix it...

The filter is something you do have a lot of control over in DeltaWave settings. You can specify the type of filter, filter size, window, etc:

1611887571416.png


If you want to make the filter less steep, just make Filter Size smaller. But even the steep filter does not ring as much as you think. DW provides you with all the tools to see what effect the settings have on the waveform, on the spectrum, cepstrum, phase, etc. Don't be afraid to look :) Oh, and a steep filter will have a larger group delay, so there's that.

Here's the effect of 1M-tap FIR filter with a low-pass 20Hz:
1611888135050.png


And here it is with a 4096-tap FIR:
1611889131361.png
 
Last edited:

KSTR

Major Contributor
Joined
Sep 6, 2018
Messages
2,024
Likes
4,431
Location
Berlin, Germany
Thanks, Paul.
For a lack of full documentation (no offense) we'll have to figure a lot of things by hands-on work with the actual product.

As for the ringing, from my speaker designer background and DRC experiences I would clarify that of course the final slope is not that much the issue wrt to ringing, rather the actual shape of the transition region. @UliBru is an expert on this, he implemented an improved (adjustable) version of Neville-Thiele style filters with infinite slope and smoothest possible transition (one with a wide transition region shown here)

Regarding IIR vs. FIR: since one specifies # of taps in both cases I may assume that IIR is also an FIR with a sampled (minimum-phase) IR? And the FIR is standard symmetric and hence linear-phase? If so, maybe "linear/minimum phase" would be a more catchy term for this selection.
 
OP
pkane

pkane

Major Contributor
Forum Donor
Joined
Aug 18, 2017
Messages
3,201
Likes
5,649
Location
North-East
Thanks, Paul.
For a lack of full documentation (no offense) we'll have to figure a lot of things by hands-on work with the actual product.

As for the ringing, from my speaker designer background and DRC experiences I would clarify that of course the final slope is not that much the issue wrt to ringing, rather the actual shape of the transition region. @UliBru is an expert on this, he implemented an improved (adjustable) version of Neville-Thiele style filters with infinite slope and smoothest possible transition (one with a wide transition region shown here)

Regarding IIR vs. FIR: since one specifies # of taps in both cases I may assume that IIR is also an FIR with a sampled (minimum-phase) IR? And the FIR is standard symmetric and hence linear-phase? If so, maybe "linear/minimum phase" would be a more catchy term for this selection.

Most of the documentation is on the website:

https://deltaw.org/mydoc_quickstart.html
https://deltaw.org/settings.html
https://deltaw.org/plot_start.html
https://deltaw.org/comparator.html
https://deltaw.org/mydoc_supported_features.html
https://deltaw.org/pk_metric.html

Also, try mousing over controls if you don't know what they do. Tooltips will have more information in most cases.
 
OP
pkane

pkane

Major Contributor
Forum Donor
Joined
Aug 18, 2017
Messages
3,201
Likes
5,649
Location
North-East
Could you include a chart or graph of the ERB buckets. Maybe original, comparison, and a Delta bucket chart for each ERB range. Even listing it as a numerical chart might be useful into to seeing whether or not it gets tricked by certain situations. And to understand which kinds of results are more audible than others. Being able to select a certain portion of the PK metric graph and see the ERB results for that selected section would be really useful. But that might be huge chore to create.

Hi Dennis. I've been playing with this for a while, and haven't found the best way to show this ... yet. Here's the latest, and I'm not sure this is that helpful, although it is somewhat informative. This works by moving the mouse over the chart on the left. As you move, the chart on the right updates with the 400ms STFT window for that time range you're mousing over. The result is with the equal-loudness curves applied, and the ERB smoothing. The title shows the time frame and the total energy in the error signal:
1612030928777.png


Did you envision something different?
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
14,316
Likes
23,072
Hi Dennis. I've been playing with this for a while, and haven't found the best way to show this ... yet. Here's the latest, and I'm not sure this is that helpful, although it is somewhat informative. This works by moving the mouse over the chart on the left. As you move, the chart on the right updates with the 400ms STFT window for that time range you're mousing over. The result is with the equal-loudness curves applied, and the ERB smoothing. The title shows the time frame and the total energy in the error signal:
View attachment 109456

Did you envision something different?
I was hoping you had a good idea for how to display it. :)

What I was hoping is you could use this to gain more insight into areas that aren't resulting in a good low number. Like a noise spike, or some short term tone that came and went. In your example it looks like a pretty flat noise floor in the region examined. No indication for the spike on the left screen.

What I imagined wasn't this, but maybe this is better. I was thinking put the cursor over an area and have a chart show the energy in each ERB band. Your graph accomplishes the same thing except it is smoothed and I suppose a numerical chart would not be (or it could be). The question is will this tell us anything about the signal useful. Your example makes me think it may not shows us any additional information.
 
OP
pkane

pkane

Major Contributor
Forum Donor
Joined
Aug 18, 2017
Messages
3,201
Likes
5,649
Location
North-East
I was hoping you had a good idea for how to display it. :)

What I was hoping is you could use this to gain more insight into areas that aren't resulting in a good low number. Like a noise spike, or some short term tone that came and went. In your example it looks like a pretty flat noise floor in the region examined. No indication for the spike on the left screen.

What I imagined wasn't this, but maybe this is better. I was thinking put the cursor over an area and have a chart show the energy in each ERB band. Your graph accomplishes the same thing except it is smoothed and I suppose a numerical chart would not be (or it could be). The question is will this tell us anything about the signal useful. Your example makes me think it may not shows us any additional information.

Yes, that's what I'm finding -- it's interesting, and helps me test the metric, point by point, but I'm not sure it has a great explanatory value. Not everything comes out this smooth, by the way. For example, here's one of your loopback captures. That peak at 51 second appears to be weighted towards higher frequencies, despite the equal-loudness curves being applied:

1612032134622.png
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
14,316
Likes
23,072
Yes, that's what I'm finding -- it's interesting, and helps me test the metric, point by point, but I'm not sure it has a great explanatory value. Not everything comes out this smooth, by the way. For example, here's one of your loopback captures. That peak at 51 second appears to be weighted towards higher frequencies, despite the equal-loudness curves being applied:

View attachment 109458
Well, I'm not for cluttering up the software if it isn't pretty useful. Like in this case, we have a bit of short spike, and it was at a pretty high frequency, but it doesn't help figure out why.
 
OP
pkane

pkane

Major Contributor
Forum Donor
Joined
Aug 18, 2017
Messages
3,201
Likes
5,649
Location
North-East
Well, I'm not for cluttering up the software if it isn't pretty useful. Like in this case, we have a bit of short spike, and it was at a pretty high frequency, but it doesn't help figure out why.

I’ll think some more about how to best display this. I have a few more ideas.
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
14,316
Likes
23,072
I've tried looking at the PK Spectrum metric. I see for instance in one comparison whenever the metric isn't so good, the reason is around 850 hz. When the metric is good there is little energy at any particular frequency. I found this was true for three different files I had sent thru loops using the same two pieces of gear. I wonder if this means there is some interaction between these pieces at that frequency? That frequency has no relation to the difference in clock speeds. There was no excess energy in the originals at this frequency. So I wonder what it would be or what that tells me.
 
OP
pkane

pkane

Major Contributor
Forum Donor
Joined
Aug 18, 2017
Messages
3,201
Likes
5,649
Location
North-East
I've tried looking at the PK Spectrum metric. I see for instance in one comparison whenever the metric isn't so good, the reason is around 850 hz. When the metric is good there is little energy at any particular frequency. I found this was true for three different files I had sent thru loops using the same two pieces of gear. I wonder if this means there is some interaction between these pieces at that frequency? That frequency has no relation to the difference in clock speeds. There was no excess energy in the originals at this frequency. So I wonder what it would be or what that tells me.

Remember that the original spectrum you're looking at is highly averaged over time, so occasional spikes in a specific frequency may not show up in such a display. STFT (short-time Fourier transform) is good at displaying short-time, transitory phenomena. In the PKMetric screen you can see something that happens only for a 100ms or even less. For example, I've seen a similar thing to what you're describing with someone's recording, except the excess energy was at 50Hz, and only showed up a few times throughout the plot. I assume in that case, something was changing with regards to the AC line frequency being induced into the system. Maybe some equipment came on, or maybe someone moved a cable or some power supply near-by (PC?) was stressed a bit more. Who knows?

I've been thinking of replacing this short, 400ms window with a full spectrogram. May make it easier to see patterns.
 

mehmethand

Member
Joined
Mar 19, 2021
Messages
10
Likes
4
Hi pkane,
I have 2 test files. https://www.dropbox.com/sh/sc039f5iihpag2n/AACET6KfvRd2WjImcuhbbt5ua?dl=0

Reference : A traditional instrument recording
Comparison : Same reference file + (-65 db peak) pink noise added on it and rendered together.

When I compare this files, PK metrik shows -85,6 dBFS. If I understand correctly this value is below -50 dB so the difference should be inaudible but I can hear the difference in ABX blind tests very easyly just by listening to first second of the files.

Am I missing or misunderstanding something?

Trial 1, user: A actual: A 1/1
Trial 2, user: A actual: A 2/2
Trial 3, user: B actual: B 3/3
Trial 4, user: A actual: A 4/4
Trial 5, user: B actual: B 5/5
Trial 6, user: A actual: A 6/6
Trial 7, user: A actual: A 7/7
Trial 8, user: A actual: A 8/8
Trial 9, user: B actual: B 9/9
Trial 10, user: A actual: A 10/10
Trial 11, user: B actual: B 11/11

Probability of guessing: 0%
A=Reference,B=Comparison
Test type: ABX
pk metrik.jpg
 
OP
pkane

pkane

Major Contributor
Forum Donor
Joined
Aug 18, 2017
Messages
3,201
Likes
5,649
Location
North-East
Hi pkane,
I have 2 test files. https://www.dropbox.com/sh/sc039f5iihpag2n/AACET6KfvRd2WjImcuhbbt5ua?dl=0

Reference : A traditional instrument recording
Comparison : Same reference file + (-65 db peak) pink noise added on it and rendered together.

When I compare this files, PK metrik shows -85,6 dBFS. If I understand correctly this value is below -50 dB so the difference should be inaudible but I can hear the difference in ABX blind tests very easyly just by listening to first second of the files.

Am I missing or misunderstanding something?

Trial 1, user: A actual: A 1/1
Trial 2, user: A actual: A 2/2
Trial 3, user: B actual: B 3/3
Trial 4, user: A actual: A 4/4
Trial 5, user: B actual: B 5/5
Trial 6, user: A actual: A 6/6
Trial 7, user: A actual: A 7/7
Trial 8, user: A actual: A 8/8
Trial 9, user: B actual: B 9/9
Trial 10, user: A actual: A 10/10
Trial 11, user: B actual: B 11/11

Probability of guessing: 0%
A=Reference,B=Comparison
Test type: ABXView attachment 119225

You may have the setting to trim files enabled. This removes differences at the start and end of a file.
 

mehmethand

Member
Joined
Mar 19, 2021
Messages
10
Likes
4
You may have the setting to trim files enabled. This removes differences at the start and end of a file.
first second of files belongs to my comparison I can't trim them. I need to analyse 1 second too. That 1 second is a quiet part of the original recording, so I can more easyly recognize the difference than louder parts.
 
OP
pkane

pkane

Major Contributor
Forum Donor
Joined
Aug 18, 2017
Messages
3,201
Likes
5,649
Location
North-East
first second of files belongs to my comparison I can't trim them. I need to analyse 1 second too. That 1 second is a quiet part of the original recording, so I can more easyly recognize the difference than louder parts.

I'm not sure what you're trying to do then. If you want to keep the first/last seconds in DeltaWave comparison, just turn off the trim option in DeltaWave settings.
 

mehmethand

Member
Joined
Mar 19, 2021
Messages
10
Likes
4
I'm not sure what you're trying to do then. If you want to keep the first/last seconds in DeltaWave comparison, just turn off the trim option in DeltaWave settings.
So sorry for my bad english I think I couldn't express myself well. I meant I'm not listening to whole file in ABX blind tests but listening the beginging of the files. Because I can hear the difference more easly at the begining of the files. I'm not trimimng anything.
 
OP
pkane

pkane

Major Contributor
Forum Donor
Joined
Aug 18, 2017
Messages
3,201
Likes
5,649
Location
North-East
So sorry for my bad english I think I couldn't express myself well. I meant I'm not listening to whole file in ABX blind tests but listening the beginging of the files. Because I can hear the difference more easly at the begining of the files. I'm not trimimng anything.

Can you please confirm that the setting "Auto-trim start & end" is turned off in DeltaWave?

1616205147823.png
 

mehmethand

Member
Joined
Mar 19, 2021
Messages
10
Likes
4
Can you please confirm that the setting "Auto-trim start & end" is turned off in DeltaWave?

View attachment 119232

Yes, Auto trim-start is turn off.

I comparing these 2 files

https://www.dropbox.com/sh/sc039f5iihpag2n/AACET6KfvRd2WjImcuhbbt5ua?dl=0

Reference : A traditional instrument recording
Comparison : Same reference file + (-65 db peak) pink noise added on it and rendered together.

PK Metric shows -85,6 dB => I'm expecting the sound difference between 2 files must be inaudible because it is below -50dB.
Blind Test results shows me I can hear the difference easily.
 

Attachments

  • settings.jpg
    settings.jpg
    17.4 KB · Views: 39
OP
pkane

pkane

Major Contributor
Forum Donor
Joined
Aug 18, 2017
Messages
3,201
Likes
5,649
Location
North-East
Yes, Auto trim-start is turn off.

I comparing these 2 files

https://www.dropbox.com/sh/sc039f5iihpag2n/AACET6KfvRd2WjImcuhbbt5ua?dl=0

Reference : A traditional instrument recording
Comparison : Same reference file + (-65 db peak) pink noise added on it and rendered together.

PK Metric shows -85,6 dB => I'm expecting the sound difference between 2 files must be inaudible because it is below -50dB.
Blind Test results shows me I can hear the difference easily.

Are you comparing these files after matching them in DeltaWave, or outside of DeltaWave?
 
Top Bottom