Any way to create a filter which can never clip during the convolution step?

[shaunp]

Hello all,

I'm working to add some amplitude+phase correction to my room and using convolution to apply the filter, and have a question about how to avoid clipping during the convolution step. Is there any way to create a filter which cannot clip during the convolution step? I'll try to give some more info below (and info on an experiment I'm trying out to avoid clipping):

The background is I'm measuring my room response (n.b. using REW), creating filters to flatten the response (n.b. using Rephase to create FIR filters), and applying the filter in my convolution software (n.b. CamillaDSP), before hitting my recording IF and speakers. Sometimes I get clipping reported by the convolution software when I play some tracks or boost the track level, and am trying to understand how to avoid this entirely. I know there are many ways to clip in the signal chain, but I really just want to dial in the convolution stage to narrow the focus a bit. There's a lot of info online, which I'm reading but still digesting, but the basic approach I've found is to a) reduce the level of my input or b) create a filter which also lowers the output. I really like b), the idea being to create a filter which never clips for any input, rather than risk clipping depending on the input level or track. Do you have any ideas how to set the filter level so clipping cannot occur at convolution time? So far, in Rephase I can lower the 'Output Settings' fader then regenerate my filter, and this helps, but I don't know which level can *always* prevent clipping. If you have any info/tips/tricks on this point, it'd be very helpful!

I've also done some Googling, and think I saw some info which may help prevent convolution clipping. I'll share the info here, but am still experimenting with it and don't know enough to vouch for it. If anyone knows if the approach is correct or not, I'd really love to hear it. Basically, if I export my filter as a .csv file from Rephase, import into Excel/Sheet, convert all values to absolute values using ABS(), then add them all up into a single total using SUM(), then I get a total. I think I'm reading that as long as the total is <=1 there can never be clipping during convolution for any input. If the total is something >1, then I believe I can just lower my 'Output Settings' fader in Rephase and regenerate the filter until I get a <=1 value. I'll put the link below, where I'm seeing a comment about the math and how convolution clipping could be avoided by the above approach. But, does this pass muster at all? Would love to know any thoughts on whether the math checks out (or if there's another way to get any 'no convolution clipping' guarantee.

Reference:

Is there a way to mathematically prove whether the convolution of a digital signal will lead to digital clipping?

Hello, I had a question regarding the convolution of digital signals. Assuming the following digital signal: [2_2826.png] ...is convolved by the...

www.dsprelated.com

See "Reply by kaz●November 21, 2018". I think if "sum(abs(coeffs))" <= 1, then the max output is never boosted and cannot clip. This is essentially the motivator for the Excel/Sheets procedure above.

Thanks very much for your comments and advice.

Best regards,
Shaun

 

[ppataki]

I would just simply lower the target curve in REW when doing the EQing
So for example if you measured your system at 75dB, make sure that your target is below 75dB
That will ensure that you use negative filter values only making it impossible for the filter itself to cause digital clipping

 

[shaunp]

I would just simply lower the target curve in REW when doing the EQing
So for example if you measured your system at 75dB, make sure that your target is below 75dB
That will ensure that you use negative filter values only making it impossible for the filter itself to cause digital clipping

Thanks ppataki. I'll check. I've been assuming that adjusting the gain in Rephase would have a similar effect on the filter as lowering the target curve in REW, but so far I've only told REW to automatically set the target curve (i.e. 'Target Settings->Set target level') and can definitely try a lower curve out.

For setting the target curve, I'm setting my Flatness Target in REW to 1dB, so I think I could click Set Target Level then reduce the curve by 3dB (or 1dB, if I want to test my luck), generate+apply a filter, and check again for clipping. To your eyes, does that seem about right or do you think a lower value or different approach would work better?

Also, is there any advice on how to add bass boost to a filter without convolution clipping? The background is that next I'll be moving to extending the bass of my speaker to give a few extra Hz with a flat response (after this convolution study) - thanks

Best regards,
Shaun

 

[levimax]

The only way I have found to reliably test digital filters (FIR in my case) for digital clipping is running a 0 dB sweep and listening for digital clipping which is quite obvious. I have read that calculations are not accurate and any auto level convolution program I have tried does not prevent digital clipping on a 0 dB sweep. My only concern with this method is that it may be overkill although I don't understand why, and it incinerates headroom.

 

[shaunp]

The only way I have found to reliably test digital filters (FIR in my case) for digital clipping is running a 0 dB sweep and listening for digital clipping which is quite obvious. I have read that calculations are not accurate and any auto level convolution program I havr tried does not prevent digital clipping on a 0 dB sweep. My only concern with this method is that it may be overkill although I don't understand why, and it incenetates headroom.

Hi levimax, thanks for that tip! A 0db sweep is really good test I should try out. I appreciate you sharing your experience about how reliable it's been for you too.

BTW - Your comment on headroom resonated with me. I also don't quite know the reason, but I'm starting to get the feeling that the amount of filter gain reduction needed to feel confident about clipping is really big and costs a lot of headroom. I'm going to guess that a no clipping result on a 0dB sweep is a high confidence type of test, with corresponding high headroom cost. Probably there are some lower headroom cost approaches with perhaps more clipping risk. Feels like a trade off may just be the reality here, but not completely sure.

Just 'talking out loud here':
If there is a tradeoff, I've got to figure out my target volume level and see what my options are. I've got a Focusrite Clarett+ recording IF with 124dB of dynamic range going to my speakers, so I'm hoping I can boost there to compensate for the FIR filter drop without getting too quiet. If I have to spend say 20-40dB of its headroom to compensate for the gain drop for a 'no clipping filter', I'm hoping I can still get a workable volume out of the speakers. Fingers crossed!

 

[ppataki]

For setting the target curve, I'm setting my Flatness Target in REW to 1dB, so I think I could click Set Target Level then reduce the curve by 3dB (or 1dB, if I want to test my luck), generate+apply a filter, and check again for clipping. To your eyes, does that seem about right or do you think a lower value or different approach would work better?

Actually setting the target level drives what/how REW would compensate: anything below the curve won't get impacted (unless boost filters are set accordingly) and anything above the curve will be adjusted.
Now this gives you the ability to control what you want to compensate for and what not to.

See an example below; this is the measurement of my CHR120 based system at 75dB (no filter applied except for a Linkwitz Transform in Jriver):

Now I have set my target curve to 62.5dB since I do not want to compensate for all the dips around 50Hz
Another approach could be to set the target to 67dB to have even less compensation for the dips throughout the spectrum
(in any case use 0dB boosting and a flatness of 1)

Yes, you will be losing volume but you will have a much more linear system at the end of the day:

Actually if you use Jriver's Internal Volume (64-bit precision) then you would not really need to worry about filters clipping, etc.
Just set your amp to max or to a fairly high volume (if it has a volume control) and just simply adjust volume in Jriver
This is what I have been doing and never ever had any issues with clipping + it ensures the best possible quality volume control can ever have (since it uses 64-bit precision)

Also, is there any advice on how to add bass boost to a filter without convolution clipping? The background is that next I'll be moving to extending the bass of my speaker to give a few extra Hz with a flat response (after this convolution study) - thanks

REW will show you the amount of headroom here - this is how much volume you will need to cut to avoid clipping

But again, if you use volume control upstream, you won't need to worry about clipping

This is how to set it up:

It will apply volume control upstream:

Set clipping like this to avoid dynamic range compression:

 

[voodooless]

Essentially, this should be simple: plot the response of the filter: check how much boost you have, and lower the gain by at least that amount. For good measure, you can lower the gain by 1 or 2 dB more.

 

[levimax]

Essentially, this should be simple: plot the response of the filter: check how much boost you have, and lower the gain by at least that amount. For good measure, you can lower the gain by 1 or 2 dB more.

You would think that would work but it doesn't for me. In order to eliminate digital clipping with a 0 dB sweep I find I have to use -4.5 dB less gain than the calculated amount which is - 5 dB in my case. This means -9.5 dB which is a lot of headroom. Even though I have 750 watts per channel with an active 4 way system I added a pre-amp for extra analog gain. Of cource this trades potential analog clipping for potential digital clipping which I think is a good trade. I would be interested to hear if others running a 0 dB sweep have a similar experience.

 

[voodooless]

You would think that would work but it doesn't for me. In order to eliminate digital clipping with a 0 dB sweep I find I have to use -4.5 dB less gain than the calculated amount which is - 5 dB in my case.

At what point in the sweep does that occur? How does this correlate to the frequency response of your filter?

 

[levimax]

The filters I use are generated by REW and Rephase so there is a lot of "overlap" of the different filters but the digital clipping occurs where you would expect it to based on the filters i.e. at the points of maximum boost. Again my surprise was the Convolver software I use (plug in for Foobar200) says it analyses the filters and will automatically reduce gain (in my case -5 dB) so it won't clip but it doesn't work for a 0 dB sweep. I find this behavior consistent with different filters I have generated over the last several years. Maybe not clipping on a 0 dB sweep is overkill as I only noticed an issue on one music track before I tried the 0 dB sweep and lowered the gain. When I lowered the gain the issue on the 1 track disappeared. I have a feeling that digital clipping for people using room correction is a much bigger issue than most believe, especially with today's super loud recording style where 0 dB peaks are not rare but I don't really know. It would be interesting for people with the automatic adjustments like Dirac to run 0 dB sweeps and see what happens.

 

[shaunp]

Hi ppataki, a lot of good info in here. I'll try to add some good replies.

Actually setting the target level drives what/how REW would compensate:

Based on this, I should definitely check the target level and see if I can get some better results than REW's automatic setting. I don't think I fully appreciated this aspect of the curve - thx

Now I have set my target curve to 62.5dB since I do not want to compensate for all the dips around 50Hz
Another approach could be to set the target to 67dB to have even less compensation for the dips throughout the spectrum
(in any case use 0dB boosting and a flatness of 1)

For the 50Hz dip, is the idea to avoid boosting for the room mode dips?

Yes, you will be losing volume but you will have a much more linear system at the end of the day:

Makes sense, that seems to be the tradeoff I'm definitely ok dropping some level for flatness (and no clipping), so no problems for me.

Actually if you use Jriver's Internal Volume (64-bit precision) then you would not really need to worry about filters clipping, etc.
Just set your amp to max or to a fairly high volume (if it has a volume control) and just simply adjust volume in Jriver
This is what I have been doing and never ever had any issues with clipping + it ensures the best possible quality volume control can ever have (since it uses 64-bit precision)

Given there's a big gain drop in the filter, I like your idea of just setting the amp to max and adjusting the level in software. I think this gives the most headroom given the gain drop. I just need to remember to turn the amp off when not in use

Regarding precision, I read that CamillaDSP also uses 64 bit floating point internally. In my case, I have 24-bit integer audio values going in, a filter with 64-bit floating point values going in, and 24-bit integer audio values coming out. I guess the clipping occurs when the convolution outputs a 64-bit floating point value which is larger/smaller than the maximum/minimum 24-bit value. I'd expect any convolution hw/sw would have the same basic problem, but I cannot say I understand fully how it's handled.

REW will show you the amount of headroom here - this is how much volume you will need to cut to avoid clipping.

That's really helpful. I'll plug this into my filter gain reduction and check the results. I am now realizing I didn't quite understand what this field was telling me or how to use it.

I can't wait to get to add some of these ideas in and try them out. Thanks!

 

[ppataki]

For the 50Hz dip, is the idea to avoid boosting for the room mode dips?

Well, there are books on this subject so I will be brief here
Normally you shall not try to fill room mode dips with EQ - it will increase ringing and it will put an extra burden on your amp
However, based on my personal experience, you may indeed want to try it - you will be able to fill almost any room mode dip but there will be a certain point when ringing will become an issue (ringing will be visible in the Waterfall and the RT60 diagrams in REW + yours ears will tell you about it)
So playing around with the target curve comes into play here - I suggest experimenting with how low you shall bring it (try 2-3 different values and listen/measure)

I can't wait to get to add some of these ideas in and try them out. Thanks!

You are welcome
Let us know how it went

 

[ernestcarl]

Set clipping like this to avoid dynamic range compression:

Eh, that setting is not actually recommended: https://yabb.jriver.com/interact/in...HPSESSID=bgsu27el2sft6qjt0fjvnlfbn7#msg813434

I'm sure there are other better ways to "avoid dynamic range compression"...

 

[3ll3d00d]

I'm sure there are other better ways to "avoid dynamic range compression"...

Well basically don't run your setup such that it routinely clips

Better explanation of how that setting works in https://yabb.jriver.com/interact/index.php?topic=117545.0

It is basically a compressor but plain clipping will produce ugly distortion so it's surely the lesser of two evils

To the OP, why don't you just convolve your filter with an appropriate test signal to verify if it clips or not?

 

[shaunp]

To the OP, why don't you just convolve your filter with an appropriate test signal to verify if it clips or not?

Hi 3ll3d00d,

Yep, gotta test it. I'm building a list of how to test. In broad strokes, here's what I got:

1) Can run some known loud test tracks and adjust things until no clipping occurs. This helps avoid clipping for many practical cases. The most popular approach I've seen.
2) Can run some test signals to stress test and adjust things until no clipping occurs. The 0dB sweep mentioned above fits here. My feeling is it'll catch more cases and give additional confidence for those who want it (in addition to doing to #1).

Some notes:
For #2, I'm taking a quick hack at understanding the convolution math to see what the worst test signal would be for clipping at this stage. I have a picture, but not done yet. Will post if I get it working.

Related to adding additional stages, like JRiver's compression IIUC, I think there's some checking I should do here. Currently, I'm trying to narrow my focus to the convolution stage only. Afterwards, I'd really like to check more and see what additional stages are being used by other software to help with clipping. Also, I've been thinking a high-pass filter with cutoff below 20Hz could be good to add before the convolution stage since it'll remove any unneeded DC that may have crept in the audio signal and which could be eating up headroom and helping to trigger clipping. Glad other stages been part of the conversation!

 

[ernestcarl]

To the OP, why don't you just convolve your filter with an appropriate test signal to verify if it clips or not?

Yep, measure to see what changes...

I would simply adjust the input gain knob/switch (if it's an active speaker) or adjust volume level and/or alter the EQ via DSP.


0 dBFS sweep (DSP volume adjusted to within transducer limits).

Past a certain point, the Fostex 6301's internal 10 watt amp (20 peak) and DSP start clipping and distorting -- it sounds awful and is audible even with earmuffs on.



Two additional EQ changes to prevent clipping at max seen with full scale signal tests.

 

[3ll3d00d]

I mean convolving offline, e.g. in REW this is A * B trace arithmetic

 

[shaunp]

I mean convolving offline, e.g. in REW this is A * B trace arithmetic

Hi 3ll3d00d, thanks for the tip (and apologies for the delayed response). I guess I haven't tried since I haven't learned how to use trace arithmetic in REW yet . But, I get your idea and may try and check into this part of REW. At the moment, I *think* I have offline convolution going on in REW and in Rephase instead, but via a different path than A*B trace arithmetic. I'll try to explain below - If I'm missing the idea, please let me know - thanks!

My setup (in a nutshell):
I have a test rig and procedure of sorts which lets me quickly create+apply+measure new convolution filters in a loop and get more and more frequency flattening on each loop. Basically, I use REW to get my original measurement, then export the measurement into Rephase (one time only, and never change the Rephase measurement again). Then, I have REW autogenerate filters for Rephase (17 filters max at a time), export the filters into Rephase into an unused filter bank (16 banks available, each supporting 17 filters max), then export the convolution coefficients from Rephase, and point CamillaDSP at the coefficient file to apply the new convolution filter, then I jump back to the earlier REW measurement step and remeasure to see the frequency response with the new filters applied. If the results need more flattening, I continue with all steps again, and repeat the overall procedure as many times as needed until REW measurement is flat and frequency response stops changing significantly. The multibank approach helps me really flatten things out since I can apply way more than 17 filters overall.

I'm leaving out a bit of detail and am making some simplifications in the above description, but that's my setup at a glance.

Coming back to offline convolution, If I'm understanding right it seems the same as getting a graph of the predicted frequency response with filters applied. So, I think I have offline convolution going on at two points above. One in REW during the autogenerate filters step, since it'll show the predicted frequency response with filters applied. And a second point is in Rephase after importing filters to an unused bank, since it'll show the predicted frequency response for the new filter. I'll get the actual frequency response when during the 'set filters in CamillaDSP' and 'remeasure in REW' steps, then compare the actual with predicted to make sure nothing strange is going on.

Anyway, thanks for letting me brain dump this out. I hope I've got the right picture. Please do let me know if I'm off-track - thanks!

[EDIT - add]
Procedure is heavily influenced by https://rephase.org/tutorials/rePhase Tutorial-en.pdf by [pda0] & [Bear]!

 

[shaunp]

Some notes:
For #2, I'm taking a quick hack at understanding the convolution math to see what the worst test signal would be for clipping at this stage. I have a picture, but not done yet. Will post if I get it working.

In case it's useful, I'll describe my current understanding since I haven't been really able to get back to it and would like it 'out there' in case I forget.

[Begin Braindump]
Basically, seems that a convolution filter is a list of numbers, negative or positive
Ex. Filter may be like -0.7,+2.35,...,-3.11,-1.977

I think convolution takes your input signal (which is also a list of numbers, negative or positive) and cross-multiplies it
Ex. If input is in1,in2,...,inY,inZ then seems input and filter is cross-multiplied like below:
-0.7,+2.35,...,-3.11,-1.977
in1,in2,...,inY,inZ
---------------------------
-0.7*inZ + 2.35*inY + ... + -3.11*in2 + -1.977*in1=output value

So, I think the worst signal is for convolution clipping could be the one where the output value hits the highest possible positive or lowest possible negative number, which can be determined by looking at the sign of the coefficients and knowing the number of bits per sample:

For 24-bit signed input samples:
Highest possible positive number case:
inZ=<if 1st filter value sign is '+', then set value to maximum possible 24-bit number (+8,388,607), if '-' then set value to minimum possible 24-bit number (−8,388,608)>, inY=<same as inZ, except use 2nd filter value sign>,..., etc.
Lowest possible negative number case:
inZ=<if 1st filter value sign is '+', then set value to minimum possible 24-bit number (−8,388,608), if '-' then set value to maximum possible 24-bit number (+8,388,607)>, inY=<same as inZ, except use 2nd filter value sign>,..., etc.

NOTE: Minimum+maximum values taken from https://en.wikipedia.org/wiki/Audio_bit_depth

I bet there is a program somewhere which can generate above worst-case signals, but so far I haven't found one. If one did exist, I'd expect it would sound horrible since it's designed to trigger clipping, and I'd expect it to be *loud* so would want my speakers off.

Overall, I think I'd rather not use a program to generate a horrible 'worst case' output then have to listen to it through my speakers. I'd rather just get a notification that convolution clipping was possible with my current convolution filter, and then get advice on how to adjust to avoid it (or some autoadjustment could be neat). Currently, I think I can stick with my current approach of checking that my convolution coefficients meet the condition that "sum(abs(coeffs))" <= 1. That should tell me convolution clipping is not possible with my filter (to the best of my knowledge). If I get >1, then I can just adjust my filter tool to output at a reduced volume level and recalculate until the above condition is met. So far, seems to be working well for me. It'd be neat if REW or Rephase or CamillaDSP could tell me this info, instead of manually having to use Excel/Sheets to manually do the math. But, I'm ok with the manual approach. Maybe it's a good feature request to make for the excellent developers of these programs, though I haven't suggested anything yet.

Just to mention it, the main downside I've seen is that whenever I try to meet the above condition, I have to adjust my filter in ways so that the convolution stage outputs a much quieter volume then the input audio. I don't really want to trade volume/headroom for no convolution clipping. My current solution to addressing the drop in volume is by adding a volume boost stage after my convolution stage to add 'makeup gain'. Without getting too far into it, I'm using Rephase to generate my filter, and I'll lower it's output level as much as needed to meet the above criteria and avoid convolution clipping. I use CamillaDSP as my convolution stage to actually apply the filter to my audio, and after CamillaDSP I send the output through a program called Element, where I can add the makeup volume boost, then send to my speakers. This seems to be working for me. I hope the basic idea above will work even if using different tools than Rephase/CamillaDSP/Element, so I hope these thoughts are useful to share

[End Braindump]

 

[shaunp]

The only way I have found to reliably test digital filters (FIR in my case) for digital clipping is running a 0 dB sweep and listening for digital clipping which is quite obvious. I have read that calculations are not accurate and any auto level convolution program I have tried does not prevent digital clipping on a 0 dB sweep. My only concern with this method is that it may be overkill although I don't understand why, and it incinerates headroom.

Hi Levimax,

I was rereading this and think I've made some progress since then. I wanted to let you know what I've settled on in case it was useful for you.

1) I'll get my filter coefficients into Excel/Sheets, then check that sum(abs(filter coefficients)) < 1. To the best of my knowledge, I think this means no convolution clipping is possible for the filter. I really hope I'm right, and I'm no math wiz, but I'm getting some confidence now that I've played with it a bit. My convolution stage hasn't given me any clipping messages at all (I'm using CamillaDSP). If you'd like to send me your filter coefficients as a .csv or .txt file, I could try running the numbers for you - let me know.
---If the math is giving >= 1, then I'll adjust the output gain down in whatever tool is giving me my filter coefficients and eventually it'll get there. I'm using Rephase for the output gain adjustment for my filter.
2) After above is done, then my understanding is that you can run any input signal and the convolution stage shouldn't clip (can't speak for other stages, like your amps/etc.). Probably best to turn speakers off and just check your convolution tool to see if it reports clipping or no clipping. I'm using CamillaDSP.
---Full disclosure: I keep getting distracted and haven't actually gotten to try out a 0dB sweep test yet. But after weeks, not even a single clip, so feeling pretty good.
3) After above is done, I find that getting my filters to give the '<1' result never gives any clipping during the convolution stage, but the convolution stage dramatically lowers the output volume. I don't really think I have to trade volume and headroom to avoid convolution clipping. So, I'm adding a volume boost stage after my convolution stage to compensate and make up for the drop. I'm using Element to add the volume stage.
FYI - #3 is one of the more recent things I've added - no issues so far for me, but still trying it out.

I hope the above info is interesting at least, and even better if it ends up being useful.