• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Any way to create a filter which can never clip during the convolution step?

3ll3d00d

Active Member
Joined
Aug 31, 2019
Messages
212
Likes
176
Anyway, thanks for letting me brain dump this out. I hope I've got the right picture. Please do let me know if I'm off-track - thanks!
stacking filters to make it flatter and flatter just suggests your 1st filter isn't right to me but anyway, this is just convolving the filters so if you have 3 filter banks then a*b*c * inputsignal = output
 

levimax

Major Contributor
Joined
Dec 28, 2018
Messages
2,388
Likes
3,515
Location
San Diego
Hi Levimax,

I was rereading this and think I've made some progress since then. I wanted to let you know what I've settled on in case it was useful for you.

1) I'll get my filter coefficients into Excel/Sheets, then check that sum(abs(filter coefficients)) < 1. To the best of my knowledge, I think this means no convolution clipping is possible for the filter. I really hope I'm right, and I'm no math wiz, but I'm getting some confidence now that I've played with it a bit. My convolution stage hasn't given me any clipping messages at all (I'm using CamillaDSP). If you'd like to send me your filter coefficients as a .csv or .txt file, I could try running the numbers for you - let me know.
---If the math is giving >= 1, then I'll adjust the output gain down in whatever tool is giving me my filter coefficients and eventually it'll get there. I'm using Rephase for the output gain adjustment for my filter.
2) After above is done, then my understanding is that you can run any input signal and the convolution stage shouldn't clip (can't speak for other stages, like your amps/etc.). Probably best to turn speakers off and just check your convolution tool to see if it reports clipping or no clipping. I'm using CamillaDSP.
---Full disclosure: I keep getting distracted and haven't actually gotten to try out a 0dB sweep test yet. But after weeks, not even a single clip, so feeling pretty good.
3) After above is done, I find that getting my filters to give the '<1' result never gives any clipping during the convolution stage, but the convolution stage dramatically lowers the output volume. I don't really think I have to trade volume and headroom to avoid convolution clipping. So, I'm adding a volume boost stage after my convolution stage to compensate and make up for the drop. I'm using Element to add the volume stage.
FYI - #3 is one of the more recent things I've added - no issues so far for me, but still trying it out.

I hope the above info is interesting at least, and even better if it ends up being useful.
Thanks for the update. I would be interested in your results with a 0 dB sweep as well for a couple of reasons.
1. Since most of these convolution tools are open source and often "mix and matched" open source I am not convinced that they always work right together. The Convolver I use has an "automatic level control" which is supposed to "calculate" how much to reduce the filter output to prevent digital clipping but it doesn't on a 0 dB sweep.
2. I have a feeling that no clipping on a 0 dB sweep is overkill in most cases and that you give up more headroom than necessary for actual music but not sure and it is probably dependent on the type of music.
Regarding trading volume and headroom for linearity, unfortunately by definition ANY filer operating in the audible range is going to reduce headroom / perceived volume. There is no free lunch.
 
OP
S

shaunp

Member
Joined
Oct 15, 2022
Messages
13
Likes
1
Hello all,

I finally got time to check and wanted to reply with my 0dB sweep results. I can confirm no convolution clipping in my system with a 0dB sweep in REW (20Hz-24kHz). I generated my filters to meet the <1 constraint discussed earlier, applied the filters to my convolution engine (i.e. CamillaDSP), and no convolution clipping was reported whatsoever during my 0dB sweeps. For comparison, CamillaDSP was reporting convolution clipping before I reduced my filter gain to meet the <1 constraint, so the "<1" approach seems to work for me.

FYI - As I don't need to operate this loud, I had my speakers off and amps down during the test (to avoid killing my ears and equipment) and just checked the CamillaDSP text output to see if it detected convolution clipping.

At the moment, I feel this result gives me more confidence to continue with the "<1" approach, as it works for me even at the cost of reduced headroom. I've not measured max headroom yet, but indications are I can get 90dB max or more after filters applied to flatten my freq response and apply phase correction and also have some protection against convolution clipping.

For other environments, I imagine the "<1" approach would also avoid convolution clipping on other systems, even systems using different convolution engines than CamillaDSP. As long as there is some visual way to tell if the convolver detects clipping, I imagine it should be possible to confirm in other setups as well. So, I hope this is all useful info.

IMPORTANT: 0dB testing is dangerously loud with amps/speakers on, and I bet can also destroy your amps/speakers so please, if anyone tries this, only test with amps/speakers turned off or way down. Also, have some serious hearing protection on for safety. It can kill your ears and mistakes are *easy to make*! A fire extinguisher may be a good idea too if you get it wrong and your system decides to burn . So, try at your own risk. I am not liable, seriously. Be safe!
 
Last edited:

Geert

Major Contributor
Joined
Mar 20, 2020
Messages
1,944
Likes
3,548
My approach. After generating a convolution file in rePhase it shows you the max output (response) level. Going from that value, adjust the gain of the filter in rePhase (General tab > Output settings) so when you generate the file again is shows 0dB output. The clip and hold indicators in Brutefir or CamillaDSP don't show any clipping with this simple approach.
Screenshot_20230301_191332.jpg
 
Last edited:

levimax

Major Contributor
Joined
Dec 28, 2018
Messages
2,388
Likes
3,515
Location
San Diego
My only concern with relying on "software" to test if other "software" is working may not take into account how the whole system including the DAC is reacting. Rather than rely on "software checks" (which I have found unreliable under some circumstances) I play the 0 dB sweep and listen (with volume turned way down of course). I am usually not big on "trust my ears" but you would have to be deaf to not hear digital clipping on a 0 dB sweep (very loud second higher pitched sounds during the sweep). I have found I need about 3 dB more attenuation than calculated by my software to prevent digital clipping a 0 dB sweep.
 

Geert

Major Contributor
Joined
Mar 20, 2020
Messages
1,944
Likes
3,548
Seems very unlikely to me that CamillaDSP or Brutefir's output levels or clipping indicators would malfunction. All they need to do is translate the level of the digital signal into a number or clipping notification. If your DAC still clips then that might have to do with intersample overs, but that does not have anything to do with convolution filters. If your DAC is subject to it then adding 3dB of headroom is a good solution. (I also do this, in the convolver gain settings and not in the convolution file).
 
OP
S

shaunp

Member
Joined
Oct 15, 2022
Messages
13
Likes
1
My only concern with relying on "software" to test if other "software" is working may not take into account how the whole system including the DAC is reacting. Rather than rely on "software checks" (which I have found unreliable under some circumstances) I play the 0 dB sweep and listen (with volume turned way down of course). I am usually not big on "trust my ears" but you would have to be deaf to not hear digital clipping on a 0 dB sweep (very loud second higher pitched sounds during the sweep). I have found I need about 3 dB more attenuation than calculated by my software to prevent digital clipping a 0 dB sweep.
Hi Levimax,

Yes, there are definitely other sources of distortion that may be present in the whole system aside from convolution clipping (I'm just honing in on this 'slice' of the problem in this thread) . Identifying and addressing other sources of noise needs a more comprehensive approach (which would include convolution clipping avoidance, I'd think). As an initial guess, I'd imagine measuring the system output with a measurement mic and checking for distortion in REW could help identify clipping distortion at other points of the system (maybe studying this could be my next project, thx!).
 
Last edited:
OP
S

shaunp

Member
Joined
Oct 15, 2022
Messages
13
Likes
1
My approach. After generating a convolution file in rePhase it shows you the max output (response) level. Going from that value, adjust the gain of the filter in rePhase (General tab > Output settings) so when you generate the file again is shows 0dB output. The clip and hold indicators in Brutefir or CamillaDSP don't show any clipping with this simple approach.View attachment 268539
Thanks for the tip. I have to run now, but really want to try your approach. If the results are the same, it would be so much simpler - thx!
 
OP
S

shaunp

Member
Joined
Oct 15, 2022
Messages
13
Likes
1
My approach. After generating a convolution file in rePhase it shows you the max output (response) level. Going from that value, adjust the gain of the filter in rePhase (General tab > Output settings) so when you generate the file again is shows 0dB output. The clip and hold indicators in Brutefir or CamillaDSP don't show any clipping with this simple approach.View attachment 268539
Hi Geert,

Thanks again for sharing your approach. I just tried it and will share my results. Basically, I also saw no clipping with a 0dB sweep when I get the max response level down to 0dB. When I do the ABS/SUM checking in my approach it ends up being >1 (I get ~2.6), so I'd have to reduce the gain further if I really want to get to <1.

All in all, I think adjusting gain down to get the max response to 0dB is simply a good thing. I saw it's also recommended in the Rephase tutorial by pda0 and Bear (link below - pp14). Whether there's additional benefit of the "<1" approach and spending a bit more headroom, the jury is honestly still out IMO. I'll probably continue with it personally, mainly based on the DSP site discussion I found earlier. My gut feel is either a) I'm off and will spend a little more in headroom for no extra clipping prevention benefit or b) maybe there are some extra clipping cases hiding beyond a 0dB sweep that I can avoid. I wish I knew definitively which one I was getting :)

Anyway, thanks again!

Rephase Tutorial by pda0 and Bear:
 

Geert

Major Contributor
Joined
Mar 20, 2020
Messages
1,944
Likes
3,548
I wish I knew definitively which one I was getting
Like I said, I trust on CamillaDSP. The API https://github.com/HEnquist/camilladsp/blob/master/websocket.md offers following function: 'GetClippedSamples': get the number of clipped samples since the config was loaded.

But if you want to avoid intersample overs in your DAC it might be wise to leave 3dB headroom anyway https://benchmarkmedia.com/blogs/application_notes/intersample-overs-in-cd-recordings#:~:text=Intersample Overs are a Common Occurrence in CD Recordings&text=This means that there are,peaks that exceed 0 dBFS.
 
OP
S

shaunp

Member
Joined
Oct 15, 2022
Messages
13
Likes
1
Like I said, I trust on CamillaDSP. The API https://github.com/HEnquist/camilladsp/blob/master/websocket.md offers following function: 'GetClippedSamples': get the number of clipped samples since the config was loaded.

But if you want to avoid intersample overs in your DAC it might be wise to leave 3dB headroom anyway https://benchmarkmedia.com/blogs/application_notes/intersample-overs-in-cd-recordings#:~:text=Intersample Overs are a Common Occurrence in CD Recordings&text=This means that there are,peaks that exceed 0 dBFS.
Wow, thanks. A lot to unpack here :)

For CamillaDSP's websocket API, I haven't really checked it out. I'll try sending some JSON when I get the chance. Good to know about that command - thx

For intersample overs, I guess I should start checking into this since I haven't checked into DAC stage issues/solutions at all yet. That article is a good read (also a lot to unpack!). I think I understand how keeping an additional 3dB of headroom available can help. I'll probably want to check how to measure so I can see any current issues in my setup and to confirm they disappear after giving 3dB headroom. I saw some posts around here that l that seem like they'll explain how to measure so I have some reading to do. Not sure if overs can exceed 3dB yet, so will check this too. Thanks very much for the info and setting my mind on this problem and helping me collect some ideas about solutions for DAC stage issues!
 
Last edited:

Sokel

Master Contributor
Joined
Sep 8, 2021
Messages
6,042
Likes
6,061
Wow, thanks. A lot to unpack here :)

For CamillaDSP's websocket API, I haven't really checked it out. I'll try sending some JSON when I get the chance. Good to know about that command - thx

For intersample overs, I guess I should start checking into this since I haven't at all yet. That article is a good read (also a lot to unpack!). I think I understand how keeping an additional 3dB of headroom available can help. I'll probably want to check how to measure so I can see any current issues in my setup and to confirm they disappear after giving 3dB headroom. I saw some posts around here that l that seem like they'll explain how to measure so I have some reading to do. Not sure if overs can exceed 3dB yet, so will check this too. Thanks very much for the info and setting my mind on this problem and helping me collect some ideas about solutions for DAC stage issues!
There is some content here about intersample overloads along with measurements for filters,etc.
It may be useful.

 

Geert

Major Contributor
Joined
Mar 20, 2020
Messages
1,944
Likes
3,548
For CamillaDSP's websocket API, I haven't really checked it out. I'll try sending some JSON when I get the chance.
You should know there's a Python library to work with the API. Might be easier to do a quick test: https://github.com/HEnquist/pycamilladsp

Make sure to run CamillaDSP with the websocket enabled (and choose a port number), by defaults it's off.

For intersample overs, I guess I should start checking into this since I haven't checked into DAC stage issues/solutions at all yet. That article is a good read (also a lot to unpack!). I think I understand how keeping an additional 3dB of headroom available can help. I'll probably want to check how to measure so I can see any current issues in my setup and to confirm they disappear after giving 3dB headroom. I saw some posts around here that l that seem like they'll explain how to measure so I have some reading to do. Not sure if overs can exceed 3dB yet, so will check this too.
In my experience (tests) a lot of Hifi DAC's have this problem. Testing your DAC is easy if you have a ADC interface and a test signal (you can find on the web, and in different ASR threads like the one mentioned above).

In practise most intersample overs are less than 3dB. If intersample overs are audible and how often you run into audible ones in practice is still under discussion, so personally I wouldn't waste more then 3dB headroom on it.
 

pos

Addicted to Fun and Learning
Forum Donor
Joined
Feb 13, 2018
Messages
574
Likes
720
Hi there!

Filter clipping is an interesting topic, and one with many layers to consider, most of which have already been addressed in a way or another in this thread.

First off, let me stress the fact that IIR is no different than FIR in this regard: implementing the same minimum-phase correction in IIR and FIR will produce the same output levels and potential overs. In fact the same goes for analog filters, albeit of course without the risk of clipping (but still, saturation can happen).

The first obvious thing to consider is of course the correction itself (ie filter response): ideally you don't want any EQ point to exceed the 0dB mark, as any 0dBFS signal at these frequencies would get clipped. In rePhase this can be checked directly on the magnitude curve with any loaded measurement bypassed, or with the max response level value as @Geert noted above. This can be considered good practice to handle this adjustment within the FIR (ie adjust the fader in the General tab), but can also be done on the input, or anywhere around the convolution as long as the signal is still in floating point format.

Now, even in this case you can still get clipping when the filter is applied (and once again the same would occur with IIR), because any non linear-phase correction will change the timing at which the different frequency components of the signal are summed. An all-pass filter is a good potential example, but any EQ or filter will also be subject to that phenomenon. This cannot be triggered with a sweep (or only marginally so) as this is essentially a single frequency signal. You can indeed imagine complex test signals that will produce higher output levels, but such worst-case signals would be correction-dependent. Real-world signals will most probably produce less overs than you might fear in practice.

On top of all that, and as mentioned above, you also have to consider that keeping your output signal clear of any clipping is not a guaranty that it will not clip in the ASRC or when applying the reconstruction filter in the DAC. This can be checked by resampling your signal to some high sample rate (eg 384kHz or 768kHz) and check for clipping: you basically turn inter samples into real samples.

All in all you take the risk of clipping your signal when applying filters, but the good news is that when using digital volume control (again either on the input, or anywhere around your filtering porcess where the signal is still in floating point) you don't really have to address it directly with dedicated headroom. You can simply integrate it in your "global" headroom.
If you set your levels so that a 0dBFS signal at the output of your processor produces the maximum level your amp/speakers/ears/neighbors can handle, and if in this situation you still get an idle noise level low enough that it is not audible at your listening positions, then you are good to go. This calls for devices with a high enough dynamic range, and good level matching between them to maintain it throughout. Of course you also need a well-behaved system, ie no on/off noises, no bug susceptible of producing 0dBFS signals, no cabling issues, etc.
When approaching that max level you will increase the probability of hitting some corner cases and clip the signal during the filtering process, but in the same time your output stage, amp, speakers and ears will also see their distortion increase. This can all be factored-in and handled together, and the filtering process will most probably not be the worst offender there. In this scenario you don't really need a DAC with headroom for inter sample overs, and you can even use a DAC with rising distortion levels towards 0dBFS.
 
Last edited:

Geert

Major Contributor
Joined
Mar 20, 2020
Messages
1,944
Likes
3,548
On top of all that, and as mentioned above, you also have to consider that keeping your output signal clear of any clipping is not a guaranty that it will not clip in the ASRC or when applying the reconstruction filter in the DAC. This can be checked by resampling your signal to some high sample rate (eg 384kHz or 768kHz) and check for clipping: you basically turn inter samples into real samples.

I didn't mention it before but how I deal with that is doing all DSP in 96kHz, and I maintain this frequency until it enters the DAC (requires good understanding of the full audio path to assure there's nothing downsampling anywhere in between if you have a DAC that doesn't show the input format). This reduces the chance that the DAC still manages to create intersample overs.

(Regarding maintening the sample frequency to the DAC, the same goes for the floating number high bith depth used to do volume/gain control).
 

KSTR

Major Contributor
Joined
Sep 6, 2018
Messages
2,733
Likes
6,103
Location
Berlin, Germany
Now, even in this case you can still get clipping when the filter is applied (and once again the same would occur with IIR), because any non linear-phase correction will change the timing at which the different frequency components of the signal are summed. An all-pass filter is a good potential example, but any EQ or filter will also be subject to that phenomenon. This cannot be triggered with a sweep (or only marginally so) as this is essentially a single frequency signal. You can indeed imagine complex test signals that will produce higher output levels, but such worst-case signals would be correction-dependent. Real-world signals will most probably produce less overs than you might fear in practice.
Yeah, allpass filters can produce up to 3x FS signals when hit with clipped snare or kickdrum signal. 2x FS is often seen in practice with global phase-unwrapping convolution FIR kernels and aggressively produced/mastered music. EQ/filters for active speakers I've also seen to reach 2x FS and beyond with transient signals even when the magnitude frequency response is always below 0dB.

This is also true for analog filters as it is for IIR like you mentioned.

For FIR processing we know the theoretical max output with a very specific worst-case signal ... hardly a signal we will ever encounter in practice and thus way too much headroom wasted.

I've personally settled for <0dB gain at any frequency and a headroom for 2x FS (so, actually <-6dB gain at any frequrency) and soft-clip when the signal approaches output FS. For digital this means we'd scale level by 0.5x, then ideally upsample to 4x or 8x and soft-clip to a bit below 0dBFS, or alternately use a fast look-ahead peak limiter rather than upsampler + soft-clipper.
The audible effects of clipped vs. non-clipped intersample overs is irrelevant IMHO, we'd just make sure we don't have wraparounds in the digital processing. The systematic overload from phase shifts even with less than 0dB gain at any frequency is way more impertant in real life.
 

OCA

Addicted to Fun and Learning
Forum Donor
Joined
Feb 2, 2020
Messages
676
Likes
498
Location
Germany
Hi there!

Filter clipping is an interesting topic, and one with many layers to consider, most of which have already been addressed in a way or another in this thread.

First off, let me stress the fact that IIR is no different than FIR in this regard: implementing the same minimum-phase correction in IIR and FIR will produce the same output levels and potential overs. In fact the same goes for analog filters, albeit of course without the risk of clipping (but still, saturation can happen).

The first obvious thing to consider is of course the correction itself (ie filter response): ideally you don't want any EQ point to exceed the 0dB mark, as any 0dBFS signal at these frequencies would get clipped. In rePhase this can be checked directly on the magnitude curve with any loaded measurement bypassed, or with the max response level value as @Geert noted above. This can be considered good practice to handle this adjustment within the FIR (ie adjust the fader in the General tab), but can also be done on the input, or anywhere around the convolution as long as the signal is still in floating point format.

Now, even in this case you can still get clipping when the filter is applied (and once again the same would occur with IIR), because any non linear-phase correction will change the timing at which the different frequency components of the signal are summed. An all-pass filter is a good potential example, but any EQ or filter will also be subject to that phenomenon. This cannot be triggered with a sweep (or only marginally so) as this is essentially a single frequency signal. You can indeed imagine complex test signals that will produce higher output levels, but such worst-case signals would be correction-dependent. Real-world signals will most probably produce less overs than you might fear in practice.

On top of all that, and as mentioned above, you also have to consider that keeping your output signal clear of any clipping is not a guaranty that it will not clip in the ASRC or when applying the reconstruction filter in the DAC. This can be checked by resampling your signal to some high sample rate (eg 384kHz or 768kHz) and check for clipping: you basically turn inter samples into real samples.

All in all you take the risk of clipping your signal when applying filters, but the good news is that when using digital volume control (again either on the input, or anywhere around your filtering porcess where the signal is still in floating point) you don't really have to address it directly with dedicated headroom. You can simply integrate it in your "global" headroom.
If you set your levels so that a 0dBFS signal at the output of your processor produces the maximum level your amp/speakers/ears/neighbors can handle, and if in this situation you still get an idle noise level low enough that it is not audible at your listening positions, then you are good to go. This calls for devices with a high enough dynamic range, and good level matching between them to maintain it throughout. Of course you also need a well-behaved system, ie no on/off noises, no bug susceptible of producing 0dBFS signals, no cabling issues, etc.
When approaching that max level you will increase the probability of hitting some corner cases and clip the signal during the filtering process, but in the same time your output stage, amp, speakers and ears will also see their distortion increase. This can all be factored-in and handled together, and the filtering process will most probably not be the worst offender there. In this scenario you don't really need a DAC with headroom for inter sample overs, and you can even use a DAC with rising distortion levels towards 0dBFS.
REW will always assign an SPL offset of 109dB, 117dB or 129dB to rePhase generated impulses which would change by the exact amount of gain applied in rePhase before export. Do you think removing these SPL offsets in REW before re-exporting them should be a concern for clipping?
 
Last edited:

Geert

Major Contributor
Joined
Mar 20, 2020
Messages
1,944
Likes
3,548
REW will always assign an SPL offset of 109dB, 117dB or 129dB to rePhase generated impulses which would change by the exact amount of gain applied in rePhase before export. Do you think removing these SPL offsets in REW before re-exporting them should be a concern for clipping?

Why do you import rePhase files into REW and then export them again?
 

OCA

Addicted to Fun and Learning
Forum Donor
Joined
Feb 2, 2020
Messages
676
Likes
498
Location
Germany
Why do you import rePhase files into REW and then export them again?
Usually to convolve different filters into one i.e. a XO phase shift correction filter created in rePhase with a magnitude inversion filter created in REW
 

Geert

Major Contributor
Joined
Mar 20, 2020
Messages
1,944
Likes
3,548
Usually to convolve different filters into one i.e. a XO phase shift correction filter created in rePhase with a magnitude inversion filter created in REW

In that case you need to be careful with the total gain of the combined filter in REW indeed. I don't have experience with this approach so I would need to examine how to manage that in REW. In case of doubt, just look at the volume and clipping indicators of your convolver (e. g. CamillaDSP) to check if your filters don't have neutral gain.
 
Top Bottom