• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

GPU Audio - the future of DSP?

Davide

Senior Member
Joined
Jul 6, 2020
Messages
460
Likes
171
Location
Milan, Italy
Recently I was delighted to discover the birth of this new company (link below) that is developing an audio DSP platform based on GPU (PC graphics cards).
The benefit is given by the enormous parallel computing power of modern graphics cards, which have libraries and languages to let them do every type of processing, so not only gaming and video rendering.
In this way the company essentially promises zero latency (1 ms) even with the most complex audio algorithms, that actually compared to video calculations are still much lighter.
In fact it seems already supported and followed by various exponents of the sector, and currently they have a public beta of a FIR convolution engine compatible with Nvidia 10XX cards and above (there is also a community for feedback etc).
In their roadmap, support for AMD cards is also planned for this year, as well as additional plugins. In 2023 they should release a DAW and expand support to Mac too.
On the site they also talk about offering remote processing via TCP, always with 1 ms of latency, but I can't find any info about it.
Honestly I would very much like to try FIR convolver because I think you could make a remarkable HiFi setup without having the problem induced by upsampling and linear phase filters with many TAPs (I think of some great crossovers).
Unfortunately I don't have a compatible video card at the moment, so I would like to have someone about it who may have already tried or will try thanks to my thread.
Apart from this, it seems to me very promising as a project, the idea of exploiting the power of graphics cards for audio calculations is not new, but I think there is something very concrete here, at least compared to those few plugins that already existed.

GPU audio website
 
Last edited:

tiramisu

Member
Joined
Jun 24, 2022
Messages
97
Likes
101
Anything that gets latency down is a wonderful thing.
I still use my hardware chain to avoid latency using DSP on the pc.
 
OP
Davide

Davide

Senior Member
Joined
Jul 6, 2020
Messages
460
Likes
171
Location
Milan, Italy
Maybe those are somewhat amateur projects?
I don't want to say nonsense but the GPUaudio project seems to me to have already pitted various problems related to this type of "unconventional" processing and has therefore built a solid base for development.
 

Trell

Major Contributor
Joined
May 13, 2021
Messages
2,752
Likes
3,285
On the site they also talk about offering remote processing via TCP, always with 1 ms of latency, but I can't find any info about it

Bullshit. Less than 1ms you can implement on your wired LAN but otherwise it will be far from that, unless you got dedicated lines and a bunch of very serious money.

This claim alone should be enough to put the company on your ignore list.
 
OP
Davide

Davide

Senior Member
Joined
Jul 6, 2020
Messages
460
Likes
171
Location
Milan, Italy
Bullshit. Less than 1ms you can implement on your wired LAN but otherwise it will be far from that, unless you got dedicated lines and a bunch of very serious money.
Yes, 1 ms on the internet does not exist in my opinion. Maybe they mean on the LAN, perhaps setting up a server for processing.
 

JayGilb

Major Contributor
Joined
Jul 22, 2021
Messages
1,371
Likes
2,307
Location
West-Central Wisconsin
Their focus is around vst3 plugins and minimal latency in recording environments. For home audio or home theater, latency is not nearly as much an issue.
I don't care if there is a 20ms delay before audio is processed and reaches my ears, the same goes for home theater as long as the audio/video are time aligned.
 

Trell

Major Contributor
Joined
May 13, 2021
Messages
2,752
Likes
3,285
Their focus is around vst3 plugins and minimal latency in recording environments. For home audio or home theater, latency is not nearly as much an issue.
I don't care if there is a 20ms delay before audio is processed and reaches my ears, the same goes for home theater as long as the audio/video are time aligned.

The latency over internet will be highly variable but if the company was quoted correctly and in context, it still would be on my ignore list of yet another audiofraud company.
 
OP
Davide

Davide

Senior Member
Joined
Jul 6, 2020
Messages
460
Likes
171
Location
Milan, Italy
Their focus is around vst3 plugins and minimal latency in recording environments. For home audio or home theater, latency is not nearly as much an issue.
I don't care if there is a 20ms delay before audio is processed and reaches my ears, the same goes for home theater as long as the audio/video are time aligned.
If you want to create a multi-way system with digital volume, loudness compensation, crossover, equalizer, FIR convolver and maybe a nice limiter for safety, and at the same time use the best possible processing quality to avoid aliasing, etc ... you also have seconds of latency.
Difficult to sync the video at that point. Anyway, it's very annoying to press play or forward and have all that latency.
Then that the current systems are already working well with simpler processing, I agree ... I'm simply thinking about the potential.
However yes their main purpose is the audio production market.
 
Last edited:

voodooless

Grand Contributor
Forum Donor
Joined
Jun 16, 2020
Messages
10,223
Likes
17,799
Location
Netherlands
Maybe those are somewhat amateur projects?
They are!
I don't want to say nonsense but the GPUaudio project seems to me to have already pitted various problems related to this type of "unconventional" processing and has therefore built a solid base for development.
Actually, it's not the GPU that brings low latency, but rather how the convolution is done. With partitioned convolution, one can keep the delay lower than the fir filter length. Fascinating stuff: https://www.google.com/search?q=partitioned+convolution

Here is a paper on how to do it on a GPU: https://www.researchgate.net/public...g_using_fast_convolution_on_graphics_hardware
 
OP
Davide

Davide

Senior Member
Joined
Jul 6, 2020
Messages
460
Likes
171
Location
Milan, Italy

dc655321

Major Contributor
Joined
Mar 4, 2018
Messages
1,597
Likes
2,235
Recently I was delighted to discover the birth of this new company (link below) that is developing an audio DSP platform based on GPU (PC graphics cards).
The benefit is given by the enormous parallel computing power of modern graphics cards, which have libraries and languages to let them do every type of processing, so not only gaming and video rendering.
In this way the company essentially promises zero latency (1 ms) even with the most complex audio algorithms, that actually compared to video calculations are still much lighter.
In fact it seems already supported and followed by various exponents of the sector, and currently they have a public beta of a FIR convolution engine compatible with Nvidia 10XX cards and above (there is also a community for feedback etc).
In their roadmap, support for AMD cards is also planned for this year, as well as additional plugins. In 2023 they should release a DAW and expand support to Mac too.
On the site they also talk about offering remote processing via TCP, always with 1 ms of latency, but I can't find any info about it.
Honestly I would very much like to try FIR convolver because I think you could make a remarkable HiFi setup without having the problem induced by upsampling and linear phase filters with many TAPs (I think of some great crossovers).
Unfortunately I don't have a compatible video card at the moment, so I would like to have someone about it who may have already tried or will try thanks to my thread.
Apart from this, it seems to me very promising as a project, the idea of exploiting the power of graphics cards for audio calculations is not new, but I think there is something very concrete here, at least compared to those few plugins that already existed.

GPUaudio website

Interesting project.
I fail to understand what a gpu brings to the table though? Audio dsp is not taxing and it’s possible to reduce the latency of a pipeline to that of the components external to the convolution engine.

Maybe I’m missing something.
 

voodooless

Grand Contributor
Forum Donor
Joined
Jun 16, 2020
Messages
10,223
Likes
17,799
Location
Netherlands
Maybe I’m missing something.
Well, the special use case here is low latency, real-time reverb. These are non-linear filters, and to be high quality, you need to oversample like 8x or better before you apply the filters. This makes the amount of data quite a bit larger. Also, normally these filters have a fixed delay. For real-time monitoring you’d want to have this delay as low as positive.
 

Trell

Major Contributor
Joined
May 13, 2021
Messages
2,752
Likes
3,285
Interesting project.
I fail to understand what a gpu brings to the table though? Audio dsp is not taxing and it’s possible to reduce the latency of a pipeline to that of the components external to the convolution engine.

Maybe I’m missing something.

Latching on to the use of GPU for other purposes than graphics, like the awful waste of crypto-mining?

Of course, then there is gaming and so forth that does have a good use case.
 
OP
Davide

Davide

Senior Member
Joined
Jul 6, 2020
Messages
460
Likes
171
Location
Milan, Italy
I don't know if you've ever tried mastering in a DAW with 7 or 8 plugins that do x32 upsampling and 64-bit processing (sometimes they are necessary so that the chain of these does not excessively degrade the audio quality).

The CPU runs at very high loads and the latency is just as high. And the problem is that if you have 1 second of latency and you are there fiddling with the effects, you cannot directly perceive the effect of what you are doing (it's a bit like ABX tests).

This is why taking advantage of GPUs, that already exist widely, is a valid solution.
 

DVDdoug

Major Contributor
Joined
May 27, 2021
Messages
2,916
Likes
3,831
Latency isn't much of a problem for music listening. I listen to some recordings with 60 years or more of "latency"! :p It can be a problem if you are processing audio separately from video and you get "lip sync" problems.

The big problem is with recording/monitoring where the monitoring path goes through the computer and it's hard to perform with a delay in your headphones. IMO the best way around that is simply not to monitor through the computer, and many audio interfaces offer direct-hardware zero-latency monitoring. The only issue with that is if a singer wants reverb in their headphones you can't use a computer reverb effect. (You can still have a background track from the computer in the headphone mix and latency compensation is used to time-align the tracks.)

The CAUSE of latency is mostly multitasking... The operating system is always multitasking even when you're only running one application. During recording the digital audio stream flows smoothly into a buffer (like a storage tank or long pipe). When the operating system gets around to it the buffer is read in a quick burst and written to the hard drive. If some process, driver, or application, "hogs" the system for a few milliseconds too long you get buffer overflow and a glitch in the audio. There is also a playback/monitoring buffer that works the opposite way. It gets filled in a quick burst and the data flows-out at a smooth-constant rate. If it doesn't get re-filled in-time you get buffer underflow and a glitch. Of course the buffers are also delays.

It's mostly the "other stuff" that determines latency rather than the audio processing itself but if you can speed-up the audio processing that can also help. The processor has to be fast enough to process the audio in real-time and if it's not, more latency and bigger buffers aren't going to help.

I believe the UAD effects use a dedicated DSP with it's own audio input & outputs so it completely bypasses the computer's CPU and data bus, and simply controlled by the computer. I don't know if the MINI DSP has an operating system but if it does it's not interrupting the audio processing to check/read your keyboard/mouse, or update the display, or access your hard drive, or to do other unpredictable things.

DSP by itself doesn't necessarily introduce delay (depending on the particular effect/process) but if anything can make the processing faster does leave more time for the other stuff so you might get-away with smaller buffers.

From what I understand, GPUs are very similar to DSP chips (they may even be specialized DSP chips) but I'm not sure if they are better than other DSP chips and you'd still want to operate them separately from the computer's operating system.
 

Trell

Major Contributor
Joined
May 13, 2021
Messages
2,752
Likes
3,285
I don't know if you've ever tried mastering in a DAW with 7 or 8 plugins that do x32 upsampling and 64-bit processing (sometimes they are necessary so that the chain of these does not excessively degrade the audio quality).

The CPU runs at very high loads and the latency is just as high. And the problem is that if you have 1 second of latency and you are there fiddling with the effects, you cannot directly perceive the effect of what you are doing (it's a bit like ABX tests).

This is why taking advantage of GPUs, that already exist widely, is a valid solution.

If you do this on your own GPU that could be a great solution. Over the Internet, not so much, if you need low and/or constant latency.
 
Last edited:

dc655321

Major Contributor
Joined
Mar 4, 2018
Messages
1,597
Likes
2,235
Well, the special use case here is low latency, real-time reverb. These are non-linear filters, and to be high quality, you need to oversample like 8x or better before you apply the filters. This makes the amount of data quite a bit larger. Also, normally these filters have a fixed delay. For real-time monitoring you’d want to have this delay as low as positive.

Fair enough.
Figure 1 in the paper you linked is exactly what I implemented for my own convolution engine. This was from Frank Wefer’s phd thesis. Highly recommended reading if you have not already. That thesis should be a textbook - so well written!
 

dc655321

Major Contributor
Joined
Mar 4, 2018
Messages
1,597
Likes
2,235
The CAUSE of latency is mostly multitasking..

Not typically, no.
Data processing block lengths, filters plus input data, and any offset of impulse away from minimum phase are where latencies originate.
 

Trell

Major Contributor
Joined
May 13, 2021
Messages
2,752
Likes
3,285
Well, the special use case here is low latency, real-time reverb. These are non-linear filters, and to be high quality, you need to oversample like 8x or better before you apply the filters. This makes the amount of data quite a bit larger. Also, normally these filters have a fixed delay. For real-time monitoring you’d want to have this delay as low as positive.
But not over the Internet, which is the very thing I objected to. For external GPUs, be it wired LAN or other connections, you’ll have to test the entire chain for latency. There are technologies like AVB or Dante to show that low network latency for audio is hard to implement.
 
Top Bottom