NatAmbio: Open-source DSP framework for stereo reproduction

[RaulFO]

Hi everyone,

After many years developing it as a personal DSP project, I've finally decided to release NatAmbio as open source.

NatAmbio is a real-time DSP framework for GNU/Linux focused on stereo reproduction experiments. It combines three independent processing stages that can be used separately or together:

• NAE (NatAmbio Ambient Extractor): stereo signal decomposition using PCA to separate the principal and ambient components.
• XTC: FIR-based crosstalk cancellation for loudspeaker listening.
• PCA4DRC: a method that applies PCA to multiple room impulse responses to generate a representative impulse response for digital room correction.

Everything is released under the GPL-3.0 license and includes source code, documentation, examples and installation guides.

GitHub project:

GitHub - raul-fernandez-ortega/natambio: Real-time stereo ambience extraction, crosstalk cancellation (XTC) and room correction (DRC) for immersive loudspeaker reproduction using low-cost hardware

Real-time stereo ambience extraction, crosstalk cancellation (XTC) and room correction (DRC) for immersive loudspeaker reproduction using low-cost hardware - raul-fernandez-ortega/natambio

github.com

The repository also contains three technical papers describing the algorithms and their implementation in detail.

This is not a commercial project—just something I've enjoyed researching and developing over the years. In fact, NatAmbio is the DSP processor I use every day for listening to music at home. I finally decided it was time to share it.

Feedback, measurements, criticism and ideas are all very welcome.

I hope you find something interesting in it.

 

[Newman]

Thanks for doing this, Raul. All communities that benefit from the OS community can be grateful for the spirit behind it.

 

[LinuxGuru]

Hi, RaulFo, thanks for so great contribution. Is is possible to deploy NetAmbio on RaspberryPi? Or Pi doesn't have enough CPU power for so complex software?

 

[RaulFO]

Is is possible to deploy NetAmbio on RaspberryPi? Or Pi doesn't have enough CPU power for so complex software?

Hello LinuxGuru,

I don't have any experience with Raspberry Pi, so I'm afraid I can't give you reliable advice.

From a computational point of view, NatAmbio is not especially demanding. Although the DSP algorithms behind it are quite sophisticated, the actual processing cost is relatively modest.

The NatAmbio ambient extraction stage has a CPU cost roughly comparable to the first convolution partition in the zita-convolver library. The remaining processing consists almost entirely of FIR convolutions, so CPU usage mainly depends on the filter length.

The aspect I have always found more challenging with Raspberry Pi is audio interface support. I'm used to older FireWire interfaces (through FFADO), and as far as I know those are not a practical option on Raspberry Pi. I don't have enough experience with modern USB audio interfaces on Raspberry Pi to comment on their performance or compatibility.

So, the honest answer is that I simply don't know. However, NatAmbio's CPU requirements are relatively modest. I've documented its CPU usage on my Intel N100 system in the GitHub documentation, where it is around 5%:

https://github.com/raul-fernandez-ortega/natambio/blob/main/docs/hw_setup_en.md

 

[RaulFO]

I've just published a small collection of open-source tools for experimenting with stereo reproduction and XTC (crosstalk cancellation).

Testing_XTC

The repository includes listening test signals and scripts that let you modify and explore the XTC algorithm itself, making it easy to experiment with different parameter settings and immediately hear their effect.

There are two ways to use the tools:

• A real-time version integrated into NatAmbio under GNU/Linux and JACK.
• A completely offline version that only processes WAV files, requiring no NatAmbio, no Linux, and no JACK. The offline scripts can also be used to experiment with headphone listening, although I haven't explored that application myself (yet!).

While developing these tools, I found it particularly interesting that two largely independent mechanisms seem to emerge:

• the well-known ITD/ILD spatial cues;
• and another related to panning together with correlation and anticorrelation relationships.

Whether this interpretation is ultimately correct remains an open question, but I believe the tools themselves may be useful for anyone interested in understanding how XTC changes the perceived stereo image.

The current release focuses on practical testing and experimentation. The perceptual analysis behind these observations is still a work in progress and will be documented separately once it has matured.

Feedback, experiments and alternative interpretations are very welcome.

 

[RaulFO]

As an audio lover, I really enjoy spending time (and money!) on hardware and developing open-source software for my home audio system. These are typical activities in this hobby, but from time to time I also like to think about the perceptual models behind our stereo systems.

Personally, I do not think that stereo has been superseded by multichannel audio. I think there is still considerable room for exploration, particularly when DSP-based crosstalk cancellation (XTC) is introduced into the reproduction system.

For me, the best way to explore perception in stereo systems, especially with XTC, is through small experiments at home. The experiment I introduced yesterday seems particularly interesting to me. I have been experimenting with it on my system and have reached some surprising conclusions — at least, surprising to me.

Apparently, I have found a way to create a localization sweep for a mono signal in which the apparent position of the sound is manipulated through a combination of conventional panning and anticorrelation.

The process begins like conventional panning: the level of one channel is progressively reduced. But instead of stopping when that channel reaches zero, its polarity is inverted and its level starts increasing again. The full panning experience does not seem to end when one channel is muted. It continues as that channel becomes increasingly anticorrelated with the opposite channel. In my experiments, the apparent extreme position occurs at around a 3 dB level difference between the correlated and anticorrelated signals.

Why can this combination produce such a strong and apparently continuous spatial displacement?

In conventional spatial-hearing models, localization is primarily associated with ITD and ILD, as represented, for example, in HRTF-based models. And this approach also works with XTC: the other localization sweep tool I presented is based on these cues. Yet, in an XTC reproduction system, a controlled combination of panning and interchannel anticorrelation can also produce a very clear change in apparent spatial position.

I am not suggesting that these mechanisms are equivalent. My question is more basic: what perceptual mechanism allows increasing anticorrelation, combined with panning, to act as such an effective spatial cue under XTC reproduction?

If we can better understand how anticorrelation is perceived in this context, it may become a very interesting tool for manipulating stereo sound scenes, rather than simply a signal property to be measured or avoided.

As you can probably tell, I am a little obsessed with correlation, decorrelation and anticorrelation. I think they provide a fascinating framework for exploring spatial perception in stereo systems.

I am curious to hear how others interpret this.

 

[bmc0]

My understanding is that below ~500Hz, horizontal localization depends largely on ITD estimated from one-way zero-crossing times (after frequency separation by the cochlea). Above ~2kHz, a combination of ILD and ITD is used where the ITD is estimated from transient onsets.

I wouldn't expect that inverting one channel would work very well at high frequencies, but it does result in significant interaural phase differences at low frequencies. Here's a demonstration with a very simple HRTF approximation (first-order lowpass at 700Hz plus a delay for the opposite ear):

The first is the left input (speaker) only. Second and third have the right input at -3dB with positive and negative polarity, respectively.

 

[RaulFO]

I completely agree with you about the role of ITD and ILD in sound localization. This is the classic HRTF model, and it has been measured very precisely in many labs.

My intention here is just to do some brainstorming. Let's think about what stereo really is. Stereo is to real sound localization what a FIFA game on a PlayStation is to a real football match. OK, just a small joke.

Stereo is a simulation of the real sound world. And, as in any simulation, some things from the real world work quite naturally, while others do not work exactly as we could expect. Two speakers, each one playing a different signal, can create the illusion of a sound field.

In this situation, the role of ITD and ILD is not so simple to model. We do not have only one sound source reaching two ears, but four acoustic paths: each speaker reaches both ears.

And if stereo is a simulation, I think there may be DSP algorithms that can improve or extend this simulation.

My interest in correlation and anticorrelation did not start from a theory. I do not have one. It came from a very simple experiment while I was trying to maximize the XTC effect at home. What I wrote about lateralization and anticorrelation is simply what I perceived with my audio setup.

This is why I share these simple tools here: as a kind of open brainstorming. I would like to understand how this effect is connected with ITD and ILD, and maybe use that understanding to build new and useful DSP effects for stereo reproduction.

 

[john61ct]

The aspect I have always found more challenging with Raspberry Pi is audio interface support. I'm used to older FireWire interfaces (through FFADO), and as far as I know those are not a practical option on Raspberry Pi. I don't have enough experience with modern USB audio interfaces on Raspberry Pi to comment on their performance or compatibility.

With RPi5 support for standard 1x PCIe cards, Firewire interfaces are possible.

As of course are PCIe interfaces, RME HDSPe being very affordable these days.

USB interfaces have been used routinely, even RPi4b likely has plenty of DSP power for this use case.

Given Class compliance, or proper ALSA driver support.

And ARM compatibility of software used of course.

Finally, the 8ch HiFiBerry DAC and add-on ADC may not have the same pristine measurements as pro interfacii

but they make for a Very Cheap system for experimenting, and for many home listening contexts, are likely good enough SQ.

that is, if only 8 channels are enough.

 

[bmc0]

We do not have only one sound source reaching two ears, but four acoustic paths: each speaker reaches both ears.

Yes, and that is what is shown in my examples.

I don't have speakers set up for XTC to properly test it myself, but here's a suggestion for an experiment: try a signal which has been sharply highpass filtered at around 2-3kHz. Does high negative correlation still cause significant lateral displacement of the image?
In my brief experiments with a conventional 2-channel stereo setup, I perceive what you describe only at lower frequencies (below 1kHz or so).

 

[RaulFO]

In my brief experiments with a conventional 2-channel stereo setup, I perceive what you describe only at lower frequencies (below 1kHz or so).

That is an interesting experiment, thanks for the suggestion. I will try it.

My experience is somewhat different, although this may depend on speaker geometry, azimuth, etc. I really don't know. As a curious anecdote, I have also tested musical signals: for example, a Wynton Marsalis trumpet solo moving across the scene ("The Legend of Buddy Bolden" – City Movement). The effect is clearly audible, even without XTC, although with much more moderate spatial displacement. Of course, tones and complex musical sources may not use exactly the same perceptual mechanisms, so I find your proposed test interesting.

Regarding your plots, I don't doubt that the spectra are correct. I think perhaps we are asking slightly different questions. My question is not only what signals reach the ears, but where the virtual source is perceived. I am thinking about some kind of predictive model: what set of signals would move a virtual source to a particular perceived location?

Thanks again for the suggestion. I will try the high-pass experiment and report what I perceive.

 

[bmc0]

I think perhaps we are asking slightly different questions. My question is not only what signals reach the ears, but where the virtual source is perceived.

I wonder if you may have missed part of my point in post #7 as I think it shows a possible explanation for the perception you describe, at least for lower frequencies (<1kHz or so).

First, the mechanisms for estimating ITD are known to be different for low frequencies vs high frequencies. At low frequencies, what is actually detected is the interaural phase difference: the neural firings synchronize to one-way zero crossings of a signal which has been filtered through a narrow bandpass (the cochlear filter). As the detected phase difference between the ears grows, the perceived image moves further to the side (up to a point, beyond which it tends to split into two images).

Comparing plots #1 (left speaker only) and #3 (left and right, where right is -3dB and inverted), it can be seen that the predicted interaural phase difference is much larger in #3 below 1kHz. The perceived image location should then be the extreme left, well beyond the simulated ±30° locations of the speakers.

 

[RaulFO]

With RPi5 support for standard 1x PCIe cards, Firewire interfaces are possible.

As of course are PCIe interfaces, RME HDSPe being very affordable these days.

USB interfaces have been used routinely, even RPi4b likely has plenty of DSP power for this use case.

Given Class compliance, or proper ALSA driver support.

And ARM compatibility of software used of course.

Finally, the 8ch HiFiBerry DAC and add-on ADC may not have the same pristine measurements as pro interfacii

but they make for a Very Cheap system for experimenting, and for many home listening contexts, are likely good enough SQ.

that is, if only 8 channels are enough.

Great information, thank you!

As I mentioned before, I'm familiar with Mini-ITX solutions, but I'm completely new to the Raspberry Pi world, so this is very useful to know.

 

[RaulFO]

I wonder if you may have missed part of my point in post #7 as I think it shows a possible explanation for the perception you describe, at least for lower frequencies (<1kHz or so).

Sorry if it seemed that I overlooked your post #7. I agree that it is a very plausible explanation for the phenomenon, at least at low frequencies.

The question I have in mind is a slightly different one. How could I intentionally generate a virtual source at, say, a 60° azimuth using loudspeakers placed at ±20°? And then make that virtual source move smoothly from -60° to +60°?

That is the kind of question I am trying to explore.

 

[john61ct]

familiar with Mini-ITX solutions, but I'm completely new to the Raspberry Pi world

CM5 version of RPi5 can mount a x1 standard PCIe card on that formfactor board

Not cheap though

Sentinel Core

Sentinel Core

A Mini-ITX Raspberry Pi CM5 I/O board with PCIe

www.crowdsupply.com

 

[RaulFO]

I have expanded the NatAmbio XTC technical note:

https://github.com/raul-fernandez-ortega/natambio/blob/main/docs/xtc/xtc_filters_en.md

The new section makes it possible to follow the complete reasoning path from the original iterative cancellation model to the more conventional matrix formulation commonly used in XTC literature.

Starting from the successive cancellation and recancellation terms, the direct and cross filters can be grouped into a symmetric matrix. The resulting series converges to the inverse of the normalized crosstalk-coupling matrix:

The important point is that NatAmbio does not invert the complete acoustic matrix H. It only approximates the inverse of the normalized coupling term:

Therefore, after ideal crosstalk cancellation:

The direct acoustic path is deliberately preserved. XTC is responsible for spatial decoupling between the two channels, while the remaining direct-path response — loudspeaker response, room modes and spectral coloration — is the responsibility of conventional DRC.

This distinction is important because it clarifies which part of the overall stereo-system response should be treated by XTC and which part should be equalized by DRC, as well as why both operations should remain separate.

It also reinforces a practical point I have already mentioned: XTC should normally be used together with DRC. XTC can improve spatial separation and apparent image width, but if the direct response of the loudspeakers and room is not adequately controlled, the resulting spatial impression may still be unbalanced or tonally inconsistent.

In practice, XTC and DRC solve two different but complementary parts of the same acoustic problem:

For this reason, I consider the combined use of XTC and DRC not merely desirable, but in many systems almost essential for obtaining a convincing and stable spatial result.

 

[digitalfrost]

I have a hard time calling something finished, but I didn't fiddle with this for some days now, so I guess it's time:

It's a stereo-in stereo-out plugin for Windows x64 and Linux x64 that implements the NatAmbio Ambient Extractor and crosstalk-cancellation in a VST3. That should make it much easier for people to test out NatAmbio without much setup. You can load it into any VST3 host, for example Audacity https://www.audacityteam.org is free and cross-platform. The VST3 should also compile on OSX, but I cannot provide a OSX binary at the moment.

GitHub - digitalfrost84/NatAmbio-VST3: Independent VST3 adaptation of NAE and XTC inspired by Raul Fernandez Ortega's NatAmbio.

Independent VST3 adaptation of NAE and XTC inspired by Raul Fernandez Ortega's NatAmbio. - digitalfrost84/NatAmbio-VST3

github.com

For context, I made an ambiophonics plugin of my own based on the original algorithms with some custom tweaks. After seeing the post here I integrated NatAmbio into it and basically decided to abandon what I had built because NatAmbio was just that much better.

The ambiophonics effect you can achieve without coloration or affecting the timbre is the best I heard yet.

The plugin comes with a lot of tests to make sure it behaves well under strange, irregular or even dangerous signals, as well as that is is faithful to the original NatAmbio implementation.

The default settings are conservative. Be careful when increasing ILD GAIN it can start ringing.

You can slide the right side out to get more information. The idea is that stuff you rarely need is hidden on the right side, so unless you need the info or want some eye candy you can use the compact UI.

Just set the output trim in the beginning so it's not clipping depending on your gain staging and you are good to go.

As for the settings I've added a tuning guide here https://github.com/digitalfrost84/NatAmbio-VST3/blob/main/docs/TUNING.md but when in doubt always refer to the original https://github.com/raul-fernandez-ortega/natambio documentation.

I did extensive testing under Windows, I have not tested it under Linux. So if you encouter any issues feel free to open a github issue or DM me. If anybody manages to compile and run on OSX would also be interesting to hear from you.

 

[iGude]

After many years developing it as a personal DSP project, I've finally decided to release NatAmbio as open source.

Awesome, this is great work, thank you very much for sharing!

It's a stereo-in stereo-out plugin for Windows x64 and Linux x64 that implements the NatAmbio Ambient Extractor and crosstalk-cancellation in a VST3.

Very nice, thank you very much, too.
Quick question though: in the original work from RaulFO there are two stereo dipoles, one in front and one behind. Wouldn’t this require four output channels?
Or where is the extracted ambience output from the plugin?

 

[digitalfrost]

It only models the front dipole at the moment. It still contains the cross-talk cancellation for the front dipole so the effect works, even in stereo. Yes if the rear dipoles were added it would be a 2-in 4-out plugin. As I do not have rear speakers and I cannot test it, I implemented this first. But it shouldn't be too hard to add the rear dipole as well. If somebody has a 4 channel setup and wants to test it I could provide a version for this.

 

[iGude]

It only models the front dipole at the moment. It still contains the cross-talk cancellation for the front dipole so the effect works, even in stereo. Yes if the rear dipoles were added it would be a 2-in 4-out plugin. As I do not have rear speakers and I cannot test it, I implemented this first. But it shouldn't be too hard to add the rear dipole as well. If somebody has a 4 channel setup and wants to test it I could provide a version for this.

Fair enough, but in this case does it make sense to extract the ambience?
Or would it be possible to use two instances of your plugin running in parallel, one for the front (ambience level slider put to zero) and the other one for rear (front level slider put to zero)?