32 Bit Float Explained

[Eurasian]

32-Bit Float Files Explained - Sound Devices

The MixPre II models introduce the ability to record 32-bit floating point WAV files. For ultra-high-dynamic-range recording, 32-bit float is an ideal recording format. The primary benefit of these files is their ability to record signals exceeding 0 dBFS. There is in fact so much headroom that...

www.sounddevices.com

 

[DonH56]

Note that the mantissa sets the resolution, exponent sets the ultimate dynamic range. You get a huge dynamic range using 32-bit floating-point numbers but resolution is limited to 24 bits, still more than enough for audio, but it is not the same resolution as a 32-bit converter. Not that anyone makes or needs a 32-bit ADC or DAC for audio...

 

[shuppatsu]

A very silly format from the perspective of an audio engineer. But it makes more sense from the perspective of how efficiently CPUs and DSPs process 32-bit float.

 

[earlevel]

Note that the mantissa sets the resolution, exponent sets the ultimate dynamic range. You get a huge dynamic range using 32-bit floating-point numbers but resolution is limited to 24 bits, still more than enough for audio, but it is not the same resolution as a 32-bit converter. Not that anyone makes or needs a 32-bit ADC or DAC for audio...

25 equivalent bits of mantissa—besides the normalized (omitted) leading "1", don't forget the sign bit.

 

[antcollinet]

A very silly format from the perspective of an audio engineer. But it makes more sense from the perspective of how efficiently CPUs and DSPs process 32-bit float.

Do they process it more efficiently than 32 bit fixed point?

 

[earlevel]

Do they process it more efficiently than 32 bit fixed point?

As DSP is going to either be optimized for fixed point of floating point, so take what they give you. CPUs are optimized for floating point, and generally don't support fixed point on-chip, so floating point will beat coded fixed point. Also, 32-bit float is the sample format of things like Core Audio. It's doesn't really make sense to do anything other than floating point on a host computer system, for instance. The exception would be taking advantage of hardware vector math (SIMD) for parallelism, but even that has gone float on modern processors.

 

[voodooless]

CPUs are optimized for floating point, and generally don't support fixed point on-chip, so floating point will beat coded fixed point.

Basically any general purpose CPU architecture was fixed point from the get go. Floating point was always a tacked on feature. The only reason why in some cases nowadays floating point math is faster is because even more tacked on instructions (as you mentioned yourself), like AVX, that can do array multiplication really fast. But there are fixed points versions as well, often used for certain AI workloads (but some need floating point for this).

Edit: please see one down..

 

[earlevel]

Basically any general purpose CPU architecture was fixed point from the get go. Floating point was always a tacked on feature. The only reason why in some cases nowadays floating point math is faster is because even more tacked on instructions (as you mentioned yourself), like AVX, that can do array multiplication really fast. But there are fixed points versions as well, often used for certain AI workloads (but some need floating point for this).

Ummm..."integer", not "fixed point", from the get go. Whether you roll your own or use a library, if you want 32-bit fixed point suitable for audio processing, it's going to cost you more than float. DSP chips such as the venerable 56k family, had 24-bit fixed point built in, but not CPUs.

That is, with integer math, the binary point is fixed to the right of the lsb. For audio processing, you want it on the other end, basically (all the way, for 24-bit; for 32-bit you can afford some headroom). Since a CPU won't let you define where the binary point is, you have to use code to manage it. And you need to manage a higher resolution accumulator in software. Floating point handles those details for you.

 

[voodooless]

Ummm..."integer", not "fixed point", from the get go.

Ah yes, now I see what you mean. Yes, CPUs
Don’t have fixed point math generally.

 

[antcollinet]

Ummm..."integer", not "fixed point", from the get go. Whether you roll your own or use a library, if you want 32-bit fixed point suitable for audio processing, it's going to cost you more than float. DSP chips such as the venerable 56k family, had 24-bit fixed point built in, but not CPUs.

That is, with integer math, the binary point is fixed to the right of the lsb. For audio processing, you want it on the other end, basically (all the way, for 24-bit; for 32-bit you can afford some headroom). Since a CPU won't let you define where the binary point is, you have to use code to manage it. And you need to manage a higher resolution accumulator in software. Floating point handles those details for you.

Isn't fixed point just integer with a magnitude/bitshift? Addition/subtraction "just works". Multiplication/division requires an additional shift operation?

 

[KMO]

Isn't fixed point just integer with a magnitude/bitshift? Addition/subtraction "just works". Multiplication/division requires an additional shift operation?

Yes, but that "just" is a big deal.

A SIMD CPU unit might do, say, 4 integer multiplies in 1 cycle, or 4 float multiplies in 1 cycle.

If it doesn't natively support a fixed-point form of multiply which gets the binary point in the right place, then that adjusting shift is a whole extra instruction. Even assuming you don't worry about the loss of precision or overflow - as an extra wide intermediate would cost more too.

Then adding one extra shift to every multiply halves the speed. CPUs designers have spent so long optimising multiplies, that shifts and multiplies are now largely equivalent - the CPU above could do 4 shifts in 1 cycle. So the floats can end up faster than fixed point.

However, SIMD instruction sets in CPUs used for DSP generally DO have fixed-point multiply forms that their original instruction sets lack(ed) - it's not uncommon to have SIMD-only instructions - so they can process integer audio data effectively. It just can be hard to access those fixed-point instructions in high-level languages that were originally designed with only scalar floating-point and integer in mind.

 

[sarumbear]

A very silly format from the perspective of an audio engineer. But it makes more sense from the perspective of how efficiently CPUs and DSPs process 32-bit float.

Really? A format that gives you the ability to record audio with no possibility of clipping without having to adjust the gain is silly?

In my view what is silly is commenting on a subject that you obviously have no grasp of it.

Here is a range of recorders that are created purely because of the format's existence.

32-Bit Float: Everything You Need to Know

zoomcorp.com

 

[BeerBear]

Really? A format that gives you the ability to record audio with no possibility of clipping without having to adjust the gain is silly?

In my view what is silly is commenting on a subject that you obviously have no grasp of it.

Here is a range of recorders that are created purely because of the format's existence.

32-Bit Float: Everything You Need to Know

zoomcorp.com

That's some misleading marketing from Zoom. The magic doesn't come from using 32bit float, but from using multiple ADCs stacked on top of each other.
You could just as easily record the same signal with 32bit integer. But Zoom pretends that 32bit integer doesn't exist, even though some devices have been using it for a long time and, unlike 32bit float, it works with the standard Windows audio driver.
Hell, even 24bit integer would likely be enough. Julian Krause measured the Zoom UAC-232 and, despite it having dual ADCs, the dynamic range is still "only" 136dBA.

 

[sarumbear]

That's some misleading marketing from Zoom. The magic doesn't come from using 32bit float, but from using multiple ADCs stacked on top of each other.
You could just as easily record the same signal with 32bit integer. But Zoom pretends that 32bit integer doesn't exist, even though some devices have been using it for a long time and, unlike 32bit float, it works with the standard Windows audio driver.
Hell, even 24bit integer would likely be enough. Julian Krause measured the Zoom UAC-232 and, despite it having dual ADCs, the dynamic range is still "only" 136dBA.

I think you are mistaken. The figures you gave are for dual ADC mode of the device. If you watch the video you will see that the device has a 32-bit float mode as well but that is accessible via an ASIO driver. That is a limitation of Windows not of the device. However, it’s not a reason for it not to be tested. ASIO is a standard method for any serious audio processing.

 

[antcollinet]

Yes, but that "just" is a big deal.

A SIMD CPU unit might do, say, 4 integer multiplies in 1 cycle, or 4 float multiplies in 1 cycle.

If it doesn't natively support a fixed-point form of multiply which gets the binary point in the right place, then that adjusting shift is a whole extra instruction. Even assuming you don't worry about the loss of precision or overflow - as an extra wide intermediate would cost more too.

Then adding one extra shift to every multiply halves the speed. CPUs designers have spent so long optimising multiplies, that shifts and multiplies are now largely equivalent - the CPU above could do 4 shifts in 1 cycle. So the floats can end up faster than fixed point.

However, SIMD instruction sets in CPUs used for DSP generally DO have fixed-point multiply forms that their original instruction sets lack(ed) - it's not uncommon to have SIMD-only instructions - so they can process integer audio data effectively. It just can be hard to access those fixed-point instructions in high-level languages that were originally designed with only scalar floating-point and integer in mind.

Thanks for that explanation. My knowledge of CPU operations per clock cycle is probably about 25 years out of date - from the days of arithmetic co -processors

 

[DonH56]

25 equivalent bits of mantissa—besides the normalized (omitted) leading "1", don't forget the sign bit.

IEEE Standard is 1 sign bit + 8 exponent bits + 23 mantissa bits, thus 24-bit resolution (23 + sign). There are other FP formats but that is the only one I remember off-hand (not a digital guy so I could well be wrong).

 

[BeerBear]

The figures you gave are for dual ADC mode of the unit. The unit has a 32-bit float mode but that is available only via an ASIO driver. That is a limitation of Windows not of the unit.

There's no "dual ADC mode". The device automatically records using dual ADCs and 136dBA of dynamic range is what it can do. Unless @Julian Krause made a mistake there and tested it at 24bit or something, the digital format is not the bottleneck here.
I suggest also watching another video Julian made, where he talks about 32bit float recording more generally.

And Zoom wasn't the first one to use multiple ADCs. Sonosax SX-R4+ came out in 2016 if not earlier and it had 135dB of dynamic range. Here's what they said (source):

The signal can be recorded as a 24 or 32-bit stream. There is however no quality loss, as 24-bit is enough to encode a 135dB dynamic stream.

 

[sarumbear]

There's no "dual ADC mode". The device automatically records using dual ADCs and 136dBA of dynamic range is what it can do. Unless @Julian Krause made a mistake there and tested it at 24bit or something, the digital format is not the bottleneck here.
I suggest also watching another video Julian made, where he talks about 32bit float recording more generally.

And Zoom wasn't the first one to use multiple ADCs. Sonosax SX-R4+ came out in 2016 if not earlier and it had 135dB of dynamic range. Here's what they said (source):

It seems you are not understanding what the reviewer is saying. Please listen from this time onwards.

Here are the screen grabs from the video.

Here is the 32-bit float description from Zoom. Can you see the similarities to Sound Devices page linked above?

The solution to clipped and distorted audio is a larger dynamic range, and 32-bit float offers that in spades. It contains over 4 billion amplitude values, resulting in a dynamic range louder than anything found on earth. In other words, everything from whispers to explosions (and virtually anything that you’d encounter on set) falls well within its dynamic range.

32-Bit Float: Everything You Need to Know

zoomcorp.com

 

[BeerBear]

No, it's you who doesn't seem to understand what the Zoom Mix Control software is doing...

When you boost the signal there you're amplifying an already captured digital signal. In other words, it's the same as if you imported a file into a DAW and boosted the volume. You won't get clipping of course, because a DAW typically operates at 32bit float internally.
The only difference is that this Mix Control software/driver lets you do this in real time, before it even reaches an app, which can be useful.
But you're not getting an increased ADC recording dynamic range. That's still limited to 136dBA.
And the reason the device doesn't clip in typical use is because it has dual ADCs with separate gain levels. If you feed it extreme input levels it's still going to clip and the 32bit float recording format won't save you. I suggest you carefully watch that second video I posted.

 

[sarumbear]

But you're not getting an increased ADC recording dynamic range. That's still limited to 136dBA.

I suggest you complain to Zoom for lying if their device manages 136dB according to this tester but they specified over 192dB.