Loudspeaker measurement precision - What to aim for?

[ctrl]

What accuracy should be aimed for when measuring loudspeakers with the Klippel NFS?

The Klippel NFS specification claims +-0.1dB tolerance in direction of of max SPL, which is really impressive. But only if as many test objects as possible are measured and checked against other sources it's ensured that all error sources are excluded.

If you intend to doubt the published measurements of manufacturers, renowned magazines or acknowledged specialists in audio measurement technology (as quite a few have done in the review thread of the Neumann KH 80), you should make sure that your own measurements meet the highest standards.

If not, no real trust can be built. To always claim that others are measuring wrong can only be successful if there is no doubt about your own measurements.

The accuracy should definitely exceed the possibilities of a "bloody" DIY developer who measures in his own living room.


What accuracy is possible for measurements in living rooms?

About 18 months ago a test was carried out among members of a DIY-forum. A small 3-way loudspeaker, equipped with inexpensive drivers and without crossover network, was measured (driver by driver) by the driver manufacturer in the company's own anechoic chamber (RAR) and then sent to 13 interested members for measurement in their own living room (some also made measurements in the garden or in a gym).

The loudspeaker consists of a tweeter, a fullrange driver, which are arranged asymmetrically at the front, and a side-mounted woofer.

The challenge was to measure the individual chassis as accurately as possible without knowing how these chassis were measured in the anechoic chamber by the manufacturer.

The expectations on my part regarding the accuracy of my own measurements were, as you can also think, not very high.

Especially since it turned out that my chosen reference point for the measurement did not correspond to the one for the anechoic chamber measurement. The blue cross corresponds to my chosen reference point for the measuring microphone, the red cross corresponds to the manufacturer measurement.

The test results
My living space measurements are reasonably accurate up to about 300-400Hz. For frequencies that are lower, a near-field measurement has to be added - which I did without. Since I did not know which reference point was used for the RAR measurements, my measurements were taken at a distance of 1.50m, which is unusual for indoor measurements.
First the comparison of the tweeter measurements (in red the anechoic measurement, in blue my measurement):

Same measurements normalized to the anechoic chamber measurement:

Below 800Hz the sound pressure has already dropped too much for an exact measurement in my living room.

In the range 1.5kHz to 4kHz there is different diffraction at the speaker cabinet due to the different reference points.

A better result should be achieved with the full range driver, since the influence of the cabinet edges is less pronounced there (due to the different reference points).

Comparison of the full range driver measurements (in red the anechoic measurement, in blue my measurement):

Same measurements normalized to the anechoic chamber measurement:

Despite the adverse conditions (wrong reference point) it was possible to achieve almost +-1dB accuracy with the indoor measurement compared to a measurement in an anechoic chamber. I found this quite impressive.

To cut a long story short, the demand on the accuracy of the measurements made here in the forum by Amir with the Klippel NFS should be much higher.

The end

 

[Pio2001]

To cut a long story short, the demand on the accuracy of the measurements made here in the forum by Amir with the Klippel NFS should be much higher.

Thanks for opening this thread.
My point of view is a bit different. The demand on the accuracy of the measurements must not necessarily be high just because it can. It should be high enough to assess the quality of the speaker.
How high is it ? 0.1 dB is an absolute threshold, as human hearing won't hear smaller differences anyway. But is it necessary to reach such accuracy ? Not if other factors are introducing bigger errors.

One source of error is the accuracy of the microphone after calibration.
Another is the variation from sample to sample for a given speaker model.
Yet another one is the minimum deviation from a setup to another because of room acoustics below the transition frequency (above, our hearing might be able to sort the direct sound from the reverberation in the room).

The accuracy of the measurement setup must be better that all of these, in order not to add an extra source of variation on top. But maybe twice better is enough, since the above ones are already adding up and messing up the data.

A side question is the reference axis. What if the manufacturer's reference axis is wrong ? What if none is specified ? How large is the variation from one reference axis to the next ?

I repost here some measurement that I've done with a Neumann KH-120 monitor, Umik-1 microphone, 2 ms windowed (enough to remove any reflections except from the arm holding the microphone, the ruler, and the microphone cable), 70 cm in front of the speaker, holding the microphone by hand, along a ruler to help me find the right position. 10 takes in front of the tweeter, then 10 take in front of the reference axis:

I then applied a 1/3 octave smoothing to see if something could be seen in average :

At 70 cm in front of the speaker, the error for the on-axis curve, averaged across 1/3 octave, below 10 kHz, peaks at 0.4 dB at 4000 Hz.
Above 10 kHz, the maximum 1/3 octave deviation reaches 0.6 dB at 13 kHz.

 

[HammerSandwich]

It should be high enough to assess the quality of the speaker.

Do we also wish to assess the quality of the manufacturer's specifications? If we're interested solely in the DUT's fidelity, then what panther goes to a DAC specced at 110dB SINAD but measured at 95?

 

[Pio2001]

The error caused by the choice of an axis of reference mostly impacts the on-axis curve. It is expected to be in the +/- 0.5 dB range. Maybe more, maybe less, according to the speaker design. More often than not, we won't have a reference axis, as the manufacturer doesn't always give one.

But the other curves should be less impacted by it. Therefore we should just keep in mind that the on-axis curve is no more accurate than +/- 0.5 dB, except maybe for speakers used in very dry acoustic conditions (RT60 less than 0.2 s) or for near-field listening (less than 1 meter away from the speakers).

I have no idea about the accuracy of the Klippel's microphone.

Sample to sample variations for a given speaker model should not be better than the one advertised by Neumann, which is probably state-of-the-art : +/- 0.26 dB. It will certainly be quite higher for other speakers.

About the error that we get in low frequencies because of placement in the room, here is a kind of worst case scenario :

That's the room correction for neutral speakers in my living room (read the curve upside down to see the room's response itself). In the context of this discussion, these deviations should be considered as very bad: large window just behind the speakers, causing the 100 Hz peak (acoustic dip), and probably the 200, 340 and 560 Hz dips (acoustic peaks). And two strong room modes caused by hard walls at 54 and 69 Hz.

We should expect a much, much better behaviour in dedicated rooms, and the accuracy of the measurements will have to comply with that accuracy. Not with mine.

 

[Pio2001]

Do we also wish to assess the quality of the manufacturer's specifications? If we're interested solely in the DUT's fidelity, then what panther goes to a DAC specced at 110dB SINAD but measured at 95?

That might be interesting, but this is not the purpose of the measurements in my opinion. I am more interested in the quality of the speakers than in the quality of the user manual.
If we can see, during the measurement, that the specs are not met, then it is worth mentioning in the review, but as I understand it, the goal of the measurement setup is not to check if the advertised specs are right or wrong.

 

[Juhazi]

ctrl, your measurements show very long IR right window gating! How much?

Typical domestic measurements show first reflection around 4-5ms, and right window should be shorter (depending on it's curve). This eliminates boundary bounce and gives enough resolution roughly above 600-1000Hz. In a large and high living room, speaker on a stand and aimed diagonally, 6ms can be reached.

Of course we can use longer gating or RTA, but then reflections come to play and we must learn to see them and eq them out in our mind. This is rational, but curves are not worth showing to others!

Measuring individual drivers shoud be done closer to speaker/driver. Often 50-60cm is enough to grab baffle interferences and it shifts first reflections even 2ms further. These measurements reveal cone resonances etc,problems. Directivity can be measured this way too for single drivers. Distortion is most reliable this way too, but be careful not to overload the microphone!

Bass driver must be measured in nearfield (also port sound), for simulation, but the complete speaker's bass QC measurement is best done using ground plane method, preferably outdoors. This is best for distortion measurements too, because we can use high voltage/spl, we can go all the way down without rattling furniture!

When we measure for crossover work, distance must be at least 1m, preferably 1,5-2m. Then we get in trouble with reflections too and we must recognize their effect. We don't need to look at response details anymore, we can use eg. 1/3oct smoothing. Timing/phase match is not harmed by reflections. mid-tweeter is easy, but woofer-mid around 200-500Hz is extremely difficult indoors (better to trust simulations?)

The final outcome shuld be measured in farfield at least 1,5m, we want to see on- and off-axis responses and GD or step response.

More about measurements for loudspeaker design (different task from room response/modes measurements!)
https://www.soundandvision.com/content/speaker-measurements-101
https://kimmosaunisto.net/Software/VituixCAD/VituixCAD Measurement Preparations.pdf
https://en.wikipedia.org/wiki/Loudspeaker_measurement
https://www.linkwitzlab.com/Loudspeaker-Room/tests&measurements.htm
http://audiojudgement.com/measure-speaker-frequency-response/
http://audiojudgement.com/near-field-speaker-measurement-using-soundeasy/
https://www.prosoundtraining.com/2014/04/24/ground-plane-measurements/

And we need to measure much more than spl response!
https://data-bass.com/#/articles/5cc0bc36a75a260004255c88?_k=mg1fc6

 

[ctrl]

The error caused by the choice of an axis of reference mostly impacts the on-axis curve. It is expected to be in the +/- 0.5 dB range. Maybe more, maybe less, according to the speaker design. More often than not, we won't have a reference axis, as the manufacturer doesn't always give one.

When large loudspeakers with horn are measured, the difference is much greater.
A JBL M2 measured on the tweeter axis will differ significantly from measuring between horn and woofer. Especially when Amir measures in his garage using NFS with 30cm distance to the speaker.

I have no idea about the accuracy of the Klippel's microphone.

If Klippel claims that the NFS works with +-0.1dB accuracy (on the axis with highest sound pressure), then the errors caused by the microphone should already be included there.

Measuring individual drivers shoud be done closer to speaker/driver. Often 50-60cm is enough to grab baffle interferences and it shifts first reflections even 2ms further. These measurements reveal cone resonances etc,problems.

Yep, but the challenge was, without knowing where the reference point for the anechoic chamber measurement was, to get as close as possible to this measurement.
Therefore I chose a distance of 1.5m to keep the error caused by a wrong measuring angle as small as possible.
The price for this was a small gate of only 3.32ms, which smoothes everything below 1kHz.

Bass driver must be measured in nearfield (also port sound), for simulation

As mentioned in Post#1, I did not mention this here because I only wanted to show the accuracy with which DIY indoor measurements can be performed, especially in high frequencies.


I think nobody expects measurements with 0.1dB accuracy, but aiming at +-0.5dB should be the goal.

But how can Amir be sure that everything really works flawlessly, only by comparing his own measurements to anechoic chamber measurements of the same speaker. Maybe with his connections to Harman he can borrow a loudspeaker (with the associated anechoic chamber measurements) and then measure it with the NFS.
The next best thing is to measure loudspeakers that are as linear as possible on axis and compare them with the manufacturer measurements and other independent anechoic chamber measurements.

And we need to measure much more than spl response!

Of course, but first of all the basics should work as error-free as possible.

 

[ctrl]

Another small remark, quite a few have argued in the KH 80 thread that a few dB deviation in accurancy on axis does not matter, because the angle measurements and the indexes calculated from them will compensate for this.

It should be noted, however, that Klippel "only" gives an accuracy of +-1dB for angle measurements.

Therefore, it cannot be the goal to measure with +-2dB on axis, plus another +-1dB under angle.

The errors thus accumulated can also affect the generated indexes.

 

[Hipper]

Surely most of us will be using these measurements to compare speakers, not for exact results.

I can see that exact results would be useful to compare with manufacturers' and others' specifications/results but I'm looking more for 'which speaker is better'.

 

[Juhazi]

Here is NFS measuring a tall Magico speaker - distance looks like more than 1m, but there is not lateral shift in the video and the room looks too small for that (perhaps the speaker is on a turntable).

https://www.facebook.com/video.php?v=609534989795383

In this Warkwyn video the scanner rotates, but the mic is very close to the speaker 30-50cm, which is bad for large speakers.

https://www.facebook.com/video.php?v=1252153234854627

 

[Pio2001]

When large loudspeakers with horn are measured, the difference is much greater.
A JBL M2 measured on the tweeter axis will differ significantly from measuring between horn and woofer.

But in this case, are there several possible reference axis ? Won't the front of the treble horn be the reference ?

 

[Juhazi]

JBL doesn't tell it exactly, but we have the manual, pages 6-7
https://adn.harmanpro.com/site_elem...571427306/M2_Owners_Manual_Rev.A_original.pdf

"The M2 is designed so that when placed directly on the floor, the ears of a seated listener will be in the center of the speaker’s vertical coverage window, eliminating the need to elevate the speaker. " (that is roughly 42" which is midpoint of the waveguide)
"This table provides minimum listening distance recommendations for a seated listener when the speaker is elevated. As an example, when the speaker is elevated 20 inches above the floor, a seated listener should be 92 inches or greater from the speaker"

Brochure https://adn.harmanpro.com/site_elem...1_1556556110/M2_Brochure_Jan2013_original.pdf

 

[ctrl]

I can see that exact results would be useful to compare with manufacturers' and others' specifications/results but I'm looking more for 'which speaker is better'.

Hypothetical case: Imagine you are looking for a near-field studio monitor and have two to choose from.

The measurement accuracy is +-2dB on axis (let's ignore the accuracy under angle).

Monitor A has a 1dB hump around 8kHz that slowly disappears with increasing angle - an insignificant error.

Monitor B has a design flaw and causes a 2dB dip at 8kHz due to interference, which does not disappear even with increasing measurement angle (I know, extremely hypothetical) - an significant error.

If our measurement system has a 2dB bump at the same position, this will be effective for all measurement angles.
Now we measure both monitors with our measuring system.

I think you have a pretty good idea where this is heading.

Monitor A will show a hump with 3dB on axis, which also shows up in the "listening window".

Monitor B will be perfect in the 8kHz range.

Therefore, even when comparing loudspeakers alone, it is important that the measurement system works as precisely as possible.

 

[napilopez]

Thanks for starting this thread @ctrl. Probably good to move some of the discussion away from the Neumann thread. That way the discussions can be divorced from feelings people have about a particular speaker. And I wasn't aware about that DIY vs anechoic test. Cool stuff.

Personally, I've never much doubted the reliability of indoor data and DIY measurements, even before I started making my own. Assuming a reasonably low noise floor, then gated data for all intents and purposes *is* anechoic. The data doesn't "see" the reflections, so it's as if they don't exist. If I remember correctly, the Klippel even uses gated measurements for the upper portion of its frequency response too. It's just when resolution starts to drop that things get complicated.

Surely most of us will be using these measurements to compare speakers, not for exact results.

I can see that exact results would be useful to compare with manufacturers' and others' specifications/results but I'm looking more for 'which speaker is better'.

I agree with you in a sense. I think it's very important to make a distinction between measurements that matter for designing a speaker and the most important measurements that matter for assessing and recommending speakers. For example, CSD might be useful when designing speakers, but it says very little in their assessment because audible issues would show up in the spinorama.

To me the important things are frequency response and directivity since those are the only things we can tie directly to preference. I consider SPL compression with active speakers an extension of these since the frequency response is altered. With volume.

But I think what some people weren't grasping in the Neumann thread is that some seemingly inoccuous differences would be audible, and that they could be much worse with different speakers.

We are more sensitive to low q deviations than high Q ones. So I think small differences in squiggles are relatively innoccuous, but broad scoops, tilts, slumps and bumps are concerning. This image from a Toole And Olive study (1988) shows the threshold of audibility of resonances. While resonances are more audible than single-curve deviations, I believe the trends are representative.

We are very sensitive to low q deviations, easily less than a fraction of a dB. It takes a *much* larger amplitude to hear a high-q deviation. Moreover, it's easier to hear deviations in high frequencies, which somewhat helps the DIY crowd, as that's gated measurements have most resolution.



And some speakers show big deviations with small changes in the reference axis. Hence the concern. I posted this in the KH80 thread, but here is how the Buchardt S400 measures when off by just 10 degrees vertical. 5 degrees is somewhere in between the on axis and 10 degrees, and still a notable deviation.

S400 10 degrees above reference axis:

S400 10 degrees below reference axis:

Chords 806 10 degrees below reference axis:

For these particular speakers, these deviations are not too audible in because they are intended for far field listening, where you get more reflections to balance things out. The S400, For example, is pretty unique in that it is better controlled at larger angles than within the listening window. But they definitely sound worse in the nearfield because of these issues.

Luckily, I think as Amir tests more and more speakers it will be become easier to notice when things seem off. A lot of measuring speakers is trial and error. The Klippel NFS may reduce a lot of the error, but experience is nevertheless useful. I think he might see why some of us were concerned with his results for that particular speaker down the road. Who knows.

 

[NTK]

FWIW, here is the verbiage in CTA-2034-A, appendix A. It says for non-anechoic measurements, which IMO the Klippel NFS should be counted as one, an agreement to within ±1.5 dB of a known good anechoic chamber is considered good.

 

[napilopez]

FWIW, here is the verbiage in CTA-2034-A, appendix A. It says for non-anechoic measurements, which IMO the Klippel NFS should be counted as one, an agreement to within ±1.5 dB of a known good anechoic chamber is considered good.

View attachment 47557

Fair point. I'm not sure if the klippel should really count in this regard though considering the claim(and my belief) is that it is as accurate or more so than an anechoic chamber for SPL measurements. That seems worded for outdoor and gated measurements.

 

[ctrl]

It says for non-anechoic measurements, which IMO the Klippel NFS should be counted as one, an agreement to within ±1.5 dB of a known good anechoic chamber is considered good.

This is due to the high error rate of the composite curve of a quasi free field measurement. For this purpose we combine the low-frequency near-field response and the gated quasi-free-field response.

This is where the Klippel NFS shows its advantages. Instead of performing a near-field measurement with a baffle correction for the woofer, the NFS calculates the disturbing room reflections from the measurement and can therefore theoretically be more accurate than an anechoic chamber at very low frequencies.

Therefore we can expect the NSF measurements to reach the level of an anechoic chamber.