Modern Measurement Tools Are Tricking Audiophiles Into Trusting Bad Data, Warns Veteran Speaker Designer

[a4eaudio]

Well, I do. The mathematics behind Klippel NFS is impenetrable by any acoustic designer who doesn't have masters degree in signal processing. Here is the formula from NTK thread:
...

There is no way, no how, a speaker design has any idea what this equation means.

...

Strong claim. I'm sure few people would understand 100% of the math behind NFS, but "any idea"? You posted a single equation with a double summation, not exactly impenetrable.

 

[whereisbrian]

Modern Measurement Tools Are Tricking Audiophiles Into Trusting Bad Data, Warns Veteran Speaker Designer​

When I read this kind of sentence in the header of an article, I always ask myself the right questions (at least I hope so).

He mentions his veteran status, in other words, he highlights his experience, which would then allow him to legitimize his criticisms.
It's a common "I know" (and you don't know) approach.

The second question is what his objective is.
One might initially think that this is for educational purposes, when in fact it's primarily about denigrating a certain type of measure.

 

[Curvature]

You cannot predict my personal preference--times I don't like any speaker because it makes some noise, that simple. Conversely, all the ratings depend on an unspecified mindset of the audience, if we nonchalantly go statistics to begin with. The mindset when sitting in "that room" at Harman's may differ from that of a couch potato. The latter type is more common than audiophiles tend to consider.

You are human, yes? Shared biology and psychology.

Every science demonstrates that we fall into groups and patterns. Why would psychoacoustics and audio be any different? Olive's work with headphone preference shows that people, generally speaking, are fairly close to each other on an average basis and then diverge into clear groups based on the amount of bass or treble emphasis they prefer. Same goes for speakers. There is consistency and a clear design blueprint: no resonances, no directivity errors, deep bass, low distortion. Those few rules are general enough to allow many types of speakers to satisfy them.

 

[Ageve]

Mofi SourcePoint 10, Erin's Audio Corner (2m, NFS) vs Stereophile (1.27m / 50 inches, gated + nearfield bass):

0/10/30 deg horizontal average for Erin's data, to match Stereophile.

Ignore < 300Hz (No baffle edge diffraction correction applied to Stereophile measurement)

1/6 smoothing:

This is my attempt at correcting the Stereophile bass response (stitched together at 300 Hz). Dubious method to say the least (without full measurement data), but still pretty close):

https://www.stereophile.com/content/mofi-electronics-sourcepoint-10-loudspeaker

https://www.erinsaudiocorner.com/loudspeakers/mofi_10/

 

[HalSF]

For those who might miss it, here's the actual talk by Mr. Jones (a little over an hour)

I would be disinclined to take this headphonesty person seriously or trust that they didn't cherry-pick divisive quotes out of context in order to generate a page of ragebait for clicks. That's what they mostly seem to do, in my limited experience.

edit: accidentally misgendered

Modern life is making me even more paranoid than I naturally am, but in addition to the click-baity headline that Headphonesty summary is giving off AI vibes. I’d be careful about criticizing what Andrew Jones is supposedly arguing without citing this YouTube video and/or its transcript directly.

 

[Curvature]

Even in the case of other more complex speaker designs, I've not seen evidence of inaccuracy. Lots of speculation, but no evidence.

Example of a highly complex speaker: https://www.spinorama.org/compare.h...CBT+70J-1&origin1=Vendors-JBL&version1=vendor

JBL CBT70J. I love the theoretical work and results of Keele's effort.

I am specifically comparing the normalized measurements. They show high agreement. The raw measurements, as you'll see if you compare on-axis, are very different not due to measurement error but because the speaker has several settings and the one chosen by Amir doesn't match JBL's default.

 

[sam_adams]

On the contrary, the biggest shysters and shillers of snake oil come wearing suits n' shirts.

What someone wears, or chooses not to wear in these modern times has very little bearing on expertise.

Really . . . ?

 

[kyuu]

Really . . . ?

View attachment 482179

Yes really? The fact that he's wearing a shirt has approximately zero to do with anything.

Thread has devolved into arguing about clothing, and the usual suspects using this to advance their personal feelings about why the Klippel NFS doesn't measure their preferred speakers correctly. Wonderful.

 

[jhaider]

Try using a Klippel NFS on a planar panel design, or a fully horn loaded design (multiple aperture)--you'll immediately see the problems that Mr. Jones was [allegedly] quoted as saying. But if you put those type of loudspeakers into an anechoic chamber and back off with the microphone distance to 3m (which is the typical listening distance of floorstanding loudspeakers)

I made no claim of understanding the math — it’s well above the training of my lawyer brain. However, let’s pressure test this statement.

There have been many complex-pattern speakers measured on NFS by @amirm or Erin: Erin has measured at least two MEH (a Danley and a DIY) as well as Dutch&Dutch 8C (front firing MT, cardioid vents for mid, rear firing subs). @amirm has measured tall multi-way towers such as Revel 328 as well as a CBT array. These measurements, compared with other data for the same speakers, tend to disprove your assertion.

From my view, the only notable discrepancy is polar pattern at extreme HF. NFS is always narrower. I believe that is easily explainable: NFS is not so advanced as to dynamically rotate the microphone. It is aimed straight. Capsules themselves have HF rolloff, which results in less HF (>10kHZ) captured. By contrast you see the microphone arrays in chambers generally aimed in an arc, with each one aimed on axis. So the HF looks narrower on NFS because both the speaker off axis rolloff and the mic off axis rolloff is reflected.

And as to MEH specifically, Mr. Danley has been writing since the Basslist days that you can stick your head in the mouth and it sounds the same. So any argument about summation distance required for such speakers invalidating the NFS approach seems especially ill-founded.

 

[amirm]

I’m fascinated by the NFS’s ability to calculate far field response from close mic measurements in the context of the DUT being a large multi-way speaker where layman understanding suggests that correct/intended driver summation doesn’t occur until the measurement position is far enough away, something like 2-3x largest driver spacing. Is there a qualitative explanation you can help with as I highly doubt I could understand the maths (which I found hard enough when taking my acoustics degree 25 years ago!). Thanks.

There is no discontinuity in wave propagation. What happens at close distance will be a direct predictor of what happens far.

That aside, if that point of convergence is far, then typical measurement distance of 3 meter will be wrong as well. Indeed, Dr. Toole mentions that the 2 meter distance in CEA-2034 spec is a compromise for such tall speakers. Klippel doesn't have this compromise. If the source is complex, you just need more measurement points to compute its far field results accurately.

 

[amirm]

Compared to earlier methods, the Klippel system is probably accurate - but how do we define accurate?

You don't know it but we do. NFS is self-checking. It compares its computed results to actual measurements and computes the degree of match. This plot is always provided although unless I have a good reason, I don't show it. Here is a recent example:

We see that as soon as the tweeter takes over, the soundfield becomes hugely more complex due to many reflections/diffractions at shorter wavelengths. And interference between the woofer and tweeter. Klippel has proposed a "good enough" test of accuracy of 1% error or -20 dB. That is more than sufficient when we are talking about speakers. But if it is not, we can increase the number of measurement points and accuracy will improve. Notice how this is in play below 1 kHz for this speaker where accuracy is incredibly good.

At low frequency accuracy is still there but when speaker output drops due to lack of bass output, error naturally increases.

Here is an example of where the error did go high, the Magnepan LRS:

Again, I could increase the measurement points and improve that but I didn't feel it merited to do so. Indeed, if you look at the on-axis error of computed vs measured, we see that the error is not that high:

This is why I chuckle when people claim anyone can use Klippel. To do so properly, you need to understand the underlying operation of the system and know when the results are good, vs not.

 

[amirm]

When the Klippel software computes the estimated in room response, what's the height of the speaker? Specifically, I mean do the computed response take into account that a bookshelf rests on a stand while a floorstander stands on the floor?

As noted, this has nothing to do with Klippel itself. That graph is part of the CEA-2034 standard, originally developed by Harman's research conducted by Alan Devantier. It was published by AES: “Characterizing the Amplitude Response of Loudspeaker Systems,” 113th Convention, Audio Eng. Soc.

Alan surveyed 15 different room's strongest reflections. He then computed a weighted sum that correlated with actual measurements of those rooms. Here is one sample result from the paper:

Of course, being a statistical average, there will be errors. But directionally, it gives you a good idea of the in-room tonality once you combine on-axis and off-axis response.

 

[amirm]

That is exactly the problem with NFS in my understanding, that it tries via iteration, to calculate a number of imaginary sound sources at unknown positions, which would fit to the soundfield it has actually measured at numerous positions in what is neither nearfield nor farfield. If the actual number of sound sources is closer to indefinite, they are far away from each other compared to the wavelength, or their phase relations towards each other are chaotic, that cannot be as accurate as claimed, as some measurements with bending-wave planar transducers or cardioids have proven.

Chaotic? If anyone designed such a speaker, you don't need measurements. You need run, run fast to buy a different speaker! Reminds of some vendor's tweeter arrays and such.

It is true that the nemesis of NFS is sound complexity. The worse it is, the more measurement points it needs to compute a higher order solution for the wave propagation. Fortunately, such speakers are to be avoided anyway as there is no way, no how they can produce the flat on-axis and smooth off-axis response you want. As I explained though, NFS produces the error data so if you were at all motivated to characterize such Frankenstein speakers, you can indeed.

Klippel NFS also has an optional module that segments the speaker, measures each segment separately and then combines the results. This is useful for measuring large speaker arrays and such. So again, this is not a limitation of NFS.

And it is not like you have a proper solution any other way. Coarse measurements on a table are not going to reveal such complexity and interference patterns either. Nor are they in sufficiently far field.

 

[amirm]

With a very huge room, you can go pretty low just with the help of time windowing.

As I explained, you then run into temperature gradients and phase errors. You will also have play the speaker at very high SPLs if the microphone is very far, pushing it into compression and such.

 

[amirm]

Btw you do not need to go flat as low as 20Hz. Klippel does not do that accurately as well. Otherwise you would never get a reading of negative directivity index with speakers that solely employ omnidirectional bass sources without major phase differences. Not a big deal, though, as the frequency plot itself is not too far off in most of standard cases, but indirect indication of limitations of the calculation. It simply should not happen.

There is no limitation in NFS. The operator has to optimize the parameters for bass when those sources are not co-located. Again, I opt for good enough there as response is already way down anyway.

Same error occurs in anechoic chamber as you run into different modal response for each radiating source.

 

[amirm]

It is a good analogy. Now imagine an indefinite number of stones being thrown, slightly delayed in a chaotic manner, plus stones being thrown from under the water hitting the surface of the water, or little pumps sucking it in.

Once more, you don't want such speakers. It is the job of the speaker designer to produce a well behaved speaker that produces smooth response, not a chaotic sum of sources which likely can't even be simulated, let alone optimized. To the extent Klippel NFS reports high degree of fitting error, we already know the speaker is screwed up. We can optimize more or declare the design broken above certain frequency and not waste time on such bad designs. This all comes out of my analysis of speaker measurements.

In vast majority of speaker I test, error remains at or below 1%. If it is higher, I report it as that has rarely happened. So the problem you are imagining, is very rare and is no reason to criticize NFS for. And again, traditional measurement methods don't provide a solution to these problems.

 

[amirm]

I have the advantage of having worked with truly anechoic measurements of existing loudspeakers for decades, manyfold the number of what Amir has measured, as well as with Klippel systems, so I had the chance to compare.

Manyfold? I have measured 355 speakers. You are claiming to have measured thousands? I very much doubt it.

As for comparisons you have done, where are they? Your posts don't indicate deep knowledge of Klippel NFS. But let's see the comparisons and parameters used for testing.

BTW, who are you? Do you work for the industry? Are you an Arindal employee?

 

[amirm]

When using the Klippel NFS, do you need to specify how many drivers a speaker has and the distances between them, or can the system infer that information purely from the measurements it takes?

No. As I have explained, NFS assumes an arbitrary sound source, i.e. a black box, and uses 3-D set of measurements to compute its wave propagation. There is no way or need to specify any characteristics of the speaker.

That is a bit of exaggeration. You do have to tell NFS where the tweeter is. This is so that it knows that the source of most complex dispersion. You can choose other points if that is the center of complexity. Mis-aligning this point can cause the fitting error to shoot up so it is obvious if you have done that.

In addition to above, you can tell NFS what the reference axis is. Usually this is the same as the tweeter but if the designer specs otherwise, you can override this and choose a different point. In my experience, in far-field data, it makes very little difference as 2 meters is quite far relative to a centimeter or two difference between tweeter and reference axis.

 

[Somafunk]

Really . . . ?

View attachment 482179

Yes…..really…..

But if I posted an example of a certain suited and so called New York property tycoon that is on my mind I figure I’d annoy Rick so I’ll keep the biggest shill to myself

 

[Newman]

[to Amir] By the way, I, too dislike being credited with saying things that I didn't say. So a little reciprocity is requested.

It's the things you did say that Amir has objected to:

Try using a Klippel NFS on a planar panel design, or a fully horn loaded design (multiple aperture)--you'll immediately see the problems that Mr. Jones was quoted as saying. But if you put those type of loudspeakers into an anechoic chamber and back off with the microphone distance to 3m (which is the typical listening distance of floorstanding loudspeakers), the loudspeakers measure quite well and differently. They also tend to sound good subjectively at a reasonable listening distance. If you take any loudspeaker that's not fully coaxial and place your ear right next to a particular driver (like NFS is often doing), it typically doesn't sound very good, however. These are observations that I think we should be taking seriously.

The above taken as a whole suggests that the NFS measures too close to the speaker to give a result that represents how it will measure (and sound) at a typical listening distance. Amir has rightly objected to that.

You then reacted to criticism by repeatedly asking Amir for the NFS measuring distance or range of distances. I mean, why would you ask that, unless it was in the belief that the answer was important to whether your statement above was right or not? An incorrect belief, by the way. If you ask a question, repeatedly, and don't explain the point of the question, then it's a bit mischievous.

Then, even after Amir has explained the irrelevance of measuring distance to the Klippel NFS, you come up with another of your unexplained questions:

How large are planar dipole loudspeakers?

Hopefully, by this point it should be clear that your above question means you haven't given up on defending your original statement that I quoted above.

And then you continue by repeatedly claiming that you have not said anything disputable:

To my knowledge, I don't recall saying anything that's not true.

So, you still hold to your original statements that Klippel NFS gives false readings on larger sound sources like "a planar panel design, or a fully horn loaded design (multiple aperture)"? And that a large anechoic chamber will show that they are much better speakers than the NFS results?

hmm