• Welcome to ASR. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Modern Measurement Tools Are Tricking Audiophiles Into Trusting Bad Data, Warns Veteran Speaker Designer

Status
Not open for further replies.
This kind of response would be unacceptable from a speaker. Why is it ok for early reflections?
Because the graphs might not show the full/correct picture.
The notion of "early reflection" is not exactly the same with QRD (aka "well diffusor") as with a hard echo from a flat wall. The wells will diffuse the reflection in time too.
These graphs do not show a FR but a polar diagram for reflection from an array. For each listener-speaker combination only one point of the diagram is relevant. And then this is only for 3kHz (and probably 0° incidence). Nevertheless, I agree, the arrangement these diagrams are based on will produce an uneven FR in the reflection at about any point.

But these graphs are calculated for summation of incoming plane waves and summation in the far field where the angle to all the panels in the array is the same.
So they do not show at all what is happening in a room, where the speaker hits every panel with a different angle (and a different corresponding FR) and the angle of deflection to the listener changes too. There are only 3 periods over the width and maybe some more over the full length of the room (from behind the speakers to behind the listener, the angle of incidence and of deflection changing ±60°).
And then there will be reflections from 6 walls (4 equipped with well diffusors) instead of one isolated array of QRDs.
And these panels are not even run of the mill QRD, the wells are faceted and curved and the (effective) order is certainly much higher than 7. So the calculation from the diagrams will not apply at all. Additionally about half of the wall surface is not QRD-like but flat or smoothly curved , the ceiling is different again. (There the space behind the wavy surface might be porous absorber or not?)
After taking all that into account and convolving over the full range of angles the result will not resemble the diagrams by a long shot. There is one thing that is clear though: the sound field will be rather diffuse and that is a good thing.

A guesstimation: If one would estimate the diffusors to be similar to a QRD30+, and considering two or three panels over an angle range of ±25° the resulting reflection coefficient might be flatter than the on axis of many speakers, but with the advantage of avoiding strong specular lateral reflections that mess up the direct sound.

As I wrote, much more details and information (measurements) are needed to judge the (unknown) result.
I would not jump to conclusions from some (simple) theoretical calculations in a book.
Instead I see a great effort and nothing wrong with the way doing it either.
 
Where is it? O/k, i didn't do Laplace transformation by hand recently, and never proved the math/ peculiarities of the Hankel functions. The concept is otherwise very clear, as always w/ maths. Don't hide it behind a curtain of strange looking signs. To argue that an successful engineer wouldn't understand the Klippel is somehow strange. And not only is AJ an experienced engineer, he is a physicist by education (post #182).
He doesn't tell in the whole article that measurements are a bad tool. He tells that measurement results in the hand of uneducated audiophiles can be misleading. I agree, fully. He says, that measurements can go wrong, agreement here. As the physicist he is, he raises the focus on measurement as science in expecting the unexpected--no button-push-publish. Same here, admittedly.
So what's the point? He questions the layman's interpretation of measurements as possibly wrong. He doesn't mean you.
Well, I agree in the sense that he actually does not talk negatively about Klippel NSF, and one should watch the video. But you can see from his listed education that his is very loudspeaker oriented, and so there are likely a lot of other topics that he has not delved into, as we all have our 'holes'. However, the typical loudspeaker engineer, in my experience, will have issues when pressed on signal processing and fundamental vibroacoustics, likely because it is an application where you can get away with it, relative to other fields that I also have a lot of experience with, such as modelling of hearing aids, and even there the engineers will have issues. If you are in a field where you have to publish a lot, you will experience a different type of review than what you will get within a company, where the product is the output, so you can carry around misconceptions for a full career without ever knowing.
 
Referring to some earlier comments about understanding how the NFS works:

I think the hard to grasp part of how the NFS works isn’t really the use of spherical harmonics (which are basically just a set of simple shapes that can be combined to represent the complex shape of the sound field), but rather how sound field separation works. That’s not obvious until you look deeper at the system and realize it’s almost a byproduct of how spherical harmonic expansion naturally separates outgoing and incoming wave components.

It seems like some clever filtering must be needed, but actually, once the sound field is expressed in terms of harmonics, separating them requires nothing more than discarding the coefficients for the incoming waves - they’re already neatly sorted by the process of using harmonics.
 
Because the graphs might not show the full/correct picture.
The notion of "early reflection" is not exactly the same with QRD (aka "well diffusor") as with a hard echo from a flat wall. The wells will diffuse the reflection in time too.
These graphs do not show a FR but a polar diagram for reflection from an array. For each listener-speaker combination only one point of the diagram is relevant. And then this is only for 3kHz (and probably 0° incidence). Nevertheless, I agree, the arrangement these diagrams are based on will produce an uneven FR in the reflection at about any point.

But these graphs are calculated for summation of incoming plane waves and summation in the far field where the angle to all the panels in the array is the same.
So they do not show at all what is happening in a room, where the speaker hits every panel with a different angle (and a different corresponding FR) and the angle of deflection to the listener changes too. There are only 3 periods over the width and maybe some more over the full length of the room (from behind the speakers to behind the listener, the angle of incidence and of deflection changing ±60°).
And then there will be reflections from 6 walls (4 equipped with well diffusors) instead of one isolated array of QRDs.
And these panels are not even run of the mill QRD, the wells are faceted and curved and the (effective) order is certainly much higher than 7. So the calculation from the diagrams will not apply at all. Additionally about half of the wall surface is not QRD-like but flat or smoothly curved , the ceiling is different again. (There the space behind the wavy surface might be porous absorber or not?)
After taking all that into account and convolving over the full range of angles the result will not resemble the diagrams by a long shot. There is one thing that is clear though: the sound field will be rather diffuse and that is a good thing.

A guesstimation: If one would estimate the diffusors to be similar to a QRD30+, and considering two or three panels over an angle range of ±25° the resulting reflection coefficient might be flatter than the on axis of many speakers, but with the advantage of avoiding strong specular lateral reflections that mess up the direct sound.

As I wrote, much more details and information (measurements) are needed to judge the (unknown) result.
I would not jump to conclusions from some (simple) theoretical calculations in a book.
Instead I see a great effort and nothing wrong with the way doing it either.
The book is by Cox and D'Antonio, the latter the inventor of the QRD. His studio designs use the QRD on rear walls to breakup the first reflections, angled side walls with absorption to increase the gap from direct sound to first reflections.

Also from the book: "As the industry moved from two-channel to multichannel, early reflections began to be viewed constructively, rather than being seen as destructive to the critical listening process. This concept was further promoted by a review of psychoacoustics research by Floyd Toole."

I've yet to see scholarly literature recommend diffusers at first-reflection points on side walls or on all walls. I've never been in such a room either, probably for good reason.
 
I don't need to know the equations of the EM fields that govern the electric motors in my car to drive it, do I? Even the car mechanic doesn't (need to) know, and most of the people who designed the whole damn thing don't (need to) know.
 
Last edited:
Most of the discussion about math seems to be a Red Herring. If you want to develop the Klippel NFS and commercialize it, then you need to know the math. If you want to measure a speaker you don't need to know that in detail anymore than I need to understand the physics of my Toyota's hybrid engine to drive around town. Klippel NFS, humongous anechoic chamber or an $80 measurement mic are different tools to measure a speaker just like a wrench, pliers, and ratchet set are different tools for turning nuts and bolts. But it is a good idea to know the pros, cons, and limitations of your tools.
 
I think there is some trouble that people with knowledge of older measurment methods make assumptions about how it works based on visuals and past experience. Then they attribute assumed pros / cons.

Myself, for many years I thought it was stiching near-field measuemnts together and merging in some manner like we often do for DIY in room measurment. It was quite a bit later I discovered my assumption / mental model was incorrect.
 
Well, I agree in the sense that he actually does not talk negatively about Klippel NSF, and one should watch the video. But you can see from his listed education that his is very loudspeaker oriented, and so there are likely a lot of other topics that he has not delved into ...
That’s a bit of an accusation, really. Just because he dared to question the typical amateur’s ability to properly interpret measurement data. Besides, the math isn’t even necessary—it’s enough to understand the concept that the math represents. I’ll gladly admit I’ve noticed the same thing with headphones: even basic concepts just don’t seem to register with some people. Instead, they’d rather argue over half a decibel and deem themselves scientific. You cannot teach them. To call out: listen! Is that so bad?
He keeps those nasty hair-splitters and "pedantic sticklers" out of his business--engineering, measurement, monetary success ...
 
FROM GOOGLE

Andrew Jones's CV

Education and expertise
  • Education: Degree in Physics with Acoustics from Surrey University and postgraduate research at Essex University focusing on computer-aided crossover network design and optimization.
  • Technical Skills: Expertise in acoustics, computer-aided design, crossover network optimization, and the development of advanced driver technologies like concentric drivers and active woofers.
  • Design Philosophy: Applies a scientific, research-based methodology to both high-end and affordable speaker designs, focusing on achieving the best possible sound quality within given constraints.

Thank you for this and will use it to emphasize that this thread should be respectful and focus on specific technical issues. Let’s watch personal attacks and stay wary of those who may be spinning content in order to gain website hits. As noted earlier, the article referenced in the OP is from a questionable source.

Let’s avoid any click and/or rage baiting and ensure the discussion here is focused and factual. Thanks!
 
Certainly not and actually I'm in the process of rebuilding my anechoic chamber at the moment. As a speaker designer I highly value being able to make a change and get measurment results in minutes. Using an NFS I'd need to wait at least 30 minutes every time I make a change and run a new 200 points scan minimum. An NFS would however be more accurate, especially below about 200hz where the chamber still shows some (very heavily damped) modes. Having both (NFS inside a chamber) would be the ultimate luxury!
Do you mind showing the process or sharing your thoughts on the design of the chamber, either here or in another thread?
 
Honestly, this sounds like a designer trying to have his measurement cake and eat it subjectively (the owner of Harbeth does something similar). Does he provide evidence of these 2-3 db errors?

Assuming there is some truth to the claim, what is the problem more specifically?
1) Would audiophiles neglect a speaker that could be great?
2) Or consider a speaker that might be flawed?
3) Or are measurements so misleading they should not be published?

For speakers, I use measurements as triage. The first triage is unavailable proper measurements. The second is proper measurements that reveal major flaws. So there is only an issue if he is arguing #3 (I don’t think he is, and I do think anyone who does should be treated very skeptically)
 
Do you mind showing the process or sharing your thoughts on the design of the chamber, either here or in another thread?

I will second as was just penning a comparable post. Will add that we already have some good established threads for DIY measurement tools. :)
 
Last edited:
That’s a bit of an accusation, really. Just because he dared to question the typical amateur’s ability to properly interpret measurement data. Besides, the math isn’t even necessary—it’s enough to understand the concept that the math represents. I’ll gladly admit I’ve noticed the same thing with headphones: even basic concepts just don’t seem to register with some people. Instead, they’d rather argue over half a decibel and deem themselves scientific. You cannot teach them. To call out: listen! Is that so bad?
He keeps those nasty hair-splitters and "pedantic sticklers" out of his business--engineering, measurement, monetary success ...
No accusation, you can clearly see what is listed education-wise. I am not pointing anything towards him specifically, but what I pointing out is that you (generic you, not you personally) cannot infer from anyone making a good loudspeaker that they can follow along in discussions on Kramers-Kronig relations, Discontinous Galerkin Finite Element Methods, Thermoviscous acoustics, or Spherical Harmonics, because the specific application they are looking at does not inherently require them too, but to work for me, they would need that PLUS loudspeaker knowledge PLUS much more. A mathematician is not likely to be able to design a good loudspeaker, so is it really that strange that loudspeaker designers are not necessarily well versed in mathematics? I don't see what the problem is.
 
Do you mind showing the process or sharing your thoughts on the design of the chamber, either here or in another thread?
Possibly, but I'll wait until it's done and know it worked out. The expensive part (besides the room) is buying all the wedges and I was lucky to grab as many as I could fit in a van from a place that was closing down.
 
It's a strange opinion since the links between direct sound and perception are well-grounded.

Maybe you missed my point. I am not doubting the link between on-axis response and perceived tonality (rather the opposite), but I pointed out how easy it is to manipulate on-axis response with DSP. So in most of cases (there are exceptions like with combfiltering effects), you can achieve a flat on-axis FR without the speaker in question having such response.

but I think the most reliable way to measure under 100Hz is correctly executed ground plane measurements.

Groundplane in an indefinite half-space relative to the wavelength and with all sound sources being sufficiently close to the plane, yes. Not really a practical solution unless you have a parking lot with a whole to half-bury a speaker.

The research for this dates back to Dr. Toole and his team at NRC. No commercial intent was involved in that, nor was a company.

Certainly true. If I understood Dr. Toole´s answer to my question correctly, this independent NRC research was concluded somewhen around 1986. So while his findings and preference results are in general still valid today, we cannot answer questions which are only possible to research with today's technology. May it be loudspeaker design, measurements to describe their properties, recording techniques, room acoustics - these factors have a huge impact on the result of preference tests.
 
...Groundplane in an indefinite half-space relative to the wavelength and with all sound sources being sufficiently close to the plane, yes. Not really a practical solution unless you have a parking lot with a whole to half-bury a speaker.
You don't really need a hole, but you do need a large concrete driveway, patio, tennis court, etc. But that seems much more practical than building or renting an anechoic chamber or renting or buying a Klippel NFS.

Of course most people don't NEED to do this, as the farfield-nearfield merge process works quite well in typical situations.
 
Have never met an acoustician following such concept in a room dedicated to reproduction. Usually diffusors on the side walls are the first measures in a small room, as discrete early reflections from the sides are most likely to cause deterioration of localization and give our brain the image of a small room. Front wall treatment is in many cases obsolete, and ceiling and back wall might be necessary if their typical flaws were identified in listening tests.



I think we agree on this point. The interesting question is: Why are people in the audio world relying decreasingly on measurements, as they are getting better and better, becoming more publicly available (Kudos to Amir, Erin and JA!) and being prominently promoted by sites like ASR? It is kind of an irony.


I don't get this guy. Claims not to be in the industry, but hangs with acousticians, has done thousands of speaker measurements, and speaks for 'people in the audio world'.
 
I agree with you here - and I believe @Curvature has also agreed with you in the sense that there's no reason for you personally to seek other options or engage in certain test procedures if you are already content. While I am very much a believer in measurements and 100% agree with Curvature's (and many other members') point that our uncontrolled listening impressions can't tell us much of anything reliable about the linearity of a given speaker and how it compares with other speakers, I also have increasingly come to appreciate a pragmatic approach (and here I think I'm agreeing with some points @Heinrich is making, above).

I've done my due diligence with measurements and have speakers that I know are solid performers and whose limitations are generally irrelevant to my listening levels and expectations. The room imposes certain issues, most of which are relatively minor and/or addressable with DSP, and one or two of which are more challenging to deal with, but which I don't mind and/or don't really notice in day-to-day listening. There's no point in making myself unhappy by trying to more rigorously test my system's flaws and limitations under artificial listening conditions. If and when I want to seriously consider changing speakers, or if I were to move to a different listening space, then a rigorous investigation of these room issues and/or current-speaker limitations would be in order.

So in this respect I agree with you 100%.

But this cuts both ways, and that's where I think @amirm is correct in pushing back against some of your comments earlier in this thread. You've every right to your preferences and enjoyment - but that does not mean that the research that showed strong correlation between objective speaker performance and listener preference in controlled blind listening conditions is not valid. It does not mean that research conducted under the aegis of a nonprofit research organization whose results are now, at a later date, used for marketing purposes, can be equated with tests specifically designed by a vendor for the purposes of creating a result that fits with their existing design and marketing objectives. That's just you trying to hand-wave away the real research in order to justify your own choice and preference of speaker.

To be clear, you do not have to justify your speaker choice to anyone with research or anything else - you like what you like, as we all do. But by the same token, the fact that you like what you like does not mean that the research is magically invalid or that it did not produce sound evidence for its findings.

Excellent post!

Whenever I bring up the relevance of measurements and blind testing in the other audio forums I tried to also emphasize that, IMO, no audiophile needs to engage with measurements or blind testing if they don’t care to practice the hobby that way. We can all do this however we want.

But as the same goes “People are entitled to their own opinions, but they aren’t entitled to their own facts.”

It is worth countering erroneous or misleading public claims about “ what measurements can’t tell us” or anti-blind testing or
“ only your ears are trustworthy”…. And inserting real information about those things into the audiophile world.

I think the delicate balance between those two viewpoints his to not become too much of an obnoxious White Knight - trying to save others from themselves. (And I don’t always avoid that charge).

That’s why I think ASR is such a valuable place - it’s not invading other audiophile spaces, but has created a space where audiophiles can come if they want to, to learn more about the relevance of measurements in a rigorous approach to understanding the performance of audio gear.

As to your mention of pragmatism:

I have adopted a somewhat similar pragmatism, and it plays out in my choices this way:

When I’m shopping for new speakers and I’ve auditioned loudspeakers that I know measure very well, in terms of what would be highly rated under blind conditions (eg Revel, KEF, Kii Audio), and I’ve also auditioned loudspeakers that measure fairly well, though not as textbook, and I find myself preferring, the latter speakers over the former, I have a decision to make: do I go with the speakers that aren’t producing a really compelling experience when I listen under sighted conditions, knowing that I might select them as superior under blind conditions?

Or do I go with the loudspeaker that I find myself much more struck by, under sighted conditions?

My pragmatism amounts to recognizing I will be listening to the speakers in sighted conditions at home. And I combine that with these facts: I have never once in all my years budged from the perception “ this speaker doesn’t do it for me” to “ wow I was really wrong about that speaker now I think it’s great.”

Likewise, I have virtually never had the experience of auditioning a speaker, and my opinion of that speaker significantly changing once I had it in my room or owned it. In other words, everything I seem to really like about a loudspeaker when I audition it, shows up and remains consistent once I have it at home. (with one very minor exception I can remember). And my perception of the characteristics of a loudspeaker do not change over time. Among many examples: decade after owning my Thiel speakers I would not describe them any differently as when I first got them.

What this means for me is that my listening perceptions under sighted conditions - whether they are a mix of accurately hearing the sound and bias effects - seem to be very persistent.

Therefore, I would consider it too much of a role of the dice to depend on that pattern changing, by purchasing a loudspeaker just because I know it measures better even though I haven’t found it compelling enough in auditions.

Whereas for somebody else, purchasing the loudspeaker they know to measure best would work out perfectly fine. So it makes sense for others to take that route.

It’s like my friend who reviews speakers, I always ask him about a new speaker “ could you live with this?” And he usually says “sure!” Whereas I am so picky about what moves me, I would select almost none of the speakers he has reviewed, to own. In the context of all the speakers, I have heard an extremely rare for me to think “ I would like to own this and listen to it every day.”

So ultimately, it’s a type of pragmatism that reduces to “Know Thyself.

Works for me :-) (and I think ultimately is at the bottom of what works for everybody)
 
Found it, at about 27:30 he's talking about this new (?) magnet structure he adopted for Mofi's (and older TAD;s perhaps, that's unclear)

I watched him talk about designing that driver. Not a word about computer simulation in advance. Indeed, he says he had to build a prototype, not knowing at all if it would work or not. But got lucky that it did. A person schooled in the mathematics and physics of the topic we are discussing, would be simulating the design and optimizing it in that domain before anything physical is built.
 
Status
Not open for further replies.
Back
Top Bottom