What role should listening play in speaker design

[amirm]

We all have been spoiled by Dr. Toole and Dr. Olive, because I recall reading somewhere that as part of their employment agreement with Harman, they were allowed to publish their scientific research. Not all companies are willing to publish research they have, as a matter of business priorities.

The rule by far is that they can't and won't publish them! But I don't think that is the reason behind AJ's statements.

 

[Audionaut]

But when I did this eyes closed comparison with a vocal track Jeff was playing on his flagship Pearls I was just astonished at how they nailed the timber and tonality of the human voice… there was very little difference between the voices coming from the speakers and some voices occasionally chatting in the same room. The timber and color, and the natural warmth “matched” which to me is very rare.

What I perceive from the Joseph speakers is a purity of tone… an effortless relaxed clarity that comes from a completely etch or grain-free presentation. Yet also with some warmth and richness (not thin or anaemic sounding).

The effect on the sound of instruments and voices is like looking through a clear stream from which all sand or silt had been removed revealing all the vivid colours of the rocks and pebbles. In the same way it felt like the distinct variety and beauty of instrumental timbre and tones came through in the way I had rarely heard before.

Matt, I think you'd be the perfect salesperson at an audiophile hi-fi store.

"The effect on the sound of instruments and voices is like looking through a clear stream from which all sand or silt had been removed revealing all the vivid colours of the rocks and pebbles."

This is pure hi-fi poetry for subjective audiophiles with deep pockets

 

[amirm]

"Two ears and a brain respond very differently to a complex sound field — and are much more analytical — than an omni-directional mic and analyzer. "

That's with respect to perception of sound above transition frequencies. The two ears hear differently with brain analyzing and combining them. As such, a single microphone can't capture that. This has nothing to do with wanting to have a speaker with proper measurements. Indeed, this is the sum total of decades of research by Dr. Toole and his team. So let's not be confused about that.

 

[Sokel]

"Two ears and a brain respond very differently to a complex sound field — and are much more analytical — than an omni-directional mic and analyzer. "

That goes for home users roughly doing REW, Dirac and stuff and try to "correct" (his quotes) speakers+room in an amateur fashion , that's Dr's intention.
Speaker building is another subject.

 

[RexrothPigeon]

Measurements taken lightly may not account for that that's why companies has to state the intended use, as Genelec does for example.

Do the measurements taken by Amir account for that? Erin? It certainly isn't an unmeasurable phenomenon

 

[Sokel]

Do the measurements taken by Amir account for that? Erin? It certainly isn't an unmeasurable phenomenon

The predicted in-room response is a good index but does not account any distance and condition.
That's up to the company to define and up to the user to follow.

 

[Audionaut]

I think you can add a ton of high-end speakers to the list. Wilson would be one that comes to mind right away.

We’re mainly discussing the following question here:
To what extent do manufacturers who rely on subjective judgment use measurements?

Although the more interesting question that arises from this is actually:
To what extent do manufacturers who rely on objective measurements also use their ears?

So how do AsciLab, Kii, Revel, Neumann, and Genelec do it? Do they just measure, or do they listen as well?

If they also listen, why?

 

[SigbergAudio]

Perhaps @Thomas Lund can chime in for Genelec?

 

[TimVG]

Some thoughts:

In order to design a speaker without listening, we need targets of course. So here are some, perhaps incoherent, thoughts.

In terms of amplitude response the original studies dictate a flat direct sound, and smooth off-axis performance.
For those looking more closely, the actual preferred slopes (regression) can also be deduced. So we have a very good idea of how a loudspeaker needs to behave in terms of frequency response.

A couple of potential discussion points:

-The original tests were performed mainly using standard forward firing cone/dome loudspeakers. A couple of loudspeakers using a horn were also present, and one dipole. We know the latter had a severely compromised frequency response.

-From blind tests we know while the better measuring horn loudspeakers could still win blind tests compared to loudspeakers with flawed frequency response/directivity issues, they tend to lose compared to equally good forward firing cone/dome loudspeakers as far as I have seen (there was the blind Salon2 vs M2 test on AVSforum some years back).

-Recently we've seen different designs, such as cardioid patterned speakers, become more prominently available. I've been looking at more studies involving half-space designs (in-wall) as there is correlation there, although also not entirely of course. Less energy vs no energy towards the back (cardioid vs in-wall).

Although I cannot find with absolute certainty what is the deciding factor in preference when it comes to off-axis sound. Is it the smoothness of the curve, or is it the regression slope?

Using Gemini: "The Preference Intervention: In actual practice, the "target" for in-wall speakers is simply linearity. The research (Welti/Olive) suggests that for in-walls, listeners prefer the smoothness of the slope over the steepness of the slope."

But is this the case? I haven't been able to track any specific studies with more technical information and certain -quite credible- reports, have noted that non-standard cone/dome loudspeakers have a tendency to not sound entirely balanced at times due to the nature of the slopes, while smooth, being shallower - which is then countered by manual intervention (tilting).

And then there are constant directivity loudspeakers, which present a unique challenge because their ideal anechoic performance contradicts the ideal statistical target for a standard room..

/rant

 

[goat76]

I find it interesting that Buchardt Audio did go for the subjective approach when finalizing the sound characteristics for their E50 loudspeaker, even though they had been measurement-driven before releasing that speaker model. They seem to have taken that same approach yet again with the upcoming S400 Mk3. The interesting part is that they used a group of listeners in a blind test, and they got to listen between a flatter frequency response tuning and a more subjective tuning, and most of the people in the listening group preferred the latter, so they decided to go with that.

The question is, why did this group of people not prefer the objectively better-measuring tuning, as would be the obvious choice for most people according to Dr. Toole’s research?

 

[TimVG]

I find it interesting that Buchardt Audio did go for the subjective approach when finalizing the sound characteristics for their E50 loudspeaker, even though they had been measurement-driven before releasing that speaker model. They seem to have taken that same approach yet again with the upcoming S400 Mk3. The interesting part is that they used a group of listeners in a blind test, and they got to listen between a flatter frequency response tuning and a more subjective tuning, and most of the people in the listening group preferred the latter, so they decided to go with that.

The question is, why did this group of people not prefer the objectively better-measuring tuning, as would be the obvious choice for most people according to Dr. Toole’s research?

The big issue with these sort of tests are that they are (I assume) not well documented, and therefore anecdotal. Without knowing the full details of this test and the DUT we don't really know anything.

 

[goat76]

The big issue with these sort of tests are that they are (I assume) not well documented, and therefore anecdotal. Without knowing the full details of this test and the DUT we don't really know anything.

Yes, you are right. Maybe @Mads Buchardt can give us more details on how these listening tests were done?

 

[mtmpenn]

Here is my major question on this topic:

- do we really know the optimal target frequency response for all circumstances?

While the Harman/Toole/Olive data are compelling, i would argue 1) a couple hundred subjects is not sufficient to provide “the truth” in most fields of science and 2) it does not seem likely, to me, that it could provide a definitive answer that spans the entire range of loudspeaker design and use.

The obvious issue here would be designing a speaker for near wall placement vs placement out in the room. But differences in dispersion characteristics, baffle size/shape, driver dispersion characteristics, room width and wall treatment, etc, could all impact observed sound in the listening position and therefore “ideal” frequency response.

And while it might be argued to design for optimal placement in an optimal room and assume dsp by the end user, I think it’s totally reasonable to design for real rooms or real world placement or unique driver characteristics.

I suspect that the solution in most cases would be mostly flat on axis response, but I can imagine minor deviations being “preferred” in some circumstances.

 

[Newman]

I believe Andrew Jones is in fact in that age range, but definitely correct me if I'm wrong!

He started his undergraduate BSc in 1974 which suggests a DOB of 1956 or earlier, so 70-ish seems just about right.

cheers

 

[RDoc]

Here is my major question on this topic:

- do we really know the optimal target frequency response for all circumstances?

<snip>

Well, no we don't, furthermore I'll go out on a limb and say we never will, actually I'd say it's impossible. "All circumstances" includes an truly infinite number of cases. For example "all circumstances" includes diving in an underwater cave under 5 atmospheres of water pressure. That wasn't tested, but personally I don't feel it's a significant missing data point.

The point to the Harman/Toole/Olive research is to get a handle on what people like and it's far better than anything else we have to go on. Certainly not the final word, but a pretty good snake oil solvent.

 

[mtmpenn]

Well, no we don't, furthermore I'll go out on a limb and say we never will, actually I'd say it's impossible. "All circumstances" includes an truly infinite number of cases. For example "all circumstances" includes diving in an underwater cave under 5 atmospheres of water pressure. That wasn't tested, but personally I don't feel it's a significant missing data point.

The point to the Harman/Toole/Olive research is to get a handle on what people like and it's far better than anything else we have to go on. Certainly not the final word, but a pretty good snake oil solvent.

I think we largely agree, but your example is a bit extreme. There are more common circumstances than an underwater cave that could, reasonably be expected to impact the ideal response - like planned for near wall placement, narrow room vs wide room, placement on a desk, near vs far field listening, acoustically treated vs untreated room, or unique characteristics of drivers (for example some tweeters have a drop in radiation width at the highest frequencies).

I also think there is a difference between a broken design that has obvious flaws and a design that deviates intentionally from the Harman standard in a reasoned way. Accepting this requires a level of nuance beyond reading a preference score.

 

[PristineSound]

I never heard him say that. I heard him say that it is difficult to trust the measurements, not because there's something inherently wrong with measurements, but because it's hard to take accurate measurements and there is no reference to use to check if you did it right.

If we for the sake of discussion assume the Klippel is perfect, that still doesn't make it easy, as that tool costs 100,000USD.

Short summarize of what Andrew Jones said:

• Dr. Toole's work is based on preference listening to commercial recordings. As such who knows what was done to those recordings, so it's not accuracy to the source. "You don't know what source is supposed to be."
• Measurement is hard to be accurate. He believes 99% of people with a "microphone and computer" don't get that right. Anechoic chamber isn't accurate for bass, outdoor measurement for bass is hard, need to float the speaker, wind noise, temperature, etc. Ground plane has issues with cabinet diffractions. Klippel NFS, 200Hz to 800 Hz aren't accurate with "wiggles." Because of this inaccurate measurement: "I'm hearing something, but I can't trust my hearing. I've got to trust the curves. And that's what kills me. 'Just trust the curves.' No, I'm not going to trust your curves."
• "It's in the imperfections that define the sound character of the speaker, and we can't claim that a speaker has no sound character. Every speaker has a character. So, it's those inaccuracies, those imperfections, that define what that character is. And so it's my personal choice as to which ones I try—I want to try and fix, and which ones I don't or cannot afford. I can't afford to fix all of them, or else it would be a $20,000 speaker, not a $2,000 speaker. . ." So what he decides to fix and not fix ultimately reflects on his personality what his own preference is within the "within a broad outline of the general balance of the speaker but you never meet that perfection."
• He then says: "I think it's still good to make, to kind of fix every imperfection you see, if only because it'll make it work better with every recording you're going to listen to."

 

[SigbergAudio]

Short summarize of what Andrew Jones said:

• Dr. Toole's work is based on preference listening to commercial recordings. As such who knows what was done to those recordings, so it's not accuracy to the source. "You don't know what source is supposed to be."
• Measurement is hard to be accurate. He believes 99% of people with a "microphone and computer" don't get that right. Anechoic chamber isn't accurate for bass, outdoor measurement for bass is hard, need to float the speaker, wind noise, temperature, etc. Ground plane has issues with cabinet diffractions. Klippel NFS, 200Hz to 800 Hz aren't accurate with "wiggles." Because of this inaccurate measurement: "I'm hearing something, but I can't trust my hearing. I've got to trust the curves. And that's what kills me. 'Just trust the curves.' No, I'm not going to trust your curves."
• "It's in the imperfections that define the sound character of the speaker, and we can't claim that a speaker has no sound character. Every speaker has a character. So, it's those inaccuracies, those imperfections, that define what that character is. And so it's my personal choice as to which ones I try—I want to try and fix, and which ones I don't or cannot afford. I can't afford to fix all of them, or else it would be a $20,000 speaker, not a $2,000 speaker. . ." So what he decides to fix and not fix ultimately reflects on his personality what his own preference is within the "within a broad outline of the general balance of the speaker but you never meet that perfection."
• He then says: "I think it's still good to make, to kind of fix every imperfection you see, if only because it'll make it work better with every recording you're going to listen to."

Without remembering everything he said verbatim, that looks like a fair enough representation. He also mentioned that whenever he changes anything based on listening, he goes back to measure to verify/document the effect of the change.

He also said something to the effect that you can keep changing stuff until you go crazy, and that a broad change like for instance adjusting the treble 0.5dB up or down is clearly audible, but it can be hard to know what is "correct", and may sound better or worse depending on the track.

 

[eddantes]

TLDR - Once measurement confirms you are in the ballpark - the remainder is meaningless because nearly all people will not be able to meaningfully discern the difference. So AJ is free to add any seasoning he likes and ensure that those that like his brand will buy his brand with the confidence that they are getting something special. ASR obsessives will however continue to argue over largely meaningless though measurable differences. Did I get this right? :-D

 

[PristineSound]

Without remembering everything he said verbatim, that looks like a fair enough representation. He also mentioned that whenever he changes anything based on listening, he goes back to measure to verify/document the effect of the change.

He also said something to the effect that you can keep changing stuff until you go crazy, and that a broad change like for instance adjusting the treble 0.5dB up or down is clearly audible, but it can be hard to know what is "correct", and may sound better or worse depending on the track.

My only issue I have is that:

1. "You don't know what source is supposed to be." that's nonsense. The source is the source, whether the source was compromised or not, the source is the starting point and is the truth in sound reproduction. If you are going to question the source material and then try to compensate for what you believe is right, that is by definition the circle of confusion. Which is why every control room should have a linear in room response such that creating the source material is mixed and mastered in one standard and the playback is also in the same standard. So this is absolutely nonsense what AJ is saying.
2. He can't trust the measurements because of the inherited flaws due to the complexity, then why not just do a better job with the measurements? If it's costs, that's fair. Unfortunately, starting a new business is not cheap and for those companies who invested upfront, they get to reap the benefits.