What role should listening play in speaker design

[Lars Risbo]

so.

Can I ask what technique you use for measuring? the bass response is super clean.

 

[sigbergaudio]

There may be an inherent difficulty with that approach, which seems to be touched on by Harwood's experience, noting in particular the very last sentence:

(long quote)

From: "Some factors in loudspeaker quality", H. D. Harwood, Wireless World, May 1976, pages 45–48, 51–54.

It would seem that such vagaries of human perception can be largely removed by accurate measurements of loudspeaker on-axis and off-axis frequency responses (including distortion), supported by an understanding of the typical variations inherent in domestic listening rooms inhabited by users of those same loudspeakers.

If I understand you correctly, this is a slight strawman, no one is suggesting to only listen, but to listen supported by measurements. So you won't be fooled to believe you hear something else, as you can double check with measurements what is actually happening.

Related to your quote; In a more casual scenario, I've certainly experienced people having a learnt preference from their own system, assuming that is "correct" and then listening to a more correct speaker, perceiving this as "wrong", for instance with regards to bass reproduction.

"The conclusion therefore was that it is essentially the direct sound which determines the sound quality and not the spherical response. The measurement of frequency response at various angles in a free-field room is therefore a much better indication of performance than the spherical response even when listening in the reverberant field, and this has been confirmed by careful listening tests many times since. ... The sound quality of a loudspeaker is determined much more by the direct response at any given angle than by the spherical integrated response, and at any rate for stereophonic purposes there may well be a degree of omnidirectionality beyond which it is inadvisable to go."

Ref my bold sentence above, I find it interesting that what matters in the measurements are confirmed through listening tests, in a discussion where we question if ears have any merit together with measurements in loudspeaker design.

With regards to your point, I am not directly opposed of a flat on-axis response, but I maintain that the in-room result and how it is perceived by the listener is the end goal and what actually matters. So if I need to deviate from a flat on-axis response to get a natural sounding in-room response, I will do that.

Supporting that approach, Toole noted that:

"It is well-documented that the timbre of voices and musical instruments is mainly dependent upon the amplitude-versus-frequency spectrum of the sounds. Other linear and non-linear effects can be influential but, for simplicity, this discussion will concentrate on the dominant factor. The accurate reproduction of timbre, therefore, requires a wideband, flat and smooth amplitude response. Wide bandwidth is necessary to reproduce all of the sound, flat response is required for neutral spatial balance, and a smooth response is a good indicator of the absence of audible resonances."

I am not in general disagreement with Toole here.

"The deviations from a linear, flat amplitude response that can be detected are very small indeed, a few tenths of a dB for low-Q resonances and other wide-bandwidth deviations. The problem in assessing loudspeakers has been that their sound is very much dependent on the listening conditions. Different organisations and different individuals have different views about what constitutes ideal listening conditions."

Here we again get a confirmation that our ears actually do work (to the despair of some users on this forum).

(..)

So, the question seems to be, which hi-fi loudspeaker manufacturer evaluates their loudspeakers in four or five different listening rooms that are representative of the totality of their (discerning) customer base? That seems to be the final frontier, but also an inherently expensive and time consuming one to tackle with any alacrity.

I do.

 

[TimVG]

Can I ask what technique you use for measuring? the bass response is super clean.

Not representative in this case as it was a gated (8ms) measurement as you probably guessed, sorry should have clarified
Normally I would do a merge of nearfield and farfield, but for a sealed 4l enclosure with a 5" driver the behaviour is predictable.

I will be doing measurements again next week of a different system, but for sake of accuracy I will do a nearfield measurement of this one as well.

 

[DanielT]

I read multiple comments about we audio designers tell stories just to please the crowd or simply for commercial gains. I find this depressing. If I wanted to make money then I would not be designing audio gear. My motivation is to create stuff that I would love to have myself but does not yet exist. I can then only hope that others may like it too.

For speakers, I don't see any problem at all with describing them in subjective terms. Speakers sound different after all.

For electronics used in hifi. If we take amplifiers. What I quoted in #404 may seem a bit strange if amplifiers/amplifier modules are designed to only amplify signals without adding anything. But okay, you can say that amplifier X sounds (any subjective description) even when pushed really hard. That's what I mentioned about clipping. An amplifier that clips doesn't sound good vs. one that doesn't then you can say that it sounds warm, open, detailed and so on. Under those conditions.

If you take it to its absolute extreme, you could also say that cables sound different.
If you compare, for example, extremely long, extremely thin speaker cables with extremely thick ones. By extremely, I mean ludicrous extreme.

 

[Thomas_A]

There can be reasons, for commercial designs for example: the less crossover parts you can get away with without obiously degrading the sound the better for cost management.
Or, you're trying out some configurations in which you are less or not experienced, for example you're adding a rear firing ambiance tweeter which affects sound power.



Good catch, since these will not be used nearfield I did opt to balance out the curves somewhat. I have been listening to them in the nearfield for the past week, can't say anything is bothering me. It's still remarkably flat overall.
View attachment 522504

While measuring, could you also do a measurement with room reflections in your intended listening environment? You mentioned they will not be used near-field? Although there will be a lot of disturbing reflections they can give indications of balance. Like below, Gated 70 cm measurement (blue) with speaker at intended position vs. speaker at intended position and measurement at LP (≈3 m) with "no gating".

 

[RickS]

Dropping thread curation back to moderation. While the discussion has improved, do not have time to curate the deep dive many want to have here. Previous thread bans have been rescinded as well.

Will likely start a more focused thread but this one will remain open. For those who celebrate it, hope you have a great holiday weekend!

 

[Jim Taylor]

For speakers, I don't see any problem at all with describing them in subjective terms. Speakers sound different after all.

This has been covered in the pages of ASR for years.

Yes, speakers sound different. And yes, that difference can be described in subjective terms. The problem is transferring the meaning of the subjective terms from one person to another ... especially for finer differences that are NOT grossly obvious.
One person's subjective description might (and usually does) mean something different to a second listener who is not at the same location (same room) at the same time, especially if the signal being played is not at all the same.
Taking into account that the performance in the bass and upper bass can be subject to the whims of preference, subjective values become problematic for two people comparing notes on bass. Even subjective descriptions of the treble can cause arguments among otherwise well-meaning listeners.
Expand that to thousands of people all over the world, and subjective descriptions can become not only useless, but downright misleading.
(A few years ago, two ASR members argued about their impressions of Focal treble. They weren't dogging each other ... they were serious. The impressions, because they were in opposition, were useless for readers.)

In the end, subjective descriptions are meaningful only for the person who generated them and only under the conditions that person had to contend with. Even a couple years later and in another venue, all bets are off for the same person reliably offering the same impression. Every professional speaker designer has to contend with this problem, every single day, and even the best can do so with less-than-ideal effectiveness.

 

[Pearljam5000]

I think someday AI will replace the hearing in the development process

 

[Audionaut]

In the end, subjective descriptions are meaningful only for the person who generated them and only under the conditions that person had to contend with. Even a couple years later and in another venue, all bets are off for the same person reliably offering the same impression. Every professional speaker designer has to contend with this problem, every single day, and even the best can do so with less-than-ideal effectiveness.

I am, among other things, a social scientist, and I have worked with medical professionals on methods of qualitative data collection that are based entirely on subjective experiences. For example, it is necessary to determine how test subjects react physically and psychologically to certain medications. Your point is absolutely valid, but the problem you describe has been known for decades and has been addressed in medicine, particularly in pharmacological research, and, as far as I know, similar principles also apply in acoustic research.

In medical and pharmacological research, for example, researchers deliberately work with subjective sensations such as pain, discomfort, or well-being, even though these are individual and context-dependent. The key is not to attempt to eliminate subjectivity, but to structure it in a way that makes it comparable. This is achieved through standardized control groups whose composition is clearly defined and whose members, in some studies, develop a shared understanding of scales and terms, and whose judgments are collected under controlled conditions, such as through randomization and blinding of the samples. In this way, individual perception is transformed into a data type that becomes statistically significant and scientifically analyzable.

The underlying scientific principle is that while individual judgments are not very reliable on their own, aggregating them under controlled conditions yields stable patterns. Subjectivity is thus operationalized in a way that makes it manageable and reproducible, and this is precisely the crucial step from individual experience to scientifically verifiable and usable knowledge.

From what I have read so far, this principle is also reflected in acoustic research when listening tests are conducted according to established standards and listeners are specifically trained to develop a consistent and comparable vocabulary for the parameters being evaluated, with the testing deliberately conducted in a blind or double-blind manner to minimize expectation effects. As in medicine and pharmacology, the results are then not interpreted as the opinions of individuals, but are statistically analyzed and correlated with measurable parameters.

So, these are entirely subjective impressions, but taken together and with enough training (not required in all studies), they become scientifically reliable and sufficiently verified data.

As for what “sufficiently” means, you’ll have to read the package insert—though that’s probably not necessary for speakers.

 

[sigbergaudio]

@Audionaut Good point.

Sometimes the McNamara Fallacy comes to mind in these discussions as well. That it's difficult to measure and quantify doesn't mean it's of no consequence.

The McNamara Fallacy

Measuring what is easy to measure, ignoring what is hard

sketchplanations.substack.com

 

[DanielT]

This has been covered in the pages of ASR for years.

Yes, speakers sound different. And yes, that difference can be described in subjective terms. The problem is transferring the meaning of the subjective terms from one person to another ... especially for finer differences that are NOT grossly obvious.
One person's subjective description might (and usually does) mean something different to a second listener who is not at the same location (same room) at the same time, especially if the signal being played is not at all the same.
Taking into account that the performance in the bass and upper bass can be subject to the whims of preference, subjective values become problematic for two people comparing notes on bass. Even subjective descriptions of the treble can cause arguments among otherwise well-meaning listeners.
Expand that to thousands of people all over the world, and subjective descriptions can become not only useless, but downright misleading.
(A few years ago, two ASR members argued about their impressions of Focal treble. They weren't dogging each other ... they were serious. The impressions, because they were in opposition, were useless for readers.)

In the end, subjective descriptions are meaningful only for the person who generated them and only under the conditions that person had to contend with. Even a couple years later and in another venue, all bets are off for the same person reliably offering the same impression. Every professional speaker designer has to contend with this problem, every single day, and even the best can do so with less-than-ideal effectiveness.

What you say is correct. What I meant was more that since it is possible to hear differences, it is not "wrong" to claim that you hear one thing or the other. Then whether there is any consensus of what, for example, a big (wide dispersion?) and or detailed, bright/shimmering (elevated tweeter?) sound is, we can discuss.

Maybe this graph as a starting point for such a discussion, about how one experiences music:

 

[MattHooper]

Taking into account that the performance in the bass and upper bass can be subject to the whims of preference, subjective values become problematic for two people comparing notes on bass. Even subjective descriptions of the treble can cause arguments among otherwise well-meaning listeners.

(emphasis mine)

“can” but not always. You do not want to use uncontrolled informal listening if you’re looking for a scientific certainty and reliability.
But “Can” does not rule out the possibility, under informal listening conditions, of convergence of impressions and useful information being passed between listeners.

In the end, subjective descriptions are meaningful only for the person who generated them and only under the conditions that person had to contend with.

This statement is simply too absolute, to the point of being false.

I wouldn’t see any problem if it were put more like: “For those seeking the most reliable information on the performance of audio gear, looking to objective measurement data is going to be more reliable and revealing than exchanging impressions of audio gear in subjective terms.”

And then from the standpoint, an audiophile can say “Subjective descriptions of audio gear are of no interest or use to me, as they do not provide the level of insight and reliability I’m looking for.”

But to say that audiophiles can not have meaningful or useful discussion about audio gear performance through describing impressions derived from informal listening goes way too far. Plenty of audiophiles have converged in their general, subjective impressions and descriptions of gear, and exchanged useful information this way, which has led to satisfying purchases.

And I’ve read plenty of subjective reviews that nailed the specific relevant traits of a loudspeaker, that I myself and other audiophiles perceive and appreciate.

I mentioned before that my impressions of loudspeakers reviewed by a friend, and our subjective descriptions tend to converge very well, and also tend to align to the speaker measurements.

Then there is Erin of Erin’s audio corner, whose subjective impressions and descriptions most often lineup with and describe what you will see in the objective measurements he presents for loudspeakers.
You constantly see comments under his video, thanking him for translating the measurements into subjective language as to “ how it sounds.” People say it helps them understand the relevance of measurements.

If his Klippel measurements don’t tell something about the actual character of a loudspeaker, and wouldn’t translate at all to some of the characteristics other audiophiles can experience if they purchased those speakers, then the measurements would be mostly pointless. But if they’re not pointless and they do describe speaker characteristics likely to be heard by many listeners, including in their home, then Erin’s accurate subjective descriptions about how those measurements “sound” are also passing some useful information as well.

Even a couple years later and in another venue, all bets are off for the same person reliably offering the same impression.

I have had essentially the same impressions and descriptions of loudspeakers that I heard in other locations (eg store) as they sounded in my own home.

Of loudspeakers I still own I would describe them exactly the same as I’ve described them many years ago.

Some years ago, I did a major renovation of my listening room, redesigned with an architect and an acoustician. Did it end up refining the sound of my systems even more? Sure. But the basic character of my loudspeakers remained obviously intact, and I would describe those essential characteristics the same from before and after the renovation.

I sold one of my favourite high-end loudspeakers to a friend. He used them in a totally different room than mine. While I could hear certain differences in the room, the essential sound of those speakers remained just as I experienced them in my home.
(and he felt the same way since he first heard them in my home, fell in love with the sound, which is why he purchased them from me).

Anyway… examples could go on… but these are some reasons why, though I absolutely understand anybody here eschewing informal subjective impressions and descriptions, I cannot agree with the level of skepticism espoused in some of these statements.

As to the role they can play in speaker designs I’ve certainly loved plenty of speakers in which “ finalizing by ear” purportedly played a role. (I’ve also heard speakers that were clearly screwed up and not to my taste through the same process).

As to whether informal subjective dialing by ear can play a role in achieving the highest accuracy, I’m kind of dubious about that, but people have been making some interesting comments about subjective choices in objective trade offs.

Cheers

 

[TimVG]

While measuring, could you also do a measurement with room reflections in your intended listening environment? You mentioned they will not be used near-field? Although there will be a lot of disturbing reflections they can give indications of balance. Like below, Gated 70 cm measurement (blue) with speaker at intended position vs. speaker at intended position and measurement at LP (≈3 m) with "no gating".

View attachment 522515

Can do somewhere late next week.

 

[snapsc]

AJ does a nice job here in the first 13 minutes of sharing his thoughts on why he goes beyond measurements when designing loudspeakers....

And then along those lines I see Mads Buchardt saying that for their new speaker, measurements are not the sole determinant of the final tuning.

New Buchardt S400 MKIII

I wonder if 1st order crossover can include a notch filter. Would need to be a broad one looking at the Satori frequency response. The e50 supposedly used 1st order on the woofer right? It will need baffle step correction and possibly other response shaping components.

www.audiosciencereview.com

At the end of the day, it doesn't seem unreasonable that the speaker has to sound "right/good" to the designer...and that they believe it has a chance of commercial success.

Beyond that, maybe there is a large enough market for people who only want speakers tuned to a "curve" with no further tuning by ear???

 

[Newman]

I am, among other things, a social scientist, and I have worked with medical professionals on methods of qualitative data collection that are based entirely on subjective experiences. For example, it is necessary to determine how test subjects react physically and psychologically to certain medications. Your point is absolutely valid, but the problem you describe has been known for decades and has been addressed in medicine, particularly in pharmacological research, and, as far as I know, similar principles also apply in acoustic research.

In medical and pharmacological research, for example, researchers deliberately work with subjective sensations such as pain, discomfort, or well-being, even though these are individual and context-dependent. The key is not to attempt to eliminate subjectivity, but to structure it in a way that makes it comparable. This is achieved through standardized control groups whose composition is clearly defined and whose members, in some studies, develop a shared understanding of scales and terms, and whose judgments are collected under controlled conditions, such as through randomization and blinding of the samples. In this way, individual perception is transformed into a data type that becomes statistically significant and scientifically analyzable.

The underlying scientific principle is that while individual judgments are not very reliable on their own, aggregating them under controlled conditions yields stable patterns. Subjectivity is thus operationalized in a way that makes it manageable and reproducible, and this is precisely the crucial step from individual experience to scientifically verifiable and usable knowledge.

From what I have read so far, this principle is also reflected in acoustic research when listening tests are conducted according to established standards and listeners are specifically trained to develop a consistent and comparable vocabulary for the parameters being evaluated, with the testing deliberately conducted in a blind or double-blind manner to minimize expectation effects. As in medicine and pharmacology, the results are then not interpreted as the opinions of individuals, but are statistically analyzed and correlated with measurable parameters.

So, these are entirely subjective impressions, but taken together and with enough training (not required in all studies), they become scientifically reliable and sufficiently verified data.

As for what “sufficiently” means, you’ll have to read the package insert—though that’s probably not necessary for speakers.

Thanks for sharing your insights, from another field, of subjective experiences feeding into scientific advancement.

Some of what Jim wrote, however, pertains to the difference between audio experiences gleaned through controlled listening experiments (like the methodologies you outlined for your field), vs the kind of uncontrolled 'sighted listening', 'casual home comparison (even if listening intently)'. The latter type of data deserves a different level of seriousness, IMHO very, very much lower, than the controlled type of data accumulation and rigorous analysis.

The sighted listening effect is so very context-dependent. It can overwhelm. Notions of one's personal invulnerability to it, usually claimed on the basis of wide experience or casual self-training, are frankly overconfident. Even the simplest psychology experiments make this clear, where the same person is introduced into one-to-one conversations with participants, but with a different back-story about his life given to the participants beforehand, show that the back-story simply dominates their assessments of the person.

This is why I would like to see loudspeaker designers, who can be the most amazing of technicians, be a little less sceptical and give a little more credence to whatever modest scientific data and rigour is available on subjective preference of the sound waves themselves, instead of granting way too much trust in their ears when doing uncontrolled sighted listening evaluations during their loudspeaker development.

That is why the title of this thread can be misleading. The "role of listening" can be the primary role, if the listening is controlled for non-sonic variables and subject to the sort of methodology you have outlined. But then there is this other, much more common sort of 'listening' in the hifi world, and if speaker designers are going to adopt that sort of listening, then the role of listening deserves strong mitigation.

cheers

 

[rf1938]

I'm in the movie business. A very common phenomenon about subjectivity can be found in a movie theater - one exhibiting actual film prints, or even by just looking at a video monitor. Watching a movie there are a number of reel changes, and at each change the release print looks slightly or even greatly different than the previous one. Within a minute or so, the viewer becomes accustomed to the new look. Which is correct? The same is true with video monitors. I have carefully calibrated broadcast monitors using expensive technology. Yet I could show a client his edit on a monitor that is slightly magenta, and, with no other reference, he or she probably wouldn't notice. In the video case, I can measure which is correct. More difficult with actual film. How would one go about describing a viewing experience to someone else - if you had liked the magenta tinted version, without knowing, you could be promoting something that is measurably wrong. In both cases, the experience is heavily influenced by the content. Many parallels here with sound in my mind.

 

[CtheArgie]

I am, among other things, a social scientist, and I have worked with medical professionals on methods of qualitative data collection that are based entirely on subjective experiences. For example, it is necessary to determine how test subjects react physically and psychologically to certain medications. Your point is absolutely valid, but the problem you describe has been known for decades and has been addressed in medicine, particularly in pharmacological research, and, as far as I know, similar principles also apply in acoustic research.

In medical and pharmacological research, for example, researchers deliberately work with subjective sensations such as pain, discomfort, or well-being, even though these are individual and context-dependent. The key is not to attempt to eliminate subjectivity, but to structure it in a way that makes it comparable. This is achieved through standardized control groups whose composition is clearly defined and whose members, in some studies, develop a shared understanding of scales and terms, and whose judgments are collected under controlled conditions, such as through randomization and blinding of the samples. In this way, individual perception is transformed into a data type that becomes statistically significant and scientifically analyzable.

The underlying scientific principle is that while individual judgments are not very reliable on their own, aggregating them under controlled conditions yields stable patterns. Subjectivity is thus operationalized in a way that makes it manageable and reproducible, and this is precisely the crucial step from individual experience to scientifically verifiable and usable knowledge.

From what I have read so far, this principle is also reflected in acoustic research when listening tests are conducted according to established standards and listeners are specifically trained to develop a consistent and comparable vocabulary for the parameters being evaluated, with the testing deliberately conducted in a blind or double-blind manner to minimize expectation effects. As in medicine and pharmacology, the results are then not interpreted as the opinions of individuals, but are statistically analyzed and correlated with measurable parameters.

So, these are entirely subjective impressions, but taken together and with enough training (not required in all studies), they become scientifically reliable and sufficiently verified data.

As for what “sufficiently” means, you’ll have to read the package insert—though that’s probably not necessary for speakers.

Fantastic comment! I’ve worked in similar fields to yours and, sadly, I started in the audiophoolery field. It was only when I decided to use the same or similar strict criteria from my professional work that I was able to quickly change my mind. I look at my old days in horror. Now I’m just trying to sell as much of my old equipment as I can and enjoy my simplified and much cheaper equipment.

 

[MattHooper]

Within a minute or so, the viewer becomes accustomed to the new look.

I had a friend who used an old business projector as his first home theatre projector in his basement. I used to go watch movies at his place.

Over time (probably due to heat damage to a polarization filter), a splotch of purple discolouration started in the bottom left corner of the screen image. Overtime it got worse and worse until I would say it was spread into something like a quarter of the image. I couldn’t even see part of the image at times, especially in night scenes.

I finally said to him “ doesn’t that big purple blotch in the corner of the image bother you? Are you going to replace your projector or get it fixed at some point?”

My friend, turned to me confused “ what purple blotch?”

It had occurred so gradually for him he hadn’t noticed! I literally had to go to the screen and pointed out to him. “ oh yeah… now that you point it out…”

Many parallels here with sound in my mind.

Definitely!

Also reminds me of my own work in film Post Production doing sound effects.

The way sound can influence perception of the image or what is happening on screen in very obvious ways, but also in subtle ways.

I can subtly influence the apparent performance of an actor simply by how I edit a door open or close. If I spread out the turn of the door knob from the click of the latch opening it can make it sound like the door is being opened cautiously. If I close it up just a few frames it will sound like a “ normal” door opening and not something anybody would notice. But if I close up the latch and door knob turn then it can add some urgency or even a sense of aggression to the door opening, which can make the actors energy feel different as he enters the room.

Same with door closes. Sometimes I have altered, or have been asked to alter the sound of a door close in order to alter the perception of the performance.

 

[witwald]

At the end of the day, it doesn't seem unreasonable that the speaker has to sound "right/good" to the designer...and that they believe it has a chance of commercial success.

When we hear the term "voicing", I'm wondering whether it really means getting the speaker to reproduce well (naturally) all of the the voices such as Soprano, Mezzo-soprano, Alto, Contralto, Tenor, Baritone, or Bass. Or is it simply the application of parametric equalisation to suit someone's preferences on their set of preferred test track(s) in their preferred listening environment based on a few ad hoc listening sessions?

 

[TimVG]

Can I ask what technique you use for measuring? the bass response is super clean.

Now with nearfield merger ~200Hz - I use @kimmosto's method from the Vituixcad measurement guide for ARTA.