I cannot trust the Harman speaker preference score

[tuga]

Almost everyone i know has an Apple phone with Dolby Atmos capable Apple earphones or headphones with headtracking.

Ha, but do they know it?

 

[tuga]

I don’t know a single person other than me in my life that has a stereo system.

Statistics back up my statements too. Headphones are not stereo either no matter how delusional you are.

This discussion is "academic" because most people don't care about sound quality. Some buy "better" headphones because it's cool and makes them look special. For them, noise-cancelling is far more important than sound quality or some whacky special spatial effect they've never heard about.
People who care about sound quality are know as audiophiles. And they're an infinitesimally-small minority.

 

[Bjorn]

Yes it did. It did have a preferred position which was outside the room unplugged

Good to know.
Does Harman leave the absorbents on the rear wall in the picture in their reasearchers? They look to be 2" thick? And the half circular ones are perhaps BAD Arcs?

 

[abdo123]

Ha, but do they know it?

What do you mean?

 

[tuga]

What do you mean?

I mean that it is pointless if the user does not know of its existence or if he doesn't makes use of the technology.
Maybe he does use it.

I still use and old iPhone for music because recent models don't have a minijack socket...

 

[Holmz]

…
As far as how the speaker couples into the room modes the optimal position of the loudspeaker will depend on the acoustical properties and dimensions of the room and the locations of the listener(s). To say that the optimal position is different for every speaker is nonsense, especially for conventional speakers where below the room transition frequency the speaker is close to a monopole and will couple into the room modes much the same way.

As an aside if there were different optimal locations for different speakers I’ve never seen this specified in the loudspeaker setup manual. And wouldn’t they be different for different rooms? And there would no need for room correction and calibration.
…

I think some speakers without rear ports can be shoved closer to the front wall?
So might there be a bit of squishiness in that aspect??

(Which sort of gets important if there are occupants in the house that want the speaker in a place that is not musically ideal… ergo WAF.)

 

[Digby]

The "Olive score" is based on frequency response. Frequency response is the single most important predictor of sound quality. Distortion is only important if its well above audible threshold. Compression is essentially distortion.

In most of our listening tests distortion was not a factor with the home speakers we tested in our listening room at the average level ( 80 dB-weighted SPL). Even with the small 5-6 inch bookshelf speakers, distortion was not a factor.

To what extent is the score based on frequency response? Does it take other factors into account (I've been trying to find the paper, could someone please link it).

If the score is heavily weighted in favour of frequency response, maybe this would explain the OP saying they can't trust the score regarding preference. They give the example of the following speakers and their scores:

The KEF Reference 2C Meta: 5.6
Sonos Roam: 5.5
JBL M2: 5.1

In reality, these are very different sounding speakers and would not be preferred anywhere near as closely as the score suggests. A speaker with the ability to play dynamic peaks at reasonably loud volumes and with bass response to 30hz (JBL M2) is going to be heavily preferred (sonically, if not aesthetically) to a Sonos Roam, yet the score suggests they are practically on par with each other.

I purchased Genelec 8030c speakers based on the high score they received in ASR testing, yet I found the speakers lacking compared to the Behringer B2031A speakers I already had. The B2031As produce most of the usable frequency range, down to 35hz or so, and have a greater sense of control at anything above moderate levels than the 8030c, yet I doubt they would score close to as highly as the 8030c.

However, I imagine if you placed people in front of these two speakers (sans subs) and asked them to choose which they prefer, perhaps 7 out of 10 would prefer the Behringer, because the missing bass frequencies are evident to the extent where any improvements offered by the Genelec (flatter FR) are not as significant.

Is there a component of testing that penalises speakers for not reproducing frequencies down to, say, 40hz.

If not, this would explain why small, but SPL & bass extension limited, bookshelf types speakers that have a flat FR (namely Genelec) score highly, when larger speakers that are more capable in both SPL and bass response score rather more poorly.

In testing, I can believe that a flat FR is preferable, all else being equal, but when SPL and bass response capabilities of a speaker are rather limited (most bookshelf speakers), then I imagine listeners will accept more variation in the FR (to an extent) in exchange for bass extension and SPL.

Is this something the score can account for? Is it something it should account for, in your opinion.

 

[abdo123]

To what extent is the score based on frequency response? Does it take other factors into account (I've been trying to find the paper, could someone please link it).

If the score is heavily weighted in favour of frequency response, maybe this would explain the OP saying they can't trust the score regarding preference. They give the example of the following speakers and their scores:

The KEF Reference 2C Meta: 5.6
Sonos Roam: 5.5
JBL M2: 5.1

In reality, these are very different sounding speakers and would not be preferred anywhere near as closely as the score suggests. A speaker with the ability to play dynamic peaks at reasonably loud volumes and with bass response to 30hz (JBL M2) is going to be heavily preferred (sonically, if not aesthetically) to a Sonos Roam, yet the score suggests they are practically on par with each other.

I purchased Genelec 8030c speakers based on the high score they received in ASR testing, yet I found the speakers lacking compared to the Behringer B2031A speakers I already had. The B2031As produce most of the usable frequency range, down to 35hz or so, and have a greater sense of control at anything above moderate levels than the 8030c, yet I doubt they would score close to as highly as the 8030c.

However, I imagine if you placed people in front of these two speakers (sans subs) and asked them to choose which they prefer, perhaps 7 out of 10 would prefer the Behringer, because the missing bass frequencies are evident to the extent where any improvements offered by the Genelec (flatter FR) are not as significant.

Is there a component of testing that penalises speakers for not reproducing frequencies down to, say, 40hz.

If not, this would explain why small, but SPL & bass extension limited, bookshelf types speakers that have a flat FR (namely Genelec) score highly, when larger speakers that are more capable in both SPL and bass response score rather more poorly.

In testing, I can believe that a flat FR is preferable, all else being equal, but when SPL and bass response capabilities of a speaker are rather limited (most bookshelf speakers), then I imagine listeners will accept more variation in the FR (to an extent) in exchange for bass extension and SPL.

Is this something the score can account for? Is it something it should account for, in your opinion.

Your worries and woes have all been thoroughly answered in the thread if you’re interested.

 

[sarumbear]

Loudspeaker technology and the science behind it is pretty mature. I am quoting papers that are 37 years old that are still valid today. The science has been peer reviewed and the results replicated in other labs of universities and other loudspeaker manufacturers. It’s no longer a controversial or disputed topic within the industry. If you think it’s controversial you are not well-informed.

The loudspeaker industry has generally accepted the science of what makes a loudspeaker sound good; there are new standards that define what is good and how to measure it, and it’s widely practiced within the industry

If you go to an ASA or AES conference there are almost no papers on what makes a loudspeaker sound good. Most of the attention is to make loudspeakers sound good in smaller form factors, cheaper, play louder, compensate for room modes, or do beam steering arrays to deal with room acoustics or simulate virtual speakers and spaces.

AR, VR, mixed reality, and immersive audio is the focus for applications in the home, the car and mobile.

Thank you for your reply but you have failed to answer my question. One wonders why you will avoid it...

As others said many times, thank you for your research work throughout the years and the breadth of knowledge you have added to science.

 

[Digby]

Your worries and woes have all been thoroughly answered in the thread if you’re interested.

Can you link the post numbers please? Apologies for repeating what has been said already, you know how it goes with these long threads.

 

[sarumbear]

I wonder how many non-audiophile people have heard of Dolby Atmos. Or cared.

You are obviously not one of the 100+ million Apple Music users. I doubt if many audiophiles have heard it but Apple Music users have, although they heard it called Spatial Audio.

 

[abdo123]

Can you link the post numbers please? Apologies for repeating what has been said already, you know how it goes with these long threads.

You can see my opinion on that particular comparision here

 

[Digby]

OK, but are you arguing that certain types of testing are invalid. Do you think the numbers produced would be significantly different if the testing was made uniform for those three speakers?

Either way, I'm not sure that post alone comprehensively answering my "worries and woes", as you put it (nice turn of phrase, btw )

 

[fineMen]

In these tests Robert and Sam heard the same four loudspeakers that have been evaluated previously by hundreds of untrained listeners including young, old, American, Asian, and European listeners, whose preferences and performances were compared to those of our panel of trained listeners. From these tests, we have found evidence that most listeners prefer the most accurate, neutral loudspeaker regardless of age, culture or listening experience.

from this blog by Olive.

Behind Harman's Testing Lab

This past week I had an enjoyable time meeting well-known technology writer Robert Scoble who was visiting our Harman facilities in Northri...

seanolive.blogspot.com

Preference means when given four speakers to choose from they rank them by which they consider best sounding to worst sounding. Seems simple enough. And they've found MOST listeners prefer accurate and neutral.

Thank You a lot!

My original question is left open, though. But I understand more, why it is so.

The original question was, if people were asked for their motivation. I do not insinuate this possibility to be realised, but it is a possibility still:

"People, driven by the test regime (lots of effort, at big Harman company, already best hifi heard in a lifetime, higher levels than used to, big science with doctors, sterile environment ...), do not choose the most pleasing speaker, but the most accirate one, as far as they think it is."

Think of the over-weight on bass in the metrics, 'preference' for heavy bass (pun not intended). Common people listen most often to severely bass restricted (a) content over likewise bass restricted (b) speakers, due to cost, footprint, neighbors, and at bass restricting (c) lower levels.

Then they come to the test and of course value that speaker more, that delivers what is neither had nor actually required at home. Only given this possibility alone, all the argumentation of Olive is (logical) tautology,

Not the least, reiterated, bass capabilities and distortion free sound power capabilities correlate, because both qualities are regularly connected to bigger speakers. This correlation isn't resolved in the query.

O/k, I don't feel that we get somewhere.

Just one thing for finish:

Looking into the list linked in post #1, nearly all specimen perform close to perfect once equalised and accompanied by a sub. Someone with the capability to e/q his speaker in-situ, and add in case a bass would be happy to dismiss the score alltogether.

 

[Bjorn]

Not many home speakers are designed for corner placement or wall placement. If they are that’s how we would test them. This might include soundbars and inwall speakers.
We built a listening room with an inwall speaker mover for that purpose.

Powered Pro monitors often include switches that provide different equalizations for different boundary conditions. We would test those using the appropriate setting for the setup.

I can count on one hand the number of dipole and cardioid speakers Ive tested in my career.

While being designed for corner placement or flushed against wall is rare, there's still a difference in room gain in regards to distance to walls. One could of course equalize the speakers to have the same low frequency level to remove this difference but especially smaller woofers would have an increased distortion and effect the test, and you're not testing the sold speaker design anymore. If you on the other hand listen with some speakers closer to the front wall and further from the listener, you increase the room influence and you're not comparing apple to apple anymore.

While I'm don't believe this influences the researchers to a major degree, it does say something about the challenge of such studies. Especially if the speakers score quite equal in other areas.

 

[abdo123]

OK, but are you arguing that certain types of testing are invalid. Do you think the numbers produced would be significantly different if the testing was made uniform for those three speakers?

Either way, I'm not sure that post alone comprehensively answering my "worries and woes", as you put it (nice turn of phrase, btw )

Oh for sure i expect the Sonos Roam to be at least 2 points lower at least. The KEF would be at least one point lower.

 

[Digby]

Oh for sure i expect the Sonos Roam to be at least 2 points lower at least. The KEF would be at least one point lower.

Ok, but say we compare the Neumann KH80 DSP to the JBL M2 (both graded from spin data, correct?), do we really expect a speaker that produces the full audio spectrum, sans sub frequencies (M2), to be topped in preference by a speaker that is incomplete, in that it does not produce the full frequency range, and will only produce sound to moderate levels at best? It seems unlikely.

They are different speakers for different uses, but I assume the preference rating is just that, in that in only measures what people prefer to hear (not what they can fit in there room/intend to use/whatever).

I have some doubts that the typical listener will prefer the KH80, sans subwoofer, compared to the M2. Probably there are a number of such comparisons that can be made.

 

[abdo123]

Ok, but say we compare the Neumann KH80 DSP to the JBL M2 (both graded from spin data, correct?), do we really expect a speaker that produces the full audio spectrum, sans sub frequencies (M2), to be topped in preference by a speaker that is incomplete, in that it does not produce the full frequency range, and will only produce sound to moderate levels at best? It seems unlikely.

I honestly think behind a black curtain with both speakers running within their linear excursion limits people will prefer the KH80 more in a statistically significant way.

The whole idea behind the model is that we put our biases aside, the opinion that people will prefer the JBL M2 better simply because it's bigger and more expensive even though it measures worse is the reason why the model exists.

To ignore it when there is a clear score difference of over one point between two speakers is to surrender to bias in my opinion.

 

[tuga]

While I'm don't believe this influences the researchers to a major degree, it does say something about the challenge of such studies.

This is a good point, and one that I have made before.

Harman has produced good research over the years, and whatever quibbles I have with the methodology used in some of the studies I can understand that simplification was necessary for practical reasons.
But there's no escaping the fact that oversimplification will produce distorted view.

 

[Digby]

Within linear excursion limits is tipping the scales heavily in favour of the KH80 (or any other small speaker). It is obviously easier to provide a flat FR, if you are not stretching the limits of a speaker.

The M2 may measure "worse" in that the FR is not as flat, but it covers the almost entire audio range, so in that sense it measures better. Significant frequencies are present in music that are not heard with the KH80, surely this counts for something. Wouldn't a speaker be expected to be docked marks for not reproducing sound down to around 40hz at sufficient levels.

How flat does FR need be and to what extent does a flat FR negate a lack of bass response?

It doesn't seem to me preference can just be down to a flat FR. If something else is lacking, then what is lacking may change perceived preferences.