Random musings on The Preference Ratings of speaker reviews

[Thomas savage]

I think the "problem" is even worse than it is for the SINAD chart. With the SINAD chart, the worst that happens is people overpay for inaudible differences in SINAD.

With the Preference score, people can mechanically buy speakers that score high, but are not suited to their use cases (e.g. bass / volume too small for room / listening distance).

I just don't see the point in using a metric that is as likely to be misleading as informative when it comes to purchase decisions.

It's never struck me as being for consumers , it's some internal metric for Harman to qualify performance of various speakers.

I think us using it has a degree of misappropriation about it .

 

[Blumlein 88]

Thanks, I didn't know! I believe dr. Toole said that they were "essentially omnidirectional" or something like that, so that's what I assumed. Very interesting.

They had identical drivers on the front and rear for tweeter and midrange and a single woofer on the M3si. The M1si had front and rear woofers. They also had assymetrical crossover slopes which is what lead to the lobing you see in the graphs off axis. I don't know, but would have thought these might give spin results that were mediocre, but maybe not.

They also were a bit too fat and under-damped in the low end. You could fix that by filling the ports with small diameter straws which dampened them down nicely.

 

[Blumlein 88]

It's never struck me as being for consumers , it's some internal metric for Harman to qualify performance of various speakers.

I think us using it has a degree of misappropriation about it .

Come now Thomas. Is it the new avatar? Are you trying to keep the holy Harman texts from the masses among the faithful?

 

[tuga]

It's never struck me as being for consumers , it's some internal metric for Harman to qualify performance of various speakers.

I think us using it has a degree of misappropriation about it .

Perhaps it's a tool to manipulate consumer preference.

 

[watchnerd]

It's never struck me as being for consumers , it's some internal metric for Harman to qualify performance of various speakers.

I think us using it has a degree of misappropriation about it .

That was my thought, too.

Internally, it would be one metric among many to evaluate design criteria.

Externally, it would be like buying cars based solely on their cornering skidpad G's.

Or Top Gear lap times.

 

[RayDunzl]

Except for the Magnepan LRS, I don't think I've paid much attention to the "ratings".

Look at the pictures, consider the listening impression, wait for some opportunity to interject someplace in the discussion...

That's the routine.

But the love/hate panel, with a preference rating of 1.07 (was that real - seems not to be listed in the database today), well, that made me look.

 

[Blumlein 88]

Except for the Magnepan LRS, I don't think I've paid much attention to the "ratings".

Look at the pictures, consider the listening impression, wait for some opportunity to interject someplace in the discussion...

That's the routine.

But the love/hate panel, with a preference rating of 1.07 (was that real - seems not to be listed in the database today), well, that made me look.

Good news is when they break out a chart of panel speakers, yours aren't the lowest scoring panels. In fact it scored 400% more than the next highest scoring panel (that we know of, I don't think they ever listed the score for the Quad publicly).

 

[MZKM]

But the love/hate panel, with a preference rating of 1.07 (was that real - seems not to be listed in the database today), well, that made me look.

The one the database is the one with the correct on-axis, which actually reduced the score. Keep in mind that it got a 5 for the score w/sub, which puts it ahead of the Edifier recently measured for instance (at a 4.5).

And again, I put in bold text in my post that dipoles weren’t considered when making the scoring formula, similar with omni speakers like those from MBL. This note is also stated in the database as well in my shared file for the LRS (which contains all my graphs and SPL specs).

 

[watchnerd]

Good news is when they break out a chart of panel speakers, yours aren't the lowest scoring panels. In fact it scored 400% more than the next highest scoring panel (that we know of, I don't think they ever listed the score for the Quad publicly).

Quads with subs preference score over / under?

 

[Blumlein 88]

Quads with subs preference score over / under?

Quad 57's...............................1.5 and 6 w/subs.
Quad ESL63's.........................2.5 and 7 w/subs.

Surely someone in the great Northwest can get a pair to Amir to actually test. Someone within driving distance. Maybe Amir needs to put out a craigslist ad. Speakers wanted for testing.

 

[weasels]

Here you go.

Sweet, that's even better. Thanks!

 

[watchnerd]

Quad 57's...............................1.5 and 6 w/subs.
Quad ESL63's.........................2.5 and 7 w/subs.

Surely someone in the great Northwest can get a pair to Amir to actually test. Someone within driving distance. Maybe Amir needs to put out a craigslist ad. Speakers wanted for testing.

Would they fit in the Klippel?

 

[Blumlein 88]

Would they fit in the Klippel?

I'm not sure, but from some comments from Amir in the LRS review, I think they would. Any panels much taller aren't going to however.

Also the Revel 208 he has reviewed is 46 inches tall. The Quad not on a stand and Quad ESL63 or 2805 are both shorter than that.

 

[Colonel7]

It's never struck me as being for consumers , it's some internal metric for Harman to qualify performance of various speakers.

I think us using it has a degree of misappropriation about it .

Agreed it's not for consumer decision. It's in the public domain although with a patent. Having put together scoring systems in the past, I figured there was a souped up or customized version that is proprietary. It's a win-win as a consultant and for Harman

 

[Senior NEET Engineer]

A 4" or 5" wooger isn't going to get the job done, let's be honest. Not on the desktop, not on the floor, not anywhere.

Sure, what those tiny little drivers can do is amazing, but even a 6.5" will blow them into the weeds. 2x 6.5" in a ported cabinet gives you a semblance of bass. Then, we start to get partially serious with an 8" , 10" or 12" bass driver.

By the time you are using a 15" in a three way, you'll just look at those toy speakers and giggle...

What would the in-room 100hz SPL and THD performance of a serious speaker be?

 

[tuga]

The CEA2034 spin graphs seem quite good at pointing out a good speaker from a poor one. Yet sometimes a great looking spin graph gets a lower Preference score than I expect. Something just isn't right about that preference formula though.

Before we focus on the formula, a spin describes the dispersion characteristics of a given speaker, its tonal balance on- and off-axis.
Is directivy enough to characterise speaker performance?
Is frequency response enough to characterise speaker performance?

As for the formula, its effectiveness has not yet been put to test by ASR.
Unfortunately Amir has limited time and resources and his listening tests are manifestly unfit for purpose.

 

[Blumlein 88]

Before we focus on the formula, a spin describes the dispersion characteristics of a given speaker, its tonal balance on- and off-axis.
Is directivy enough to characterise speaker performance?
Is frequency response enough to characterise speaker performance?

As for the formula, its effectiveness has not yet been put to test by ASR.
Unfortunately Amir has limited time and resources and his listening tests are manifestly unfit for purpose.

Frequency response and directivity seem to go a long way to characterize speaker performance.

It seems on axis frequency response will characterize a speaker's sound quite well unless some off axis problems corrupt it.

Of course beyond that you need to know about loudness capabilities for the space you intend to use it in. And distortion should be low enough though such seems rarely a problem.

I would like to see FR presented with ERB based smoothing. And think the formula likely would work better if it did the same.

It also occurs to me that perhaps the formula as used in Preference Predicted rating here might need another column. One where speakers with response below 100 hz get a rating that ignores everything below 100 hz. Harman claimed near perfect correlation with test results on bookshelf speakers that didn't go below 100 hz. So how about it @MZKM or @edechamps would this make any sense to add? Or would limiting it to 300 hz make more sense for consistency? Basically would allow us to compare speakers without the low end which appeared in their results to give near perfect correlation. Below 300 hz is where the room means we never get the graph anyway.

 

[MZKM]

Frequency response and directivity seem to go a long way to characterize speaker performance.

It seems on axis frequency response will characterize a speaker's sound quite well unless some off axis problems corrupt it.

Of course beyond that you need to know about loudness capabilities for the space you intend to use it in. And distortion should be low enough though such seems rarely a problem.

I would like to see FR presented with ERB based smoothing. And think the formula likely would work better if it did the same.

It also occurs to me that perhaps the formula as used in Preference Predicted rating here might need another column. One where speakers with response below 100 hz get a rating that ignores everything below 100 hz. Harman claimed near perfect correlation with test results on bookshelf speakers that didn't go below 100 hz. So how about it @MZKM or @edechamps would this make any sense to add? Or would limiting it to 300 hz make more sense for consistency? Basically would allow us to compare speakers without the low end which appeared in their results to give near perfect correlation. Below 300 hz is where the room means we never get the graph anyway.

The only time performance <95Hz is considered is for the LFX, which gets fixed in place for the w/ sub score.

NBD uses 1/2 octave bands starting at 100Hz, and I am using 14, one could delete the bottom 3 bands (~ 95Hz-270). Smoothness starts at 100Hz, but one could also delete the lower end of responses.

However, usually there aren’t many issues <300Hz, most of the issues are >500Hz dealing with baffle step, crossover implementation, directivity matching, etc.

 

[tuga]

Frequency response and directivity seem to go a long way to characterize speaker performance.

It seems on axis frequency response will characterize a speaker's sound quite well unless some off axis problems corrupt it.

Of course beyond that you need to know about loudness capabilities for the space you intend to use it in. And distortion should be low enough though such seems rarely a problem.

And because FR and directivity plots do not provide information regarding distortions they are mainly informing about tonal balance and predicted in-room response above the transition range if position is close to untreated side walls.
So I disagree with your assessment that they go a long way.

Harman claimed near perfect correlation with test results on bookshelf speakers that didn't go below 100 hz.

I have reservations regarding Harman's findings because of the realtively small sample, use of untrained listeners and for in my view not using the most effective methodology for assessing preference.

And for some reason you've started this topic because like me you believe that "Something just isn't right about that preference formula".

 

[weasels]

What's the concern about the use of untrained listeners? If the intent is to develop a model that is appealing to consumers, why would it be preferable to use trained listeners?