• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required as is 20 years of participation in forums (not all true). Come here to have fun, be ready to be teased and not take online life too seriously. We now measure and review equipment for free! Click here for details.

Random musings on The Preference Ratings of speaker reviews

Blumlein 88

Major Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
10,177
Likes
14,040
#1
While looking at the Preference ratings that @MZKM keeps up to date for us I noticed a few things that stuck in my mind.
https://sites.google.com/view/speakerdata/preference-ratings-graphs


DISCLAIMER: yes I know we have discussed the foibles of this rating formula and that it can't be a one stop quality number for choosing speakers. Also all my ratings in this post are your regular anecdotal sighted listening long term ownership ratings of a purely subjective nature.

Since I have a couple JBL LSR305's I noticed the rating for them with a sub is better than any stand alone speaker tested thus far except for the top spot Genelec 8341a. So does this really seem likely? I know why the sub addition boosts the preference rating as the low end counts for 30% of preference as perceived by listeners. Still that is quite a boost. It means for less than $700 I could eclipse all stand alone stereo pairs with one exception of those tested by ASR at this time. Or maybe I need a sub for each channel in which case it goes to $1000. I'm assuming use of the matching JBL LSR310 here. Maybe it requires a more capable sub to get full benefit. Also there are some choices not much more expensive than the 305s which would get you a rating with sub above the Genelecs.

I have some Revel F12's in a video system. They are clearly much preferred for music over the LSR305s. If the 305s warrant an acceptable rating of 65 on a scale of 100 then the F12 should get a 75% rating. I have an LSR310 sub. When paired with the 305s they in some ways begin to approach the F12s, but in other important ways never do. Maybe they get 72% rating. I rather doubt the F12 spin-o-rama would get a preference rating from the formula equal or nearly so of the Genelec 8341a's.

Also I can, and have paired the F12s with the LSR310 sub. It elevates the result of those too. Buy not by as much. Maybe half the gain. Still would I be getting near top of the heap results this way? I don't quite think so. Yet the preference formula would have us believing maybe it could.

And what to make of the recent LRS Maggie with a rating of -.25........?????? I'm one who thinks Amir's listening description is about right and his measurements surely are. Yet a negative .25 seems really down there. What would a 6x9 car speaker in a properly sized box or baffle score? Just for kicks maybe those of you up to date in the car world could suggest a good 6x9 car speaker we could screw to a trapezoidal shaped 1/2 inch plywood open baffle and see if it beats an LRS. Oh, and an LRS jumps to 5.01 with a sub? I expect a big jump with a sub on that speaker. But that seems like something is off in the low end ratings of the formula.

I am not sure what I hope to come from this thread. The Preference formula seems to work okay at 4 and above to me. Below that I'm doubting it considerably.

The CEA2034 spin graphs seem quite good at pointing out a good speaker from a poor one. Yet sometimes a great looking spin graph gets a lower Preference score than I expect. Something just isn't right about that preference formula though.
 
Last edited:

RayDunzl

Major Contributor
Central Scrutinizer
Joined
Mar 9, 2016
Messages
10,426
Likes
9,442
Location
Riverview FL
#2
Something just isn't right about that preference formula though.
"A general model is provided for predicting a loudspeaker preference rating, where the model's predicted loudspeaker preference rating is calculated based upon the sum of a plurality of weighted independent variables that statistically quantify amplitude deviations in a loudspeaker frequency response. The independent variables selected may be independent variables determined as maximizing the ability of a loudspeaker preference variable to predict a loudspeaker preference rating. A multiple regression analysis is performed to determine respective weights for the selected independent variables. The weighted independent variables are arranged into a linear relationship on which the loudspeaker preference variable depends."

I didn't know it was patented...
 

Χ Ξ Σ

Active Member
Joined
Jul 16, 2020
Messages
104
Likes
225
Location
Bay Area
#3
Can you direct me to the page where you saw the LSR305's w/sub score is better than all but the 8341A's? I checked the following two pages and did not see that.
Screen Shot 2020-09-22 at 9.37.44 PM.png

Screen Shot 2020-09-22 at 9.37.32 PM.png
 
OP
Blumlein 88

Blumlein 88

Major Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
10,177
Likes
14,040
Thread Starter #4
Good find Ray. I didn't know it was patented either. It has probably been discussed, but the patent actually has more than one method of correlation. Which is closest to what we are using?
 

restorer-john

Major Contributor
Joined
Mar 1, 2018
Messages
5,234
Likes
11,815
Location
Gold Coast, Queensland, Australia
#5
I am not sure what I hope to come from this thread. The Preference formula seems to work okay at 4 and above to me. Below that I'm doubting it considerably.
Sorry, I don't take the slightest interest in the "preference formula", or anything produced from it. Making any buying decisions based on such scores would be foolhardy and most likely result in disappointment.
 
OP
Blumlein 88

Blumlein 88

Major Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
10,177
Likes
14,040
Thread Starter #6
Can you direct me to the page where you saw the LSR305's w/sub score is better than all but the 8341A's? I checked the following two pages and did not see that.
View attachment 84387
View attachment 84388
Perhaps I was unintentionally misleading.

I'm referring to the first preference list in your post.

If you look under preference ratings you see Genelec 8341a with 6.75.

The 305mkII is at 4.64, but with a sub 6.58 making it just below the Genelec without a sub.

Now of course the Genelec gets a better rating with sub as well. I'm comparing the 305 with sub to Genelec (or other highly rated results) without a sub.
 

Sancus

Addicted to Fun and Learning
Forum Donor
Joined
Nov 30, 2018
Messages
886
Likes
1,906
Location
Canada
#7
Preference rating study didn't include dipoles afaik so the Magnepan pref rating is irrelevant. Can't use a model on qualitatively different things that weren't included in the data at all.

JBL 305p with subs is 6.58 and Genelec 8341A is 6.75. This seems impressive to me -- for the Genelec. It's a tiny thing, after all, and not even close to the top of the line in the range. 'with sub' assumes you have a good sub that is at least as integrated as a built-in woofer in a well-designed speaker would be. That's non-trivial.

This is ignoring all the things that are not included in preference rating at all, like power handling and distortion. We already know the 305p dies at levels the 8341a has no trouble with. And it remains controversial how exactly directivity width applies and whether the way it is included in the model is right or not.

IMO preference rating is good for separating 4s/5s from 7/8s, and that's all it needs to be. If you can narrow down 100 speakers to 10 worth consideration you've already saved yourself tons of time, and that's even assuming you could trial 100 speakers -- effectively impossible given availability and geographical limitations.
 
OP
Blumlein 88

Blumlein 88

Major Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
10,177
Likes
14,040
Thread Starter #8
Sorry, I don't take the slightest interest in the "preference formula", or anything produced from it. Making any buying decisions based on such scores would be foolhardy and most likely result in disappointment.
I do agree with you. We have seen beyond the preference rating and beyond the spin results that things like limited power, or limited bass power corrupt the result based just on testing. These other factors have to be included. For instance some well tested monitors that are small I'd see the spin graph look good, but I know a 4 or 5 inch woofer isn't going to get the job done in a large living room.

However, I might use the spin graphs to include something I'd overlooked in floor standers for a living room. And I'd still need to hear them to get the full measure of them.

And beyond that, we have that list linked in every speaker review on ASR. People are going to look at it and make use of (or misuse) the Preference ratings.
 

RayDunzl

Major Contributor
Central Scrutinizer
Joined
Mar 9, 2016
Messages
10,426
Likes
9,442
Location
Riverview FL
#9
Good find Ray. I didn't know it was patented either. It has probably been discussed, but the patent actually has more than one method of correlation. Which is closest to what we are using?
I have no idea. It's 15 years old, too, so, ???
 
OP
Blumlein 88

Blumlein 88

Major Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
10,177
Likes
14,040
Thread Starter #10
Preference rating study didn't include dipoles afaik so the Magnepan pref rating is irrelevant. Can't use a model on qualitatively different things that weren't included in the data at all.

JBL 305p with subs is 6.58 and Genelec 8341A is 6.75. This seems impressive to me -- for the Genelec. It's a tiny thing, after all, and not even close to the top of the line in the range. 'with sub' assumes you have a good sub that is at least as integrated as a built-in woofer in a well-designed speaker would be. That's non-trivial.

This is ignoring all the things that are not included in preference rating at all, like power handling and distortion. We already know the 305p dies at levels the 8341a has no trouble with. And it remains controversial how exactly directivity width applies and whether the way it is included in the model is right or not.

IMO preference rating is good for separating 4s/5s from 7/8s, and that's all it needs to be. If you can narrow down 100 speakers to 10 worth consideration you've already saved yourself tons of time, and that's even assuming you could trial 100 speakers -- effectively impossible given availability and geographical limitations.
Yes, but if I add a sub to the 305 it can play loud enough, and does it really become as satisfying as the little Genelec alone? Does the sub really turn a mid-4 into a high 6? I think it might but only with plenty of caveats and comparing more apples rather than apples and oranges.
 

waynel

Addicted to Fun and Learning
Forum Donor
Joined
Nov 14, 2019
Messages
576
Likes
621
#11
I believe that the “with a sub“ score assumes a perfect sub with perfect integration. If you could achieve perfect sub integration I’d suspect any decent speaker would do pretty well in a blind test and best most of the best systems that were limited to 60-80Hz low frequency extension. Bass is important!
 

waynel

Addicted to Fun and Learning
Forum Donor
Joined
Nov 14, 2019
Messages
576
Likes
621
#12
Yes, but if I add a sub to the 305 it can play loud enough, and does it really become as satisfying as the little Genelec alone? Does the sub really turn a mid-4 into a high 6? I think it might but only with plenty of caveats and comparing more apples rather than apples and oranges.
Also please remember that the r-squared was only 0.74, that’s good but if means that 74% of the variation in preferences could be explained by the preference score

DB717DBC-BA81-41FA-9819-ABE12332A369.jpeg
 

Sancus

Addicted to Fun and Learning
Forum Donor
Joined
Nov 30, 2018
Messages
886
Likes
1,906
Location
Canada
#13
Yes, but if I add a sub to the 305 it can play loud enough, and does it really become as satisfying as the little Genelec alone? Does the sub really turn a mid-4 into a high 6? I think it might but only with plenty of caveats and comparing more apples rather than apples and oranges.
I think it's an interesting question. I figure you would end up with two systems that would average the same but with different strengths and weaknesses. On bass heavy content, the sub-305p would obviously win. But overall tonality of the 8341A is still going to be quite a bit better.

I do think the only real way to answer these questions is by blind testing though.
 

HooStat

Active Member
Joined
May 11, 2020
Messages
285
Likes
192
Location
Calabasas, CA
#14
I have said this in other threads, but think of the preference score as just another reviewer's opinion. It has a set of criteria, and a set of weights for each of those criteria. The criteria were based on prior research and experience (these are the speaker measurements that are input into the model). The weights were estimated based on the data to tell us how much each measurement should count for.

Because it is a statistical model, it much more reliable (i.e., repeatable) than any human. The model doesn't have bad moods, and it always measures the same things, and uses the same weights. So, it is useful for comparison and consistency. It is definitely not "correct". And it is heavily influenced by how well it predicts at the extremes (high and low preferences). Those observations have a disproportionate weight in the model.

One other thing to note. Notice how there are many more points above the y=x line at the high end? That suggests that the model is UNDER-predicting the actual preference score at the high end. And see the points below the line in the middle? The model is OVER-predicting in some situations. There are 3 speakers that were rated at about a 2, but got scores between about 4 and 5.

The model is not perfect. But it is useful. Think of it as a trusted reviewer. You won't always agree, and you may value some different things, but you know that the reviewer isn't influenced by anything but the data.
 

Chrispy

Addicted to Fun and Learning
Forum Donor
Joined
Feb 7, 2020
Messages
646
Likes
433
Location
PNW
#15
Yep, preference to me is that of an individual for the most part; a formula to predict....meh. Maybe if the title was Predicted Preference score?
 
Last edited:

MZKM

Major Contributor
Forum Donor
Joined
Dec 1, 2018
Messages
2,682
Likes
5,901
Location
Land O’ Lakes, Florida
#16
I believe that the “with a sub“ score assumes a perfect sub with perfect integration. If you could achieve perfect sub integration I’d suspect any decent speaker would do pretty well in a blind test and best most of the best systems that were limited to 60-80Hz low frequency extension. Bass is important!
It also assumes you are listening to content which benefits from the deeper extension. So, if you are listening to acoustic music, it likely won’t change your rating that much compared to listening to the opening title sequence of Edge of Tomorrow.
 

edechamps

Addicted to Fun and Learning
Forum Donor
Joined
Nov 21, 2018
Messages
677
Likes
2,742
Location
London, United Kingdom
#17
Since I have a couple JBL LSR305's I noticed the rating for them with a sub is better than any stand alone speaker tested thus far except for the top spot Genelec 8341a. So does this really seem likely? I know why the sub addition boosts the preference rating as the low end counts for 30% of preference as perceived by listeners.
@MZKM's "with sub" rating overrides the low frequency extension (LFX) part of the score to always correspond to a -6 dB point of 15 Hz. This is likely well outside what most (if not all) of the speakers in the Olive study were capable of - in fact, many subwoofers can't even reach that low. In my opinion, this means an LFX of 15 Hz is a dangerous extrapolation from the Olive model. As @MZKM just mentioned, an additional problem is that, when one goes below 30 Hz or so, there are likely diminishing returns depending on the content one is listening to (many music genres don't have a lot of sub-bass); this makes the extrapolation even more dubious.

Now, the choice of fixed LFX for the "with sub" computation is not really a concern when comparing speakers "with sub" together, because changing the fixed LFX just shifts the scores by a constant amount. For example, if an LFX of 20 Hz is used instead of 15 Hz (as Loudspeaker Explorer does by default when the score calculation is set to "ideal subwoofer" mode), all that does is reduce all scores by 0.6. But of course this shift does matter if you compare "with sub" scores to "without sub" scores, as the latter is not subject to the shift.

As I've argued before, my opinion is that an LFX of 20 Hz is a more reasonable tradeoff especially if one wants to compare "with sub" scores to "without sub" scores in a more meaningful way. With a fixed LFX of 20 Hz, the "with sub" score of the LSR305 becomes 6.0 instead of 6.6, which does make it somewhat less impressive in the "without sub" speaker ranking.
 

restorer-john

Major Contributor
Joined
Mar 1, 2018
Messages
5,234
Likes
11,815
Location
Gold Coast, Queensland, Australia
#18
but I know a 4 or 5 inch woofer isn't going to get the job done in a large living room.
A 4" or 5" wooger isn't going to get the job done, let's be honest. Not on the desktop, not on the floor, not anywhere.

Sure, what those tiny little drivers can do is amazing, but even a 6.5" will blow them into the weeds. 2x 6.5" in a ported cabinet gives you a semblance of bass. Then, we start to get partially serious with an 8" , 10" or 12" bass driver.

By the time you are using a 15" in a three way, you'll just look at those toy speakers and giggle...
 
OP
Blumlein 88

Blumlein 88

Major Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
10,177
Likes
14,040
Thread Starter #19
@MZKM's "with sub" rating overrides the low frequency extension (LFX) part of the score to always correspond to a -6 dB point of 15 Hz. This is likely well outside what most (if not all) of the speakers in the Olive study were capable of - in fact, many subwoofers can't even reach that low. In my opinion, this means an LFX of 15 Hz is a dangerous extrapolation from the Olive model. As @MZKM just mentioned, an additional problem is that, when one goes below 30 Hz or so, there are likely diminishing returns depending on the content one is listening to (many music genres don't have a lot of sub-bass); this makes the extrapolation even more dubious.

Now, the choice of fixed LFX for the "with sub" computation is not really a concern when comparing speakers "with sub" together, because changing the fixed LFX just shifts the scores by a constant amount. For example, if an LFX of 20 Hz is used instead of 15 Hz (as Loudspeaker Explorer does by default when the score calculation is set to "ideal subwoofer" mode), all that does is reduce all scores by 0.6. But of course this shift does matter if you compare "with sub" scores to "without sub" scores, as the latter is not subject to the shift.

As I've argued before, my opinion is that an LFX of 20 Hz is a more reasonable tradeoff especially if one wants to compare "with sub" scores to "without sub" scores in a more meaningful way. With a fixed LFX of 20 Hz, the "with sub" score of the LSR305 becomes 6.0 instead of 6.6, which does make it somewhat less impressive in the "without sub" speaker ranking.
Yes, the 20 hz assumption looks to make more sense.
 
OP
Blumlein 88

Blumlein 88

Major Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
10,177
Likes
14,040
Thread Starter #20
A 4" or 5" wooger isn't going to get the job done, let's be honest. Not on the desktop, not on the floor, not anywhere.

Sure, what those tiny little drivers can do is amazing, but even a 6.5" will blow them into the weeds. 2x 6.5" in a ported cabinet gives you a semblance of bass. Then, we start to get partially serious with an 8" , 10" or 12" bass driver.

By the time you are using a 15" in a three way, you'll just look at those toy speakers and giggle...
Yes, I always found a speaker with less than one 8 incher was not for playing loud/low even in a small room/space.
 
Top Bottom