Master Preference Ratings for Loudspeakers

[MZKM]

I would clarify it further:

1.
View attachment 48445
'On-Axis w/o subwoofer'
or
'On-Axis without subwoofer'

2.
View attachment 48444
'On-Axis w/ subwoofer'

3.
View attachment 48446
Good.

That may be confusing, some people thinking it is only looking at the on-axis response.

 

[sweetchaos]

"Kali In-8, which scores a 6.7 with the On-Axis curve vs a 7.4 using the listening window."

So the new calculation shows that Kali is rated better when using the listening window (aka off-axis), right?
Does that mean that Kali should be placed off-axis for best results?

Im just trying to understand this new listening window graph and when it's applicable.

 

[MZKM]

"Kali In-8, which scores a 6.7 with the On-Axis curve vs a 7.4 using the listening window."

So the new calculation shows that Kali is rated better when using the listening window (aka off-axis), right?
Does that mean that Kali should be placed off-axis for best results?

Im just trying to understand this new listening window graph and when it's applicable.

In this case, it just means they focused on improving the listening window response by sacrificing the on-axis response.
Intended for no toe-in usually has excess treble on-axis, that way the treble is neutral off-axis.

The IN-8 at 20° H is more neutral, but treble is recessed.

 

[lonewolf]

I posted the below on the JBL 305P thread, but didn't get a response so figured I'd post it over here:

Looking at the preference rating MZKM has calculated, the 305P gets a 4.36 and the Control 1 gets a 2.7 for overall sound quality. Looking at amirms measurements, the 305P has much better bass response. Since 30% of the rating score is for bass response, it makes sense the 305P will get a much better score.

If we look at the scores ignoring bass response, the 305P gets 6.43 and the Control 1 gets a 6.2. I assume that is a "close" score. Looking at the measurements (I'm no expert on spins but am starting to get a handle on them. Amirm, best explanation on how to read spins that I've seen yet, thanks!) I wouldn't expect the Control 1 to be anywhere near "close" to the 305P for sound quality. If I were shopping for speakers, I would immediately disqualify the Control 1 based on the measurements and the 305P might make it on my short list. What am I missing?

 

[MZKM]

I posted the below on the JBL 305P thread, but didn't get a response so figured I'd post it over here:

Looking at the preference rating MZKM has calculated, the 305P gets a 4.36 and the Control 1 gets a 2.7 for overall sound quality. Looking at amirms measurements, the 305P has much better bass response. Since 30% of the rating score is for bass response, it makes sense the 305P will get a much better score.

If we look at the scores ignoring bass response, the 305P gets 6.43 and the Control 1 gets a 6.2. I assume that is a "close" score. Looking at the measurements (I'm no expert on spins but am starting to get a handle on them. Amirm, best explanation on how to read spins that I've seen yet, thanks!) I wouldn't expect the Control 1 to be anywhere near "close" to the 305P for sound quality. If I were shopping for speakers, I would immediately disqualify the Control 1 based on the measurements and the 305P might make it on my short list. What am I missing?

Looking at the PIR graphs, the 305 is roughly more neutral, but it's deviates frequently (a rocky road), whereas the Control 1 is more of a smooth wave, not as neutral but more smooth.

 

[lonewolf]

Looking at the PIR graphs, the 305 is roughly more neutral, but it's deviates frequently (a rocky road), whereas the Control 1 is more of a smooth wave, not as neutral but more smooth.

Thanks for the reply! I guess I'm still trying to wrap my head around these graphs. The 305P has a couple sharp points, but other than that looks pretty smooth and consistent to me overall. And should respond well to EQ. The Control 1 looks like a roller coaster ride, and not friendly to EQ in the 4-10K range. If the preference favours more neutral speakers (which I believe it does), I would have though the 305P would be considered more neutral overall, getting a better score than the Control 1. That's what I got out of it when I saw the graphs incorrectly so it seems.

Sigh, back to the drawing board I guess!

 

[Jmudrick]

In this case, it just means they focused on improving the listening window response by sacrificing the on-axis response.
Intended for no toe-in usually has excess treble on-axis, that way the treble is neutral off-axis.

The IN-8 at 20° H is more neutral, but treble is recessed.

A reminder there are dip switches. I employ the the treble boost listening off axis 2.75m.

 

[edechamps]

It's worth considering the context of the Olive paper, which had different intentions than ours. The goal was not to find which speakers were the best, it was to correlate measurements with preference. It's a subtle but important distinction; Olive did not make it a priority to optimize a particular speaker's performance, he makes no consideration of how people would actually listen to and position the speakers, nor what the intended horizontal listening axis is. His concern was only how captured data could be correlated with preferences. This is understandable, given his goal was to provide a better preference metric than the popular Consumer Reports one (at the time). On-axis was presumably chosen as the direct sound axis because it is easier to set up a turntable for a variety of speakers this way than finding the optimal reference axis for each speaker.

Thank you for writing that post, it's quite the amazing insight you got there.

I agree that using NBD_LW instead of NBD_ON could give interesting results, and could certainly provide a more realistic assessment of speakers that show problems in the response directly in front of the tweeter, such as coaxials (IN-8, LS50). I have two reservations, however.

First, in the absence of any other information, most people will assume they will get the best sound by pointing the speaker directly at them. They might not do that all the time (that's often an unrealistic use case), but they might do that when they want to do some critical listening. In such cases, they are in fact getting the on-axis response.

Second, the problem with the listening window is that's it's an average. You can't actually get that response no matter what angle you listen at, because it doesn't correspond to any particular angle. For this reason the score might not match what a typical listener would rate the speaker, even while listening slightly off-axis - it is likely an overestimate from excessive smoothing. One could imagine pathological cases where a speaker has a huge frequency response abnormality at some angles, and the opposite abnormality at other angles, which would average out to "perfect". Though maybe I'm being overparanoid here.

I can't help but think that these problems would be avoided if manufacturers carefully specified the optimal listening axis. In the case of a coaxial, for example, the manufacturer could indicate in the manual that the best sound is obtained by listening slightly off-axis (e.g. "no toe-in"). Then if @amirm measures the speaker using such an offset reference axis, the results will be good even with the original formula.

Oh, and let's not forget @MZKM's "it won't end there" prediction - this is on track to prove him right. At some point we're gonna have to agree upon a small set of dimensions by which to break down this data, otherwise Sheets will explode under the sheer number of tabs

 

[napilopez]

Thank you for writing that post, it's quite the amazing insight you got there.

I agree that using NBD_LW instead of NBD_ON could give interesting results, and could certainly provide a more realistic assessment of speakers that show problems in the response directly in front of the tweeter, such as coaxials (IN-8, LS50). I have two reservations, however.

First, in the absence of any other information, most people will assume they will get the best sound by pointing the speaker directly at them. They might not do that all the time (that's often an unrealistic use case), but they might do that when they want to do some critical listening. In such cases, they are in fact getting the on-axis response.

Second, the problem with the listening window is that's it's an average. You can't actually get that response no matter what angle you listen at, because it doesn't correspond to any particular angle. For this reason the score might not match what a typical listener would rate the speaker, even while listening slightly off-axis - it is likely an overestimate from excessive smoothing. One could imagine pathological cases where a speaker has a huge frequency response abnormality at some angles, and the opposite abnormality at other angles, which would average out to "perfect". Though maybe I'm being overparanoid here.

I can't help but think that these problems would be avoided if manufacturers carefully specified the optimal listening axis. In the case of a coaxial, for example, the manufacturer could indicate in the manual that the best sound is obtained by listening slightly off-axis (e.g. "no toe-in"). Then if @amirm measures the speaker using such an offset reference axis, the results will be good even with the original formula.

Oh, and let's not forget @MZKM's "it won't end there" prediction - this is on track to prove him right. At some point we're gonna have to agree upon a small set of dimensions by which to break down this data, otherwise Sheets will explode under the sheer number of tabs

I largely agree, and I've seen at least a couple of cases of interesting averaging. The Focal Chora is a pretty good example of it actually, at least in my sample. None of the curves are all that clean, but the listening window averages out to be almost perfectly linear above 2Khz.

That's why i suggested showing both, but yes there does need to be a limit. In this case I think it's useful to know both the direct sound and listening window as a guideline for speaker characteristics and whether or not you should be listening on-axis. I do disagree with the point about most people listening on axis for critical listening. As shown in the Devantier image, the distribution is such that most listening is off-axis. And if we want to limit it specifically to enthusiasts, I think modern audiophiles know that you have to mess around with toe-in to get the best sound. I don't know that anynone listens on axis without some experimentation unless it's a stuido monitor that specifies it.

I do wonder to what degree the balancing of sound on speakers that are finnicky within the listening window affects perception. After all, our bodies aren't a single point in space either. Were these some of the outliers in the Olive study? We may never know =]

 

[edechamps]

Thanks for the reply! I guess I'm still trying to wrap my head around these graphs. The 305P has a couple sharp points, but other than that looks pretty smooth and consistent to me overall. And should respond well to EQ. The Control 1 looks like a roller coaster ride, and not friendly to EQ in the 4-10K range. If the preference favours more neutral speakers (which I believe it does), I would have though the 305P would be considered more neutral overall, getting a better score than the Control 1. That's what I got out of it when I saw the graphs incorrectly so it seems.

I'll admit I feel the same way - I'm puzzled as to why the formula concludes the Control 1 is pretty much as good as the 305P (with LFX ignored of course), while anyone eyeballing the graphs would most likely disagree. The breakdown view makes it look like they're mostly equivalent in every aspect (SM_PIR, NBD_ON, NBD_PIR), which is even more eyebrow-raising.

@MZKM is there a way to access the underlying spreadsheets so that we can dig into the underlying calculations? I'd certainly be interested to see the individual components of the NBD and SM sums and see where the 305P loses ground to the Control 1.

 

[MZKM]

I'll admit I feel the same way - I'm puzzled as to why the formula concludes the Control 1 is pretty much as good as the 305P (with LFX ignored of course), while anyone eyeballing the graphs would most likely disagree. The breakdown view makes it look like they're mostly equivalent in every aspect (SM_PIR, NBD_ON, NBD_PIR), which is even more eyebrow-raising.

@MZKM is there a way to access the underlying spreadsheets so that we can dig into the underlying calculations? I'd certainly be interested to see the individual components of the NBD and SM sums and see where the 305P loses ground to the Control 1.

I have the radar chart for each in their respective links.
If you want the raw spreadsheets for both, I can link them when I get back to my computer.

 

[hardisj]

@MZKM first, thank you for putting these sheets together. I was looking at Olive’s research about a year ago and and was planning to incorporate his predictive formula in to my own tests as a means to help objectively determine a "value". It’s great you’ve used his works in your own efforts because if/when I do the same we will have even more to compare against.

Also, using your list here, I created a price vs performance graphic as a means to objectively determine the "value" of the speakers. I don’t necessarily know if I feel comfortable with such a “black and white” (blue and green, ha!) analysis but it’s interesting nonetheless and I thought I’d share the results. If you find it useful, it might be worth incorporating in to your plots. Just a simple $/performance calc.


- Erin

 

[lonewolf]

I'll admit I feel the same way - I'm puzzled as to why the formula concludes the Control 1 is pretty much as good as the 305P (with LFX ignored of course), while anyone eyeballing the graphs would most likely disagree. The breakdown view makes it look like they're mostly equivalent in every aspect (SM_PIR, NBD_ON, NBD_PIR), which is even more eyebrow-raising.

I'm glad to hear I'm not the only one! I thought I was getting a handle on the spin format, but now it seems maybe I'm not lol. Since the preference score is based off the spin measurements, it seemed that either there was an error in the calculations or I have a lot more to understand. I really need to get Tooles book and study it.

But damn I don't see it!

 

[dorirod]

Small note, but I would only list either price/speaker or price/pair even if you have to mulitply or divide the available listing.

Thanks for all the great work

 

[MZKM]

Small note, but I would only list either price/speaker or price/pair even if you have to mulitply or divide the available listing.

Thanks for all the great work

I list it as a pair unless it's typically bought as a single, meaning a center channel. All the charts that incorporate pricing are using pair pricing though.

 

[edechamps]

I list it as a pair unless it's typically bought as a single, meaning a center channel. All the charts that incorporate pricing are using pair pricing though.

It might be less confusing to just use the price for a single speaker everywhere. This way there would be no need to special-case center speakers.

 

[MZKM]

I'm glad to hear I'm not the only one! I thought I was getting a handle on the spin format, but now it seems maybe I'm not lol. Since the preference score is based off the spin measurements, it seemed that either there was an error in the calculations or I have a lot more to understand.

Open the charts for both, highlight the on-axis for the on-axis in the spin data (double click on the on-axis label in legend), the Control 1 has a dip at 8kHZ and a boost >10kHz, but it's mostly flat, the 305 is more consistent, but it has a rise in response, 10kHz is like 4dB louder than 600Hz, so even though it doesn't have as much of dips/peaks, it's not a neutral response.

 

[MZKM]

It might be less confusing to just use the price for a single speaker everywhere. This way there would be no need to special-case center speakers.

But then you have the KEF LS50 for instance, where you can buy a single but it's not 1/2 the pair price, so that would be confusing if I state it as ~$600/each and if someone goes to buy a single (center channel use for instance), it's actually much more. Pair pricing for everything except center channels makes more sense to me and I believe it would be less confusing (again, price : performance always uses pair pricing).

EDIT: As I'm typing this though, the LS50 (pair and single) are on their biggest sale ever, pair pricing just $100 more than original single price.

 

[lonewolf]

Open the charts for both, highlight the on-axis for the on-axis in the spin data (double click on the on-axis label in legend), the Control 1 has a dip at 8kHZ and a boost >10kHz, but it's mostly flat, the 305 is more consistent, but it has a rise in response, 10kHz is like 4dB louder than 600Hz, so even though it doesn't have as much of dips/peaks, it's not a neutral response.

Thanks! I see what you mean about the overall upward slope and how the 305P would be less neutral because of it as a whole. The Control 1 has a good upward tilt from about 9K up, but I think it was you who said the treble doesn't seem to affect the score much. I believe it was Dr. Toole who said that high-Q peaks and dips aren't terribly noticeable but low Q peaks and dips are. The Control 1 seems to have more low-Q dips that I would have expected it to negatively affect sound quality. And perhaps it does, without those the Control 1 would have received a higher perceived score. Hmmm. And the 305P deviates enough from neutral to keep it from pulling ahead too. What one gains in ground over the other, it loses in another area and ends up being a wash. They will both sound different from each other, but I can kinda see how neither would be necessarily be preferred over the other overall. Gotcha!

And great work on the graphs, they are fantastic!

 

[phoenixdogfan]

Yeah, but they (and earpieces) remain pretty much the only performing part of a hifi (unless you include watching blinking LEDs or wavy lines on a display).





I can't trust anything I say...

Bought a new TV, thought the speakers inside it sounded pretty good until I hooked up the optical output.

The famous chart almost goes negative on the speakers similar to mine:

View attachment 47533

I suppose they didn't permit scores below zero. An average of 0.5 or so is interesting. That's getting down there. How many zeroes were submitted?

I suppose the speakers didn't meet the criteria for which the listeners had been primed to score in the environment in which they were tested.

If your test criteria is for blondness of hair, blueness of eyes, and fairness skin, that will likely eliminate a great number of adequate alternatives.

To this day, I wonder which speaker (including specific model) are represented by the letters P, I, B, M?