• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Preference Rating and the case for subjective preference

goldark

Active Member
Forum Donor
Joined
Jan 20, 2020
Messages
204
Likes
352
There is no debate - Harman blind studies show that more neutral speakers were preferred over more colored speakers. But the studies didn't control for only frequency response; there were other variables that contributed to the lower ratings of the speakers that already were less neutral. Admittedly, it would be near impossible to test 2 speakers that were identical in everything except frequency response, but that would be the only way to control for other factors.

Sean Olive in his blog says as much:

"The early studies involved comparison of different speakers that varied more than bass and treble balance. Some speakers had resonances that produced serious colorations, distortions, differences in directivity. The headphone study basically takes a flat neutral headphone and asks people to adjust the bass and treble. That's where experience and age seem to take over. The same holds true for loudspeakers when we did a similar experience. "

"Prior to this study, I nor anyone I know had published a study where trained and untrained listeners were given a bass and treble control and asked to adjust to taste. In previous studies, trained and untrained listeners were asked to give preference ratings to speakers that varied in ways other than bass and treble. It seems that given some finite choices people will pick the most neutral speaker or headphone (no resonances), wide bandwidth. However, given some tone controls they will adjust for variations in program and taste. "

Source: http://seanolive.blogspot.com/2015/11/factors-that-influence-listeners.html

Here is the link to the white paper "Listener Preferences for In-Room Loudspeaker and Headphone Target Responses": http://www.aes.org/e-lib/browse.cfm?elib=17042 The abstract itself says the following: "There were significant variations in the preferred bass and treble levels due to differences in individual taste and listener training. "

The conclusion is that under the overarching umbrella of being "neutral enough," listener preferences take over, and from there, there is "significant variation" - this is the definition of subjective preferences.

This is why I believe little stock should be put into the "preference ratings" in these loudspeaker reviews - at least in how it correlates to real-world listener preferences. The preference rating rates how strictly the loudspeaker design adheres to the basic principles of flat frequency response on-axis, smooth and consistent off-axis frequency response and directivity, among other things - not listener preference. The results of the study above show as much.

This may be preaching to the choir for some; most of us understand that the preference rating isn't a literal ranking of worst to best speakers, but for newcomers to the site casually looking at these rankings, they may misinterpret the data and use the preference rating to determine what speakers they would prefer.
 
Last edited:

pozz

Слава Україні
Forum Donor
Editor
Joined
May 21, 2019
Messages
4,036
Likes
6,825
Your reply is a clear parallel to the SINAD discussions we've had before. As then, as now, when making buying decisions and attempting to guage quality don't rely on a single number.
This is why I believe little stock should be put into the "preference ratings" in these loudspeaker reviews - at least in how it correlates to real-world listener preferences. The preference rating rates how strictly the loudspeaker design adheres to the basic principles of flat frequency response on-axis, smooth and consistent off-axis frequency response and directivity, among other things - not listener preference. The results of the study above show as much.
This is stating the case too strongly. In place of tone controls the research used speakers with slopes tilting toward bass or treble, as well as varying levels of evenness. The research also established, for the first time, what basic relationships between the various curves and response behaviour are important.

As for varying preference for treble, mids and bass, the norm we should be pushing for is tone controls and EQ to be part of every signal chain. A speaker built on the basis of preference research will then allow you to control those features as you see fit.

Outside of tone, the discussions had recently about directivity (wide, narrow, constant) and links to preference are another example of differences between speakers that can only be consideted when looking beyond the number.
 

tuga

Major Contributor
Joined
Feb 5, 2020
Messages
3,984
Likes
4,281
Location
Oxford, England
This is why I believe little stock should be put into the "preference ratings" in these loudspeaker reviews - at least in how it correlates to real-world listener preferences. The preference rating rates how strictly the loudspeaker design adheres to the basic principles of flat frequency response on-axis, smooth and consistent off-axis frequency response and directivity, among other things - not listener preference. The results of the study above show as much.

I agree.
I find measurements extremely useful as a starting point for selecting speakers worth listening to.
But rating speakers according to listener preference in my view makes little sense. It's a matter of taste and there is no guarantee that the predominant preference of a given sample under a particular set of conditions will match my own.
The spinorama measurements help me predict the speaker behaviour in a room but each room has its own distinct characteristics, sometimes requiring different dispersion characteristics. The same can be said about music genres; I find that wide dispersion in a small, untreated room is not the best for the reproduction of classical music.
 

tuga

Major Contributor
Joined
Feb 5, 2020
Messages
3,984
Likes
4,281
Location
Oxford, England
the impact of rooms is pretty overblown afaic. I mean it's important, but both our hearing systems and EQ can/will largely "fix" most normal rooms. A well-behaved speaker is the best starting point...which is where measurements come in.

Can you define what you mean by a "well-behaved" speaker? I quite like the expression, a speaker that does what it's told.
 

aarons915

Addicted to Fun and Learning
Forum Donor
Joined
Oct 20, 2019
Messages
685
Likes
1,140
Location
Chicago, IL
The conclusion is that under under the overarching umbrella of being "neutral enough," listener preferences take over, and from there, there is "significant variation" - this is the definition of subjective preferences.

This is why I believe little stock should be put into the "preference ratings" in these loudspeaker reviews - at least in how it correlates to real-world listener preferences. The preference rating rates how strictly the loudspeaker design adheres to the basic principles of flat frequency response on-axis, smooth and consistent off-axis frequency response and directivity, among other things - not listener preference. The results of the study above show as much.

The main preference difference between listeners is how you shape your bass response. Some like a "house curve" while some may keep the bass flat to the midrange, this doesn't change regardless of what speakers you have. Above the bass range, I haven't seen anything that refutes that people prefer a neutral on-axis response with a closely matching and smooth off-axis response.

Olive's formula is really just the most closely matched parameters that were able to predict subjective preference of actual speakers in their blind tests and it's basically just an algorithm reading the Spin and analyzing the data for you. If you know what you're looking for, you should come to the same conclusion when looking at 2 different speakers, assuming they measure different enough. I disagree that the preference rating doesn't correlate to listener preference because that is exactly what Olive's study showed based on real listeners and had a .86 correlation. In the more closely controlled blind tests where all the speakers had similar bass performance they achieved a .995 correlation rating. Keep in mind this is a peer reviewed study that was published over 15 years ago, I'm not aware of anyone challenging the findings.
 

pozz

Слава Україні
Forum Donor
Editor
Joined
May 21, 2019
Messages
4,036
Likes
6,825
These semantic discussions about preference are largely about whether the pref scores represent the entirety of the listeners as a whole or not—they do not. But they are extremely useful in linking certain technical factors together for assessments.

About the influence of rooms: the "uniqueness" of rooms, despite it being so, just like the uniqueness of speaker builds and each person's hearing, can still be subjected to useful study if you approach it with a somewhat statistical mindset and weigh the main influencing factors. There is no need for exact correlation then. Just accuracy and a clear description of goals and so on.
 
OP
G

goldark

Active Member
Forum Donor
Joined
Jan 20, 2020
Messages
204
Likes
352
The main preference difference between listeners is how you shape your bass response. Some like a "house curve" while some may keep the bass flat to the midrange, this doesn't change regardless of what speakers you have. Above the bass range, I haven't seen anything that refutes that people prefer a neutral on-axis response with a closely matching and smooth off-axis response.

That is not true. The paper states quite clearly, "There were significant variations in the preferred bass and treble levels "

Source: http://www.aes.org/e-lib/browse.cfm?elib=17042

The previous studies simply show the preferences of neutral speakers over non-neutral ones. The 2013 studies show that given a neutral speaker and tone controls, listeners tweaked both bass and treble to a sound that they prefer even more than dead-neutral based on preferences, hearing ability, and even the track being played and that there were significant variations of preferences among listeners.
 
Last edited:

Sgt. Ear Ache

Major Contributor
Joined
Jun 18, 2019
Messages
1,894
Likes
4,150
Location
Winnipeg Canada
That is not true. The paper states quite clearly, "There were significant variations in the preferred bass and treble levels "

Source: http://www.aes.org/e-lib/browse.cfm?elib=17042

The previous studies simply show the preferences of neutral speakers over non-neutral ones. The 2013 studies show that given a neutral speaker and tone controls, listeners tweaked both bass and treble to a sound that they prefer even more than dead-neutral based on preferences, hearing ability, and even the track being played and that there were significant variations of preferences among listeners.

That's fine, but I can't think of any rational reason why a person looking for a set of speakers would choose something that's "pre-colored" over something neutral. EQ will almost always need to be applied to a certain degree to suit the room. Also, I'm not sure I value the tests where people tweaked the speakers on their own as much as I do the comparisons of speakers without tone control. I mean in comparisons of different speakers, neutral was most preferred. If something colored was more preferred, it would have been more preferred...right? I almost think adjustment of tone controls actually adds a biasing factor. People think they want a little more treble, so they twist the treble knob a bit and then "oh yeah, yeah that's more like it!"

Really, a lot of the criticism I hear regarding the pref scores ends up sounding mostly like sour grapes from folks who happen to own a speaker that didn't measure well.
 
Last edited:

QMuse

Major Contributor
Joined
Feb 20, 2020
Messages
3,124
Likes
2,785
That's fine, but I can't think of any rational reason why a person looking for a set of speakers would choose something that's "pre-colored" to something neutral. EQ will almost always need to be applied to a certain degree to suit the room. Also, I'm not sure I value the tests where people tweaked the speakers on their own as much as I do the comparisons of speakers without tone control. I mean in comparisons of different speakers, neutral was most preferred. If something colored was more preferred, it would have been more preferred...right? I almost think adjustment of tone controls actually adds a biasing factor. People think they want a little more treble, so they twist the treble knob a bit and then "oh yeah, yeah that's more like it!"

Adjusting the ratio of bass level and treble level toward mids with very broad filters (Q<=1) for a few dB doesn't really make the sound "coloured". It is done during production how producer sees it fits so I see no real harm if you do it yourself to adjust it to your personal taste or to the specific absorbtion characteristic of your room. In order for that to be effective loudspeaker still needs to have flat on and off axis response and smooth directivity.

Studio monitors very often have that feature as well.
 
Last edited:

Sgt. Ear Ache

Major Contributor
Joined
Jun 18, 2019
Messages
1,894
Likes
4,150
Location
Winnipeg Canada
well this discussion is about the notion that the preference ratings for the speakers being tested are flawed because "some people like different subjective things."
 

QMuse

Major Contributor
Joined
Feb 20, 2020
Messages
3,124
Likes
2,785
well this discussion is about the notion that the preference ratings for the speakers being tested are flawed because "some people like different subjective things."

IMO that is nonsense as those 2 things are not contradicting each other. We all have a common preference toward linear on and off-axis speakers with smooth directivity. Speaking of directivity, probably wider directivity is more favourable then narrow one. But all that certainly doesn't mean we all need to leave LF and HF shelving filters at zero position.
 

Sgt. Ear Ache

Major Contributor
Joined
Jun 18, 2019
Messages
1,894
Likes
4,150
Location
Winnipeg Canada
No of course not. The whole idea afaic is to put together a system that is a really good foundation for tuning (via positioning and EQ) to a given space - and it seems to me that the preference ratings do a pretty good job of pointing to speakers that will help in that regard. I used to believe EQ was to be avoided. Over the past few months I've realized it's absolutely necessary (to varying degrees of course)...
 

QMuse

Major Contributor
Joined
Feb 20, 2020
Messages
3,124
Likes
2,785
I used to believe EQ was to be avoided. Over the past few months I've realized it's absolutely necessary (to varying degrees of course)...

Take this graph as a final proof. It shows Revel Performa3 F208 frequency response measured at dealer's room that has been professionaly treated but without EQ. Not really similar to predicted in-room response from spinorama charts, right? ;)

F208.jpg
 

Sgt. Ear Ache

Major Contributor
Joined
Jun 18, 2019
Messages
1,894
Likes
4,150
Location
Winnipeg Canada
well...it might. I don't know how the room was treated. Or how the speakers were positioned. Or how the measurements were taken. But yeah, EQ is pretty much always going to be needed.
 
Top Bottom