• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Was this aimed at ASR?

preload

Major Contributor
Forum Donor
Joined
May 19, 2020
Messages
1,554
Likes
1,701
Location
California
Reading that article, all I can say is an "R-squared" of 86% is pretty much a gold standard.

R was 0.86. R-squared was 0.74. 0.74 is outstanding if you're trying to demonstrate that listener preferences are affected by A, B, and C characteristics. 0.74 is not that good if you want to use A, B, and C to precisely predict listener preferences. See my post above.
 

Racheski

Major Contributor
Forum Donor
Joined
Apr 20, 2020
Messages
1,116
Likes
1,699
Location
Chicago
I'm not sure what all the fuss is about. Jim Austin's opinion piece seems pretty fair and accurate to me.

The first 6 paragraphs praise and acknowledge the research that has come out of the Harman group (i.e. Toole/Olive). It's not until paragraph 7 that Austin raises the point that while Harman's target measurement characteristics tend to be preferred by most listeners, by no means do they apply to all listeners.

I agree with this. For instance, Olive's listener preference formula isn't nearly as good as people seem to think it is. If you take the time to read the original work by Harman, you'll quickly discover a few things.

As an example, folks often quote Harman's model as having "86% correlation" as if this should end the conversation. But what does that really mean? For starters, the so-called 0.86 is the Pearson r, which means R-squared is actually only 74%. In plain language, Olive's objective measurement model only explains 74% of the variation observed in listener preferences. This is the model that incorporates things like smoothness, response across a listening window, and slope. Here's what that looks like graphically below (it's from Olive's convention paper).

For a given predicted "score," based on objective measurements, look at how much variation there is in actual listener preferences! For a "5", the range is 3-7, which is a HUGE variation in listener preference. And BTW, it requires a lot of math to even arrive at that predicted preference rating - it's not something you can just "eyeball" from a series of FR curves. So the truth is that conformity to what we all accept to be important loudspeaker characteristics is "good," but not "great" at predicting how much listeners will like it.

View attachment 73538

Moreover, as some of your know, listener preferences for the amount of bass and treble in reproduced music varies by age, gender, years of listening experience, country of origin, AND musical content!
In a Harman experiment, they gave listeners a pair of headphones, the same pre-selected music tracks, and asked them to adjust bass and treble tone controls to their preference. Here are some representative slides illustrating some of these differences. They're from a Harman slide deck available on listeninc.com.
View attachment 73539

View attachment 73540
View attachment 73541

How can there be a precise "target curve" when people of different age, gender, country, experience, etc. have different FR preferences? There can't. The Harman "target curve" represents the result of all of their listeners combined, BUT it doesn't depict the variation that their listeners exhibited. (A better way to present the target curve would be to include vertical error bars, in my opinion.) People look at a precisely drawn line and assume that it is a precise target. Harman's own research indicates that it is NOT.

Bottom line: Objective measurements are only somewhat predictive of listener preferences, and even within those objective measurements, there is variation in individual preferences that are valid (unless anyone wants to argue that the preferences of a 15-25 year old male with 10 years of experience is the "gold standard," and everyone else's preferences are wrong).
....but you did not address the roast chicken analogy.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,376
Likes
234,555
Location
Seattle Area
I agree with this. For instance, Olive's listener preference formula isn't nearly as good as people seem to think it is. If you take the time to read the original work by Harman, you'll quickly discover a few things.
Jim Austin is not making any references to preference score. He only talks about Dr. Toole and he did not create the preference score.

Jim is basically saying there is no value in the measurements as a whole which is just wrong. He wants a wild west where anything goes so the random opinion of a reviewer can be always correct no matter how wrong it is.
 

North_Sky

Major Contributor
Joined
Oct 31, 2019
Messages
2,741
Likes
1,553
Location
Kha Nada
I've just read it. Looking @ the picture it looks like a nuclear war, within.

720awsipetard_0.jpg
 

Duckeenie

Active Member
Joined
Apr 7, 2018
Messages
103
Likes
112
Objectively, I'd say 'quote the proper phrase', darn it! Bah.
It should be "Hoist with your own Petard"
https://en.wikipedia.org/wiki/Hoist_with_his_own_petard

I just don't get all the angst over subjectivism in audio - everybody's ears, rooms, and musical preferences are different (and changing all the time).
One - or several - subjective reviews tell me almost nothing about whether I'd like a particular product.
Amir's objective, repeatable measurements are about the only clues I have to guide me in choosing what to audition, or take a risk on.
Music reviews are entirely subjective - and there's nothing wrong with that, either.
Both have a place, just keep them there, and be happy!
:cool:

Unlike some people that always feel they have to pick a team, I have no problem with genuine from the heart subjective reviews. The problem however, is that subjective reviews are more often than not subject to abuse. that's what people really have an issue with ... LIES! It's much more difficult to argue with the integrity of proven science.
 

preload

Major Contributor
Forum Donor
Joined
May 19, 2020
Messages
1,554
Likes
1,701
Location
California
Jim Austin is not making any references to preference score. He only talks about Dr. Toole and he did not create the preference score.

You could be right about that Amir, though I have to say, Austin's description of Toole's research sounds an awfully lot like the methodology behind Olive's regression model:

"Drawing on his own research and the research of others, he established a template for what I call the "classical" loudspeaker: flat frequency response; excellent, well-controlled off-axis behavior; nonresonant cabinet, etc. Toole's main technique was to carry out blind listening tests over many years with many subjects and analyze their preferences statistically. He learned that when it comes to loudspeakers, people mostly like the same things."

Jim is basically saying there is no value in the measurements as a whole which is just wrong. He wants a wild west where anything goes so the random opinion of a reviewer can be always correct no matter how wrong it is.

Is that what he's trying to say? I'll have to re-read it.
What I read was that: some loudspeaker designers intentionally deviate from the "classical model" described by the Harman group, doing so can appeal to a minority of listeners, and that measurements alone shouldn't be used to dismiss the validity of the subjective descriptions of the sound or the engineering competence that went into the design.
 

North_Sky

Major Contributor
Joined
Oct 31, 2019
Messages
2,741
Likes
1,553
Location
Kha Nada
It looks like a stupid prophecy of sorts and a ridiculous interpretation of matters.

Before posting my post I wrote two longer comments, I deleted them both. I could have wrote hundreds, thousands, millions, billions of comments. I went for the most simplicity of them all...Nuclear War. Both sides have their pros and cons, both sides complete each other but no one is admitting...they are both extremely stubborn...as if @ war.

So the pic is a fair representation.

My personal opinion: I like peaceful people, clear in their mind, versatile in their judgement, loving the music for the love of the music, beautiful people in their heart and spirit.

Because music matters, sound matters, emotions have no borders.
 

Hiten

Senior Member
Joined
Nov 24, 2019
Messages
367
Likes
479
Location
India
Quote from the article.
It's disheartening, then, when speakers (and other components) that so obviously do not aspire to classical behavior continue to be judged by classical standards. "That speaker doesn't have a flat frequency response!" shouts an all-caps critic on some online forum, about a speaker whose designer never aspired to a flat response. "This speaker has a resonant cabinet!" exclaims another, about a speaker with a cabinet that's tuned to vibrate in a particular way. "That designer is inept!" writes a third, about an engineer who has sold tens of thousands of speakers—perhaps more—and won awards.
If departure from flat frequency response and deliberate resonant cabinet is goal of the designer, it should be mentioned in the respective speaker advertisement/s. I may be wrong but it seems article is to justify high end bad measuring speakers.

I guess it is collective common wisdom to have a flat frequency response. Because not all music is well record. So suppose a highly reputed designer puts in deliberate 'brightness' in the speaker design; how would a well recorded trumpet sound ? (All the hard work of recording and mastering engineer gone). How would a little brightly recorded trumpet sound ? If a particular sound is required one can use equalizers.

In my opinion it is not about how chicken is roasted. It is about if given dish is chicken or not. Hope I have not offended anyone. Just trying to make sense of all this fearless no qualms HiFi scene.
Regards
 

hapnermw

Member
Joined
Oct 9, 2019
Messages
24
Likes
27
To paraphrase Floyd Toole, the art is where audio preference resides. The speaker's purpose is to accurately reproduce the art (which Floyd has devoted his entire career to quantifying).

If adjustment of frequency response is desired (for preference and/or to deal with audio production issues), use a tone control or EQ (which only works properly if the speaker is accurate to being with).

It seems Jim's purpose is to plead with those evangelizing this objective viewpoint to stop 'fat shaming' products that don't align with it and people who don't accept it.

The issue is he implies that there are equally valid viewpoints which, in fact, there are not. All there are, are personal, subjective viewpoints that invariably fail the double blind test.
 

fredoamigo

Addicted to Fun and Learning
Forum Donor
Joined
Mar 11, 2018
Messages
606
Likes
1,003
Location
South East France
in the article the analogy about chicken and the different ways of cooking it is not very relevant.
There may be a single best way to roast a chicken, but I'm glad different chefs use different recipes.


Of course, there are different ways to roast a chicken or cook it, as we all know that each speaker will sound different. But in the kitchen the most important thing is the quality of the product! Because anyone will be able to tell the difference between an industrial chicken and a free-range chicken.
 

KSTR

Major Contributor
Joined
Sep 6, 2018
Messages
2,690
Likes
6,013
Location
Berlin, Germany
I would think non-flat FR is an issue that should be pointed out by manufacturers and reviewers but if it is the only fault of an otherwise well-executed speaker it is not a problem at all, overall low-Q bumps and dips are not a severe design flaw.

Simply EQ it so that it fits your listening environment, your habits... you'll need EQ more or less anyway if you want to deal with room problems and let's not forget the varying tonal balance in different recordings.

The real problem for the customer is that even simple 2- or 3-band EQ (and a balance control) is missing in most of today's preamps / DACs etc and thus speakers and amps have become a very costly and tedious way to experiment with tonal balancing....
 

GelbeMusik

Senior Member
Joined
Apr 19, 2020
Messages
445
Likes
290
" ... admire Toole's work, but I do not admire conformists who insist, often with insufficient self-examination, that …"

and later

"It's especially disheartening when narrow-minded online critics use one aspect of our coverage—our measurements—to attack the other side: our subjective judgments. We're providing a complete picture; the two halves make a whole. You don't get that from our competition."

The classical strawman argument? People who admire a new approach (spinorama, standards) are accused to be conformists. From this 180° turned-around perspective they are said to rely on insufficient self-examination, despite the new scientific backup. The latter is not taken into question with themselves, when they insist on doing it "subjectively", which would necessarily, by definition be devoid of self-examination.

Black rhetoric, not good.
 

Absolute

Major Contributor
Forum Donor
Joined
Feb 5, 2017
Messages
1,084
Likes
2,125
Well, I don't think there's something particularly offensive or silly in that article. It's basically an argument for a philosophy that audio products should be considered art instead of tools. That's something you either resonate with (audiophiles) or don't (objectivists), but nothing inherently wrong with taking one stance or the other.

Personally I'm pure objectivist, so I would like to live in a world where every speaker sounds tonally the same so that I can enjoy my favorite music wherever I go on whatever I play it on - allowing the music itself to be the only art and let the material be the cause of the sound changing.
On a grander scale than myself, there's certainly people out there with various forms of hearing disabilities/specialties that will prefer something completely different than neutral response. People with extreme sensitivity for certain frequencies or tinnitus springs to mind.

Who am I to say they're stupid for wanting something with a specialized frequency response and have no interest in fiddling with EQ? Who are you to say that?

This artificial moral high-ground some people want to establish based on the idea that their particular opinion is of greater worth than others is ridiculous, imo.
 

q3cpma

Major Contributor
Joined
May 22, 2019
Messages
3,060
Likes
4,416
Location
France
To paraphrase Floyd Toole, the art is where audio preference resides. The speaker's purpose is to accurately reproduce the art (which Floyd has devoted his entire career to quantifying).

If adjustment of frequency response is desired (for preference and/or to deal with audio production issues), use a tone control or EQ (which only works properly if the speaker is accurate to being with).

It seems Jim's purpose is to plead with those evangelizing this objective viewpoint to stop 'fat shaming' products that don't align with it and people who don't accept it.

The issue is he implies that there are equally valid viewpoints which, in fact, there are not. All there are, are personal, subjective viewpoints that invariably fail the double blind test.
This, or more simply: speakers are supposed to play music, not make it.
 
OP
Jas0_0

Jas0_0

Active Member
Forum Donor
Joined
Oct 3, 2019
Messages
285
Likes
513
in the article the analogy about chicken and the different ways of cooking it is not very relevant.


Of course, there are different ways to roast a chicken or cook it, as we all know that each speaker will sound different. But in the kitchen the most important thing is the quality of the product! Because anyone will be able to tell the difference between an industrial chicken and a free-range chicken.

God I love audio forums. Where else would you find an in-depth discussion of the validity of roast chicken as a metaphor for speaker design?
 

ttimer

Active Member
Joined
May 28, 2020
Messages
116
Likes
161
And there is this:

View attachment 73503

Then how come we put him in controlled tests, the results are these:

index.php


All the groups similarly ranked these speakers. Error bars are there by the way to account for listener variation and it doesn't change the rankings.

The reviewers had their shot at showing differing preferences but failed.

At this risk of being nitpicky, this graph actually supports his position. Reviewers are (almost) the only group who like speakers B, P and I equally well, which shows that their tastes encompass a wide range of speaker characteristics.
But then it doesn't surprise me that people who spend their days listening to speaker after speaker have an appreciation for speakers which sound "different", simply because the excitement of a "new experience" overrules flaws in tonality.
 

fredoamigo

Addicted to Fun and Learning
Forum Donor
Joined
Mar 11, 2018
Messages
606
Likes
1,003
Location
South East France
God I love audio forums. Where else would you find an in-depth discussion of the validity of roast chicken as a metaphor for speaker design?
Yes, metaphors between audio, food and cars are very common.
Here's the stereophile article quotes one. But it doesn't hold up.;)
 

Rja4000

Major Contributor
Forum Donor
Joined
May 31, 2019
Messages
2,682
Likes
4,220
Location
Liège, Belgium
He wants a wild west where anything goes so the random opinion of a reviewer can be always correct no matter how wrong it is
Well, to be fair, an opinion is ALWAYS correct.
Because that's what it is: an opinion.

The only thing is to make it clear it's an opinion, and not present it as some kind of truth.
 
Last edited:

Koeitje

Major Contributor
Joined
Oct 10, 2019
Messages
2,293
Likes
3,884
God I love audio forums. Where else would you find an in-depth discussion of the validity of roast chicken as a metaphor for speaker design?
I just had someone argue that we would all be driving Skoda's instead of BMW's if we would blind test cars. This person actually sells hifi....
 
Top Bottom