Stereophiles editor Jim Austin publicly disagreeing with Kal Rubinson

[preload]

If none of those food critics can identify that burnt flavor unless they know the food was cooked on stove A ahead of time, I don't care about their subjective opinion because it's false. We already have a hypothesis that explains such situations - cognitive bias.

That's certainly one way of looking at it, but again, it's not thinking like a research scientist would. That would be dismissing data because it doesn't fit your original hypothesis. Maybe that approach works in engineering? It's very frowned upon research science.

Rather, a research scientist would say, "it's possible that the food critics tasted a burnt flavor because they knew the identity of Stove A. Let's test that by blinding the food critics to the identity of each stove, and see if we get the same results." This is called "controlling for bias."

By the way, I'll skip to the end. What actually happened in this hypothetical example is that the the thermocouple was place in a position that measured an average temperature, and the average temps of each stove were identical. However, Stove A had hotspots that made certain parts of the frying pan hotter than others, whereas Stove B had more even thermal distribution. Hence, Stove A was actually causing some parts of the food to be burned.

If we had taken the "engineering" approach, we would have taken the "measured" data at face value and completely dismissed the overwhelming "subjective" signal (in this case, 90 food critics out of 100 tasting something burnt when the food was cooked on stove A). And in doing so, we would have overlooked an important difference between Stove A and Stove B that actually existed.

 

[dorakeg]

There is no room for, fact based evidence and scientific measurements, in SALES endeavors. Unless said science/engineering strongly supports the SALES pitch. SALES and Truth represent the polar opposite extremes in which a SALES pitch can function to motivate readers into Buyers. Not believers. Feelings, Emotions, Desires, Imagination and Dreams are what they Sell. This should not surprise anyone who has been a member of ASR for more than a week.

I would say it depends alot on the industry and what product you are selling. I assume your post is specific to audio industry.

 

[preload]

Most of these impressions tell us more about the subjects' biases and thought processes than they do about the equipment under review. Next thing you will be telling us is that there are some as-yet-unforeseen phenomena happening in the equipment that engineers don't know about but can be picked up by audio reviewers although strangely absent when presented with a proper double-blind test.

Well, I'm glad you brought this up. Prior to the landmark Harman studies, the "understanding" of the audio engineering community was that a flat, horizontal sound power response was the most desirable. Consumer Reports even applied this doctrine and ranked loudspeaker systems in one of their magazine reports base on this understanding. And if you guessed that they ranked a Bose loudspeaker #1, you'd be correct. It wasn't until Toole/Olive said, that model is incorrect, and they more or less proved it by applying both the "Consumer Reports" measurement model and their own proposed model to a sample of loudspeakers and correlating them with SUBJECTIVE listener impressions that understanding of measurements and speaker shifted in the right direction.

BUT, if everyone had said, "Dr. Olive and Dr. Toole, you don't know what you're hearing, loudspeaker measurements should have flat sound power," we wouldn't have the wonderful loudspeakers their research inspired today.

 

[Beave]

Well, I'm glad you brought this up. Prior to the landmark Harman studies, the "understanding" of the audio engineering community was that a flat, horizontal sound power response was the most desirable.

Link to your source?

Also, weren't the studies done at the NRC, before Toole's time at Harman?

 

[dlaloum]

Exactly.

Yes and yes. This is how we know how well objective loudspeaker (and headphone) measurements correlate with subjective listener preferenes.
People get upset when I point out that the correlation is not perfect, and that Harman's paper could only show that objective loudspeaker measurements explained a lot (but not all) of the differences in subjective listener ratings, so I'll stop now.

And the tests were incomplete as there were very few bipolar/dipolar or omni speaker designs involved - it really is relevant only to standard monopole speakers

 

[dlaloum]

BUT, if everyone had said, "Dr. Olive and Dr. Toole, you don't know what you're hearing, loudspeaker measurements should have flat sound power," we wouldn't have the wonderful loudspeakers their research inspired today.

I believe that the default room response for omni speakers is in fact flat sound power....

Hence the Harman recommendations really are only applicable for a particular subset of speakers (it happens to be the most common speaker type, but still!)

 

[preload]

Link to your source?

Also, weren't the studies done at the NRC, before Toole's time at Harman?

Sure. Olive S. "Loudspeaker Preference Using Objective Measurements: Part 1 - Listening Test Results." AES 116th Convention 2004:

"All 13 loudspeakers were recently evaluated by CU and reported in their magazine Consumers Report [9]. CU uses an objective-measurement based model to predict the accuracy rating of the loudspeaker. This provided an opportunity to compare CU’s predicted loudspeaker accuracy ratings against measured preference ratings using well-controlled listening tests. To the author’s knowledge, no one, including CU, has ever published the results of such a test."

"The flat sound power criterion had a large contingent of support in the United States. In 1968 [21] Bose argued that, “when a loudspeaker is properly placed with respect to the rear reflecting wall, the frequency response measured with respect to the total radiated acoustical energy should be flat”, a view which was reinforced in a later publication [22]. Other supporters in the New England locale included McShane [23], Allison and Berkovitz [24], Torick [25], Hirsch [26], and in 1973, Consumers Union (CU) [27]."

"There are many theoretical arguments why the CU model might not work including the accuracy of the loudness model used or even the appropriateness of
applying such a model. However, the ultimate test is how accurately the model predicts listeners’ sound quality ratings."

Boys, pay attention to Olive's last sentence. These are subjective listener sound quality ratings that are numericalized into analyzable data.

 

[Beave]

Interesting. All supporters seem to be from the northeastern US. Can you show us what the citations [22] through [26] go to?

I knew the story about Consumer Reports, and I had heard/read they got the idea from Bose, but I was unaware that several others were proponents of flat sound power.

 

[dlaloum]

"The flat sound power criterion had a large contingent of support in the United States. In 1968 [21] Bose argued that, “when a loudspeaker is properly placed with respect to the rear reflecting wall, the frequency response measured with respect to the total radiated acoustical energy should be flat”, a view which was reinforced in a later publication [22]. Other supporters in the New England locale included McShane [23], Allison and Berkovitz [24], Torick [25], Hirsch [26], and in 1973, Consumers Union (CU) [27]."

Note that Amar Bose proposed a "pulsing sphere of sound" model, on which he based the 901 speaker design - basically an omni

Quad did something similar with its Quad ESL63 - where delay lines built into the panels are design for the sound radiation to emulate a point source about 30cm or so behind the panel

Both of these design approaches result in a flat sound power response

These speaker types were not included in the subjective evaluation work of Olive & Toole - so yes their results are applicable to the type of speaker category that they evaluated - the standard "box" monopole.

But do not provide a basis for evaluation of whether, or not, the conclusion are applicable to the other categories of speakers (omnis, dipoles, bipoles)

It should also cause one to approach wide dispersion designs with caution - and NOT to assume that the standard "Harman" profile is applicable... a speaker with 330 degree dispersion (eg: Gallo CDT) might have a completely different optimal preferred room response - more akin to an omni...

And then, how does one evaluate a hybrid design?

 

[Sal1950]

The kids have a great expression; 'touch grass'

Could someone explain what that means?

 

[JSmith]

Could someone explain what that means?

Yeah it means go outside basically... get away from the PC/Internet, get outside and touch some grass.

I'd suggest many would prefer to smoke grass rather than just touch it, but whatever people enjoy.


JSmith

 

[birdog1960]

Could someone explain what that means?

Urban Dictionary: touch grass

used when someone is doing something weird, stupid, or pointless. it means they need to come back to reality, they need to get some fresh air and get back in touch with how the real world works.

www.urbandictionary.com

not humble. but grass is ok. some say it makes music sound better. seemed to work for Miles Davis.

 

[Deleted member 21219]

"However, the ultimate test is how accurately the model predicts listeners’ sound quality ratings."

Boys, pay attention to Olive's last sentence. These are subjective listener sound quality ratings that are numericalized into analyzable data.

I remember that Olive did a listener preference study with both trained and untrained listeners. The study was a counterpoint to the then-accepted flat-power norm, and permitted listeners to use both treble and bass tone controls, and to set them any way that they wanted. It showed that listeners in general preferred increased bass, but that there was a difference between trained listeners and untrained listeners.

I believe this was the beginning of what eventually became the Harman curve. And if I'm not wrong, this set of listener preferences is the "sound quality" ratings to which he refers in this quote. The phrase " ... subjective listener sound quality ratings ...." would, most often, be taken by a general readership to refer to uncontrolled and non-specific ratings. I don't believe that to have been the gist of Dr. Olive's comments.

As Kal Rubinson replied in a post when I asked about the nature of "subjective" commentary:

It may involve defined choice constraints or prior training to insure agreement on terminology. It definitely will not allow for free-form commentary. That is all part of what experimental design is about.

If I have this wrong, please correct me.

Jim

 

[BDWoody]

Absolutely... clinical drug trials are always set against a harmless placebo.

We humans are amazingly sensitive to micro changes or differences, but we are also far from infallible.

Not just human trials, but animal drug trials as well. I was reviewing the research for some medicine my cat is taking, and the Veterinarians and pet owners were also 'blinded' to whether they were using placebo or the drug...as would be expected by most of us here I believe.

 

[AdamG]

Absolutely. It's a valuable service. But may I suggest doing all that with a bit more humility? I know, look in the mirror, dude.

We do try. I would say for the most part we have a very high level of civil discourse here. But human nature being what it is, this is a constant struggle to maintain high levels of “humility”. It gets harder and harder to easily determine whom is here to Troll our Membership and whom is hear for constructive dialogue and learning. The more Senior a member becomes, the more jaded they can get. I am surprised and proud of the level(s), of professionalism and chivalric behavior demonstrated here, towards new members seeking scientific knowledge and technical understanding/expertise. Many selfless acts go uncounted and under appreciated. Considering that many of our members are Engineers and Scientists with extensive education and experience. Freely sharing their collective knowledge and expertise. Humility is everywhere if you look for it.

 

[ahofer]

In this very thread we had a new member jump in to make snide and ignorant remarks. I was pleased to see them mostly ignored.

It just gets tiresome combatting the same old tropes and intentional conflations (no need for blind testing, I have golden ears, you all are flat earthers, music=equipment, ’there’s more on heaven and earth’[unintentionally arguing that ghosts exist]). So we get high-handed.

 

[fpitas]

"Listening to music is believing"
Love it. JackL is a funny guy.
Reminds me of a fond subjectivist audiophile colleague of mine who always ends our Hifi discussions (debates) with "trust your ears" or "it's all about the music".

Whenever I hear or read those phrases I keep a tight grip on my wallet.

 

[birdog1960]

In this very thread we had a new member jump in to make snide and ignorant remarks. I was pleased to see them mostly ignored.

It just gets tiresome combatting the same old tropes and intentional conflations (no need for blind testing, I have golden ears, you all are flat earthers, music=equipment, ’there’s more on heaven and earth’[unintentionally arguing that ghosts exist]). So we get high-handed.

straw men. no one argued any of those points.

 

[fpitas]

straw men. no one argued any of those points.

Just wait...

 

[ahofer]

straw men. no one argued any of those points.

On this site? You are wrong, my friend. Here’s a recent one.

And in the subject thread? Most are there. See “JHL” for instance, or the “skeptical ears” guy.

Made me go look. I need a shower now.