The “Real” Speaker Preference Test

[Arindal]

Sighted bias is extremely strong. Add the effects of sales personnel trained to manipulate the customer in order to clinch the sale, and a customer "in an existing shop under sighted conditions" doesn't have a chance.

Let us assume you were right. Sighted bias is as strong, and perfect measurements are a strong/increasingly important selling proposition, so I would either way expect products with excellent measurements, which are regularly winning blind/controlled listening tests, to be preferred under sighted conditions as well. Or at least sell at a decent number, compared to products being rated lower in controlled tests.

As this is obviously not the case, at least with more or less expensive passive speakers, I raise the question: What exactly is the decisive selling proposition, strong enough to overrule any subjective listening preference? The look of the speaker? Sounds unconvincing to me, looking at B&W or Wilson. The skills of the promoter in direct contact with the potential buyer? Possible, but I guess Harman and KEF also had their local promoters, attractive margins, running flagship stores, doing marketing campaigns, persuading dealers, reviewers. Revels and luxury-priced JBL/KEFs should be dominating the market.

Sorry to say, but the theory does not really ad up to what was happening in the real world.

Only a rigorously controlled blind test has any chance of providing a listener with truly valuable information.

I agree these are useful, if differences in sound quality are between negligible and zero. If potential buyers understand themselves, they had to pay a hefty premium for no sound quality gains, many might buy considerably cheaper electronic components.

With loudspeakers, situation is different, as audible differences are usually pretty pronounced, easily recognizable, and I would assume majority of buyers of $5k+ gear is actually doing listening tests prior to buying. So, if sighted bias is strong enough to overrule actual listening preference in majority of cases, I wonder what this strong force is?

 

[Matt_Holland]

Other way ´round: If they are reliable being rated best under blind and matched conditions, I would expect them to sell at least on a decent, if not dominant level, under sighted conditions in existing shops.

Whjy would you expect that? We know how powerful biases are.

You are basically saying that the vast majority of consumers are choosing expensive speakers solely by the looks, which they knowingly perceive as inferior in terms of sound quality.

No, looks is one factor. Reviews, brand, brand ethos/technology, price, mood on the day, demo music used, affect of the sales person, etc. Unless a speaker sound horrible, a person can easily convince themselves to buy the one that all the biases, except their ears, tells them to.

I see no evidence for that.

I see no evidence to support your viewpoint.

And I did not notice any B&W, Focal or Wilson looking so irresistible in the eyes of potential buyers, that this reverses inferior results in a listening test. If things would be that easy, Harman would have hired a few products designers from B&W and Focal, and made it to at least the market dominance they have with portable speakers.

B&W, Focal or Wilson are popular for many reasons, not just looks.

 

[Matt_Holland]

Could you list all variables and their targets included in 'Harman principles'?

No.
Any by Harman, I mean Toole/Olive principles as presented in Sound Reproduction.

 

[olieb]

Revels and luxury-priced JBL/KEFs should be dominating the market.

If that is your conclusion, you might need to revise your logic.
By the same chain of arguments the luxury watch market should be dominated by Seikos in an oyster design.
You are totally underestimating the power of bias. [Which has little to do with looks, but with brand image, "stories", price and all kinds of myths and misconceptions. Suggestion and imagination are what dominates. And there is a whole culture of bias, from manufacturers, gurus, magazines, "reviewers" to advertising, sales people, peers, ...]

 

[Jim Taylor]

Let us assume you were right.

What's the alternative? Assume I'm wrong, and go against years of research into human bias? I don't think so.

perfect measurements are a strong/increasingly important selling proposition,

I have never seen "perfect" measurements as a strong selling proposition. If they are important, they are something that are important to the buyer. The sales personnel couldn't care less whether the merchandise measures well or not. They will always find a way to manipulate the circumstances and the customer to clinch the sale.
If this were not true, there would be no sales of SET amps or garbage speakers.

I agree these are useful, if differences in sound quality are between negligible and zero.

Controlled tests are useful in ALL circumstances. Differences in volume, dispersion and frequency response can trigger the "attention" (bias) of a listener to such a degree that the listener indicates preference in that particular situation that does not carry over to their home situation nor match their long-term satisfaction.

Contrary to your statement, the more differences that exist between devices under comparison in uncontrolled conditions, the more important a controlled test becomes. The reason is that obvious differences excite powerful customer interest (auditory looming bias), and a guileful salesman will take advantage of that. More times than not, the customer will walk away with a product that has markedly inferior measurements because it exhibits differences that are obvious, not "negligible to zero". That's how bias works.

I've been involved with audio for almost 60 years, and I've seen this time and time and time again. It has fueled repetitious purchasing cycles for many people who just ... can't ... understand ... why ... it ... doesn't ... sound ... the. ... same ...as .. it ... did ... in ... the ... showroom.

I would assume majority of buyers of $5k+ gear is actually doing listening tests prior to buying.

A buyer who tells a salesman that they have done listening tests is one of the juiciest plums to pick off the tree. They give the salesman a reference from which they can destroy all the preconceptions of the customer, and a wide-open playing field to instill their own preferences in the customer's mind. And they will do so with "proofs" that the customer can't refute.

In the end, it's all about the $$$$.

P.S. - There are many articles about bias, but this is one that is easily readable, sufficiently explanatory and covers common biases that are active in our daily life. It's a good indication of why we should trust dispassionate measurements rather than uncontrolled subjective evaluations.

13 Types of Common Cognitive Biases That Might Be Impairing Your Judgment

Cognitive biases can impair rational judgment, lead to poor decisions, and cause us to believe falsehoods. Learn more about common biases that sway your thinking.

www.verywellmind.com

 

[Ellebob]

Been in this business a long time and there are two main criteria people comparing speakers are most influenced. Both of these show in research as well. I'm not talking about enthusiasts on this forum but the general public and audiophile crowd.

Aesthetics is one, we know in sighted tests better looking speakers usually scored higher than they did in blind testing. Unless some glaring acoustics differences people like the.better looking speakers if this is going in a living room. How many times on this forum alone have people said the Genelecs are ugly or don't meet the spouse acceptance factor. My wife included when I ask her opinion on speakers, aesthetics is where her comments go. I don't ask her about sound as she has low frequency loss in one ear and high frequency in the other caused by chemo treatments years ago. She wears hearing aids, so not a good candidate for Harman speaker testing.

The second one is bass, which can be a big factor in decisions when people compare speakers. They might talk about mids and highs but bass sells. Research shows this as well. We don't hear bass as well at lower and moderate volumes. If you have two bookshelf speakers and one rolls of at 60hz and one at 45hz, the one that goes lower will often be preferred. If you have two speakers that roll off about the same frequency but one has a little bass boost that is the one often preferred. If you don't plan on messing with any of your controls, this may be the better choice for some people. We can talk about getting neutral speakers and using a 'loudness' button or a dynamic volume feature but most people comparing speakers will never consider this.

Research of what sounds the best when you remove other factors like sight and the research on marketing and selling do not always coincide.

 

[kharan]

Agreed that sale figures mostly reflect good marketing and story telling, not audio performance.

How many times on this forum alone have people said the Genelecs are ugly or don't meet the spouse acceptance factor. My wife included when I ask her opinion on speakers, aesthetics is where her comments go.

You don't even need to invoke a significant other.

How many times have we heard people here say that the orange ring on the Palmer Orbit 11 immediately disqualifies that speaker for them? Or that the ridges on the Purifi drivers make any speaker using them a non-starter?

 

[Arindal]

Whjy would you expect that? We know how powerful biases are.

If differences are clearly distinguishable and strong enough to lead to statistically relevant disliking of a product in blind tests, they would not disappear and being reversed in a sighted test. And which is the bias so powerful it could reverse people´s subjective evaluation with a high degree of certainty? I simply don´t see that.

Reviews, brand, brand ethos/technology, price, mood on the day, demo music used, affect of the sales person,

Fully agreed, but all that could and should speak in favor of Revel, high end JBL, and others as well, if not lead to them being in an advantageous position, at least for a certain share of potential buyers. Other companies like KEF, Kii Audio and Nubert seemingly managed to build their brand reputation at least partially on technical arguments and ´scientific´ brand image. Why not Harman´s own brands in high end stereo, while being extremely successful with other products?

Unless a speaker sound horrible, a person can easily convince themselves to buy the one that all the biases, except their ears, tells them to.

According to Dr. Olive´s comparison tests dating from the early 2000s, they were identified as sounding horrible. Particularly B&W and ML being regularly mentioned as clearly inferior. At least B&W was pretty successful back in the days, eventually being bought by Harman and ironically presented as their primary speaker brand. So I wonder what was driving this strong bias away from Revel towards B&W?

What's the alternative? Assume I'm wrong, and go against years of research into human bias?

The alternative is to allow the easiest explanation: The majority of people having expensive speakers, has been buying the ones they liked the sound of the most, in their sighted tests at dealerships, with all imperfections, and all potential deviations these speakers might show from a theoretical ideal. @Ellebob mentioned preference for boosted bass, which seems to be preferred both in blind and sighted tests. It is one potential explanation.

The question to discuss in this case would be, why controlled blind tests under lab conditions lead, except from bass, to reversed results than real-world tests in dealerships. There are plenty of explanations for that, bias being one of them, but not the strongest in my opinion.

I have never seen "perfect" measurements as a strong selling proposition.

This was referring to what Amir and others were lately claiming in the AJ thread about the importance of perfect measurements in today´s speaker market. I ten to agree with you, perfect measurements are not a strong selling proposition nowadays, the majority of buyers ignore it or despise it. But they used to be persuasive, and so was ´Better sound through research´, back in the 1970s and 1980s.

How many times on this forum alone have people said the Genelecs are ugly or don't meet the spouse acceptance factor. My wife included when I ask her opinion on speakers, aesthetics is where her comments go.

I am partially agreeing, for the fraction of the market being into design gear, ending up with Bose, B&O, Sonos or LSX II. I was exactly in the position you are describing, tried LSX II in the bedroom which were lauded as ´the most beautiful speakers ever´. But the sound was so..., we had to send them back, bias was not strong enough.

And i don´t think, this theory applies on larger $5k+ tower speakers. They are all ugly, according to aforementioned member of Gen Y/Millenials. Maybe it´s a generation thing.

The second one is bass, which can be a big factor in decisions when people compare speakers. They might talk about mids and highs but bass sells. Research shows this as well.

Fully support your theory. And Harman, selling literally millions of boomboxes, portable bluetooth speakers, in-car systems and party gear, giving a strong indication, that they know how to apply their ´science´. So why not with passive high end stereo speakers?

 

[Purité Audio]

Agreed that sale figures mostly reflect good marketing and story telling, not audio performance.


You don't even need to invoke a significant other.

How many times have we heard people here say that the orange ring on the Palmer Orbit 11 immediately disqualifies that speaker for them? Or that the ridges on the Purifi drivers make any speaker using them a non-starter?

Just yesterday a really nice guy and potential customer came to listen to the C8Cs, his first words were, ‘they look better in real life’ he had heard some Sonus Fabers and liked their appearance, aesthetics play a huge role.
Keith

 

[Matt_Holland]

Just yesterday a really nice guy and potential customer came to listen to the C8Cs, his first words were, ‘they look better in real life’ he had heard some Sonus Fabers and liked their appearance, aesthetics play a huge role.
Keith

With all due respect to Ascilab I think that if someone has an interest in their speakers it is because they value measured performance (even if they don’t understand all the available data) over looks.

What did the guy think of the C88 sound? Did he describe how he thought it compares the Sonus Fabers?

 

[Purité Audio]

I believe he really liked the C8Cs, he auditioned the SFs in town somewhere, ‘Premium sound’?
I didn’t really ask how they compared, if both had been here, together…
Keith

 

[DSJR]

Just yesterday a really nice guy and potential customer came to listen to the C8Cs, his first words were, ‘they look better in real life’ he had heard some Sonus Fabers and liked their appearance, aesthetics play a huge role.
Keith

I'd add that excellent as the original and Matrix series' of the B&W 801s were, I'm not sure that they sold much into the domestic market, although they were placed in many mastering suites in the 80s (Polygram as-was certainly). I also liked the baby 805 AS LONG AS the listening axis was on the bass-mid driver with tweeter slightly above, as the tweeter kind of lifted up with the listener if they stood up above them, detaching the highs alarmingly from the midrange. The N series with expensive curvy carcasses came along and bang! Domestic sales began in earnest, despite the sound balance in my experience, not being anywhere near as good as their forebears (nor when subsequent measurements were taken). Tannoy tried with curvy-carcasses, but they screamed at the listener I remember (old Klipsch on steroids) and didn't last long, the old-fashioned looking Prestige doing better ultimately I remember, but never in the UK where Tannoy never seemed to make real inroads in the new millennium.

 

[kimmosto]

No.
Any by Harman, I mean Toole/Olive principles as presented in Sound Reproduction.

I have the book. It's very generic without any...much targets, requirements and different concepts with their special features/requirements. As a speaker designer, I would consider it fairly useless due to its generality for common concept and conclusions based on limited data or methods.

 

[RexrothPigeon]

What exactly is the decisive selling proposition

This is a pointless line of thought: the data don't allow us to disentangle the various factors that contribute to sales statistics, so just as ratiocination can never equal empirical evidence, sales statistics aren't a valid proxy for any specific consumer preference.

 

[RexrothPigeon]

With all due respect to Ascilab I think that if someone has an interest in their speakers it is because they value measured performance (even if they don’t understand all the available data) over looks.

Every time I step outside my home I am reminded that my tastes cannot be extended to other people. Most people like things I find hideous, and many things I like are hideous to others.

 

[sergeauckland]

I wonder how similar is buying high end hifi and cars. Do many buy these on engineering quality, or is it brand image and looks? Is a Maserati better or worse engineered than, say, a Ferrari? How many considering either would care, or have sufficient knowledge to be able to judge?

Ditto boats, aircraft or any other expensive 'toy' that has a large engineering component, but is bought mostly by non-engineers?

If that is the case, then it's not surprising if with many, especially smaller, HiFi manufacturers, the marketing department is more important than Engineering.

S.

 

[thewas]

I have the book. It's very generic without any...much targets, requirements and different concepts with their special features/requirements.

Which edition do you have?

Obviously its not a book for loudspeaker designers but thanks to it and generally the contribution of Toole and Olive understanding and choosing good sounding loudspeakers has moved from the medieval mystique of opinion gurus and problematic listening in audio sales to an objective, science based approach accessible and understandable by any interested hobbyist.

 

[Jim Taylor]

If differences are clearly distinguishable and strong enough to lead to statistically relevant disliking of a product in blind tests, they would not disappear and being reversed in a sighted test. And which is the bias so powerful it could reverse people´s subjective evaluation with a high degree of certainty? I simply don´t see that.

It is obvious to me that you did not read the article that I offered you.

The question to discuss in this case would be, why controlled blind tests under lab conditions lead, except from bass, to reversed results than real-world tests in dealerships. There are plenty of explanations for that, bias being one of them, but not the strongest in my opinion.

Which explanations do you believe are stronger?

This was referring to what Amir and others were lately claiming in the AJ thread about the importance of perfect measurements in today´s speaker market. I ten[d] to agree with you, perfect measurements are not a strong selling proposition nowadays, the majority of buyers ignore it or despise it. But they used to be persuasive, and so was ´Better sound through research´, back in the 1970s and 1980s.

You misunderstand my meaning.
In the first place, let's get the importance of Toole out into the open. His research indicates that the average listener will exhibit preferences for flat frequency response and low coloration in speakers IF AND WHEN ALL OTHER THINGS ARE EQUAL. (My emphasis.) But those things that must be controlled and equalized in a blind test ... sound pressure level, visual keys, suggestions by other people, etc. ... can override those preferences quickly and effectively. That's why a really good salesman is so valuable to the business. He can sell the proverbial ice cubes to Eskimos by manipulating biases.

Secondly, my assertion is not that all buyers value measurements, but that IF measurements are to be valued, they will more likely valued by the buyer than the seller. The seller, after all, is there to sell anything he can to whoever he can, any time he can and in any way that he can. He has no intrinsic interest in perfect measurements. If he can use them to make a sale, he will. If not, then they are the last thing on his mind.
It is the CUSTOMER who has an interest in measurements, and even then, only the educated ones. Remember that I said many people get speakers home and then wonder why they don't sound anything like they did at the dealer's? An educated customer, aware of total speaker measurements ... including spinorama, distortion and frequency response ... will be better prepared to predict whether the speaker under consideration will sound acceptable in his home environment, and will do so long-term. An uneducated customer who just goes to the dealer unprepared and listens under the "tutelage" of a salesman is, as con artists say, an "easy mark".
The same holds true of "boutique" tube electronics.

And I believe it is not true to say that the majority of buyers ignore or despise measurements. If that were true, ASR would not have the volume of traffic that it has. All indications are that more and more people are becoming educated about measurements, and more and more people are coming to use them and value them.

 

[ShadowFiend]

I have 4th edition and I agree with @kimmosto. It is great book but what it provide is the basis of available "current" science of sound production in conjunction with mostly conventional box speaker, which is the test subject most of the time. Other type of transducer, like omnidirectional, dipole, cardioid, are only briefly mentioned. In addition, the subject of the preference on different directivity patterns leaves a lot to be desired IMHO.

 

[kimmosto]

Which edition do you have?

3rd. Probably won't buy the last.
Agree that it's not targeted for designers who need more specific and concept-related information. Experience and other sources are needed to provide missing information and specifications for products/concept in question. For example, DIY books can have more useful information, though they don't focus much to generic theory and background, not necessarily refer so many studies. They usually focus to basic boxed concept too, but probably with less images of JBL products.
Possible risks with science-base are constraints and arrangements of the studies; environment, speaker and listener positions, available samples, speaker concepts and listened program. That could lead to poor compromise focusing to variable(s) which are not adequate for the concept and quality target. Some generic target full-filling "Harman specs" is not necessarily significant enough in total. In addition, results in listening tests depend on program. Harman's elevator muzac playlist could be sufficient for basic tonality tests, but it's limited. Also listening single "center speaker" in fixed setup is limited if the final goal is stereo listening. Very limited if design target includes also casual listening from almost anywhere. We also have other sensors than ears so simple audibility studies with ears (headphones) are not inclusive enough for evaluating reproduction with loudspeakers assuming that target is kinda live music with haptic experience. Science is a good helper, but just touching the surface (magnitude responses and directivity) won't get us far.