• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Do We Want All Speakers To Sound The Same ?

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,524
Likes
37,057
Pun intended, it’s much like how one feels your own lover is the best looking in the world
Excellent example. We all know at one time Amber Heard was declared the best looking woman in the world. Ask Johnny Depp how that turned out?
 

gnarly

Addicted to Fun and Learning
Joined
Jun 15, 2021
Messages
990
Likes
1,390
My opinion in a nutshell.....
of course all speakers should sound alike.

Their only differences should be in intended radiation patterns, maximum linear SPL, and low frequency extension.
Simply due to matching priorities (and musical tastes haha) to real world constraints.
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,524
Likes
37,057
My opinion in a nutshell.....
of course all speakers should sound alike.

Their only differences should be in intended radiation patterns, maximum linear SPL, and low frequency extension.
Simply due to matching priorities (and musical tastes haha) to real world constraints.
Okay, now that we have an answer. Should all speakers be heard alike?
 

Newman

Major Contributor
Joined
Jan 6, 2017
Messages
3,448
Likes
4,211
Here's the relevant plot from that book:

View attachment 249577

As you can see, the only significant change in going from blind to sighted listening was the drop in the ranking of a small cheap sub-sat system, vs three tower systems. The relative rankings of the three towers were unchanged.

IMO, this refutes your claim that sighted listening is "dominated by non-sonic factors".

OMG

and here is Toole's text around the above graph that you say refutes claims that sighted listening is not about the sound waves:
"A widespread belief among audio professionals is that they are immune to the influences of brand, price, appearance, and so on. They persist in conducting listening evaluations with the contending products in full view.
... Holt commented as follows:
As far as the real world is concerned, high-end audio lost its credibility during the 1980s, when it flatly refused to submit to the kind of basic honesty controls (double-blind testing, for example) that had legitimized every other serious scientific endeavor since Pascal. [This refusal] is a source of endless derisive amusement among rational people and of perpetual embarrassment for me, because I am associated by so many people with the mess my disciples made of spreading my gospel.
...In general, though, what listeners saw changed what (they thought) they heard."

And his summary of that section of the book (my bold):
"Summarizing, it is clear that knowing the identities of the loudspeakers under test can change subjective ratings.
  • They can change the ratings to correspond to presumed capabilities of the product, based on price, size, or reputation.
  • So strong is that attachment of “perceived” sound quality to the identity of the product that in sighted tests, listeners substantially ignored easily audible problems associated with loudspeaker location in the room and interactions with different programs."

and,
"These findings mean that if one wishes to obtain candid opinions about how a loudspeaker sounds, the tests must be done blind."

How much clearer can he be? And yet your lesson from it was your post above????

QED
 

MarkS

Major Contributor
Joined
Apr 3, 2021
Messages
1,062
Likes
1,502
I looked at the data, not the words. If sight "dominated" over sound, we would expect sighted data to scramble the blind ratings. That's not what the data that Toole chose to publish in his book indicates. It indicates that there is an effect, but far from a dominant effect.
 

kemmler3D

Major Contributor
Forum Donor
Joined
Aug 25, 2022
Messages
3,015
Likes
5,617
Location
San Francisco
I looked at the data, not the words. If sight "dominated" over sound, we would expect sighted data to scramble the blind ratings. That's not what the data that Toole chose to publish in his book indicates. It indicates that there is an effect, but far from a dominant effect.
It's one test with 4 speakers and we don't know what they looked like. Just because the rankings didn't shift massively in this one test doesn't invalidate anything. A much less drastic conclusion is just that the speakers looked about equally fancy / expensive.

As @Newman pointed out, I don't think using Toole to refute Toole is going to be a winning strategy.
 

dlaloum

Major Contributor
Joined
Oct 4, 2021
Messages
3,092
Likes
2,353
Thanks @MattHooper for a stimulating set of questions.

I've had a thought - we manage to drive down distortion in amplifiers through the use of negative feedback (and before anyone declares a dislike for feedback, I can recommend Bruno Putzeys' "The F Word".

Can we do something similar with loudspeakers? Everyone here using PEQ and REW etc to control some room nodes or to crossover to a sub is doing "manual negative feedback". But it requires a lot of work and is quite coarse. And when someone walks in the room, the settings should be tweaked, or if the curtains are opened or closed, we need to change it again.

But why can't we do this in real-time? Why can't we create a setup that is always correcting for speaker errors, driving down distortion (by subtracting the speaker distortion from the source) and handling errors in real time?
The WHY..... is because right now, it is too expensive - the level of processing power required, the cost of those processors...

we are a couple of orders of magnitude away from the technology needed being viable - we would need Audyssey or Dirac to work in real time - adjusting in sub millisecond timeframes.
 

dlaloum

Major Contributor
Joined
Oct 4, 2021
Messages
3,092
Likes
2,353
While I feel like this is above my pay grade or at least my limited experience with different speakers, some questions come to mind.

Do 2 speakers with similar frequency response and dispersion sound the same if you did a double blind test using test signals or music in the same room? What if the speakers were the same model off the same assembly line? My gut says that at least some people would be able to tell the difference. This is in spite of reviews I've read where the reviewers have said this model sounded very similar to that model.

The other issue is how do speakers accurately mimic the dispersion of live instruments including the human voice. Edgar Villchur of Acoustic Research (the largest speaker company at the time produced a series of “Live versus Recorded” concerts in which live performances by musical ensembles were compared with previously taped performances played through AR stereo equipment. Musicians participating in these concerts included the Fine Arts String Quartet and classical guitarist Gustavo Lopez, as well as performances on a thirty-two foot pipe organ and an old-fashioned nickelodeon. The Washington Post featured the Live vs. Recorded concerts with a half-page article with pictures, providing free publicity for AR, in which they said that audiences were fooled over and over by the seamless transitions between live performance and sound reproduced through the AR speakers. Quote from https://auralfetish.com/pages/acoustic-research. I've forgotten how this was explained, but it had something to with the forgiving nature of the music and possibly the audience. Obviously there have been major improvements in speaker components and cabinets since then. It brings to mind the CD motto of perfect sound now and forever which was closer to the truth then in the 1980's then it was for speakers in the 1960's.
The assumption about improvements since the 1960's is often brought into question by some of the very best speakers of that era...

The Quad ESL, and then the ESL63 - first released in 1957 and 1963 respectively, The Klipschorn (pre WWII)... and others.

Obviously all speakers have limitations - and you can point at each speaker (both these vintage examples and any speaker manufactured today) - and point out potential flaws - rooms that they would not "work" in, etc.... - but also many of these designs from 50+ years ago, did some things so well, that most speakers made today still struggle to match them.

The assumption of evolutionary "improvement" is a fallacy, both when observing Nature and Genetics, and when looking at speakers over the decades.

Marketing, Fashion, changes in living room decor, have driven speaker changes far more than actual objectively verifiable "improvements.

A Quad ESL57 will challenge pretty much any speaker made today with regards to the reproduction of the human voice - that critical midrange .... (and yeah it has lots of other aspects that can make it hard to live with)
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,524
Likes
37,057
The assumption about improvements since the 1960's is often brought into question by some of the very best speakers of that era...

The Quad ESL, and then the ESL63 - first released in 1957 and 1963 respectively, The Klipschorn (pre WWII)... and others.

Obviously all speakers have limitations - and you can point at each speaker (both these vintage examples and any speaker manufactured today) - and point out potential flaws - rooms that they would not "work" in, etc.... - but also many of these designs from 50+ years ago, did some things so well, that most speakers made today still struggle to match them.

The assumption of evolutionary "improvement" is a fallacy, both when observing Nature and Genetics, and when looking at speakers over the decades.

Marketing, Fashion, changes in living room decor, have driven speaker changes far more than actual objectively verifiable "improvements.

A Quad ESL57 will challenge pretty much any speaker made today with regards to the reproduction of the human voice - that critical midrange .... (and yeah it has lots of other aspects that can make it hard to live with)
One correction. The ESL 63 was released in 1981. It was called the 63 because that is when Peter Walker had the idea for its design. Took 18 years to make it a reality. The ESL 57 was at the time simply called the Quad ESL. It was retroactively named the ESL 57 by audiophiles because that was its first year of production.
 

dlaloum

Major Contributor
Joined
Oct 4, 2021
Messages
3,092
Likes
2,353
One correction. The ESL 63 was released in 1981. It was called the 63 because that is when Peter Walker had the idea for its design. Took 18 years to make it a reality. The ESL 57 was at the time simply called the Quad ESL. It was retroactively named the ESL 57 by audiophiles because that was its first year of production.
Thank you... I remembered that the original was retroactively named 57 based on first year of production - and assumed the same of the 63... shouldn't have done that!
 

Descartes

Major Contributor
Joined
Dec 27, 2020
Messages
2,104
Likes
1,077
….First, I very much would like significant research and engineering to continue to push the state of the art, and at least one version or aspect of that is low distortion/accuracy…..I can see why some evaluate speakers by comparing them to some ideal in terms of measured performance. I have yet to encounter a speaker that sounds as if there were no speaker in the chain, so every speaker to me has some character or another. In that sense I am not comparing speakers to a set of measurements, but to other speakers. ….

I compare speakers to actual instruments rather than other speakers, having the opportunity to have several family members who play the piano, the violin, the clarinet and the guitar, allows me to listen to speakers with these points of reference!

If only speaker manufacturers would do the same maybe we would have more speakers that are accurate.
 

Newman

Major Contributor
Joined
Jan 6, 2017
Messages
3,448
Likes
4,211
A couple of tips for neophytes to science, coming across research literature or compilations by doyens of the field:-
  1. Don't cherry-pick data as if it proves the doyen/researcher's conclusions wrong.
  2. Don't fold your arms in front of your chest and pronounce yourself unconvinced by the conclusions or the quality of the work that is before your eyes at this moment. Not unless you are an equal or bigger doyen: any takers?
Here's the deal: the deeply experienced, widely read and highly respected career researcher does not overreach to flippant conclusions. When he or she does draw a conclusion, it is not just from the evidence visible in the current piece of work: it is an experienced, expert interpretation of the most viable working hypothesis from the sum of the current work and all the relevant work carried out in the field, in which he or she is extremely widely read and highly trained. The conclusion is never drawn in isolation, or without regard to congruence or otherwise with the vast repository of other relevant work, or without regard to the as-yet-unanswered questions on the matter.

The idea, that we relative neophytes can assess and judge those conclusions, is absurd. If one feels inclined to do so, one is better advised to question oneself where the resistance is coming from: wishful thinking? Sighted listening (quelle surprise)? OTOH perhaps further research of equal merit, made since the doyen's conclusion was published, does indeed warrant a reconsideration and a new working hypothesis. After all, science moves on. But not towards the end point of confirming the myths embedded in audiophiles by sighted listening experience, LOL, so don't sit there with a sneer "waiting for the day" that those tone-deaf labcoaters finally confirm what you knew all along. Try to be here for the learning experience, not the resisting experience.

cheers
 

YSC

Major Contributor
Joined
Dec 31, 2019
Messages
3,194
Likes
2,570
I looked at the data, not the words. If sight "dominated" over sound, we would expect sighted data to scramble the blind ratings. That's not what the data that Toole chose to publish in his book indicates. It indicates that there is an effect, but far from a dominant effect.
But this data IMO is showing why the sight is dominated over sound, note that the top 2 candidate are the visually identical European samples costing $5000, and one US competition costing $4000, which in raw "sighted" brand reputation, size or material should not be too different a class to the European ones, while the small sub/sat system is kind of like normal bundle 5.1 systems in HT or even computer 2.1/5.1 sets, I will just for the sake of simpleness, call it computer speakers.

So going blind test shows preference of the 2 EU ones champing the computer speakers, which champ the USA expensive one.

Bottomline now this is what the "sound only" preference, going to sighted, the sonically "inferor" USA speaker now overtaken the computer speaker, yet still fall behind the EU counter parts.

This is exactly what you would expect when sighted bias is dominant, when all the multi grand speakers looked bad ass, powerful, professional, exotic ...... you still fall back to sonic signature preference for ranking them only, but then when the computer speaker actually sonically outperform the 5 times more expensive hifi one, the look of the hifi/pro big speaker instantly bumps it to be preferred over the cheapo looking computer speakers.

Plus even with the top two ranked ones, maybe they looked really pro or something, even their relative rank didn't change, the score is bumped up by a lot while only the compupter speaker have a lower score sighted vs blind. this is what exactly demoing "look" alone dominates the difference in sound.

Now this is a random sample of N test individuals, for a single individual, i.e. anyone of us trying to purchase, the thing like brand reputation, look or so, be it the truth or not (saying Genelec metalic sound, ATC mythical dome or so, or even Beats by Dre "sounded as the musician intended"), or sheer size of the woofers, could make identical sounding speakers sound "nah something is not right" to "this is most realistic I've ever heard", bottom line being most of the options are fairly neutral on axis (i.e. quite similar) and we can get the sighted bias to dominant even more on the speaker choice.

But then even with all those influence, there's not "wrong" about the individual which choose the one that he feels better. Just one question: Do you want the better "Speaker" which "Sound" is transparent, or you just want something that could sound worse objective, but with other "strengths" like good old pro brand, sexy look, status revealing price tag or so to get satisified.

A more extreme example is the multi grand cables which I think most if not all here agree is snake oil, if one budget consumer focusing to opt for sound improvement, buying a cheap/inferior speaker just to use that cable is a big no, but if it's Tom Cruise, yea why not put another 20 grand on cables with his million buck system, just to make him feels nothing is left behind?
 
OP
MattHooper

MattHooper

Master Contributor
Forum Donor
Joined
Jan 27, 2019
Messages
7,197
Likes
11,813
Well, those Harman tests were done sighted at times. One thing they showed is pretty much you cannot count on sighted impressions. Your knowledge of reputations or in the case of sighted employees speakers that were from their company and a host of other biases like the usual price and appearance all distorted rankings badly. This in the same room with the ability to switch between speakers very quickly. I believe there is evidence we mostly hear thru the room at least down to 500 hz and even lower frequencies in large rooms.

We know that bias is hovering over virtually everything we perceive. (Folks on other audiophile forums get annoyed by my bringing up the problem of sighted bias all the time!). The issue is a need to keep perspective: how far is it reasonable to push skepticism on what can be ascertained under uncontrolled conditions?

At some point along the continuum between "totally unreliable" and "very reliable" we will find some middle ground of "reliable enough" to come to pragmatic (even if tentative) conclusions. We do this all day long. As I keep pointing out, the only reason my efforts "work" in my job is that there is some level of predictability in how other human beings will perceive the sound I am creating. The general success of the work would be inexplicable on the hypothesis that sighted listening conditions so derange perception that it becomes too unreliable to predict. (Much of my job is manipulating perception...in a reliable manner).
Similarly, if you took two neutral speakers and equ'd a lift between 2 and 4K, at some point as you keep increasing the difference, it is highly likely it will be reliably identified in both sighted and blind conditions. One being reliably identified (by audiophiles who can still hear those frequencies!) as brighter than the other. Again, if this weren't the case, all the sonic manipulation mixers and engineers do - including EQ and mastering - would be irrational and uncesseful.

Now, when it comes to the specific study of speakers, in which subjects preferences changed from sighted to blind conditions, the one I keep seeing cited is this one, discussed in Sean Olive's blog:


It's the original 1994 test set up by Olive et al at Harmon. Admittedly, while I have seen a lot of Toole's writing, watched his talks etc, I haven't read his book yet, so I don't know what else he has in there in terms of specific blind vs sighted tests like the one described above. I note in his terrific, much watched talk on youtube, he cites the same study as Olive above when talking about the problem of sighted listening.

So looking at the results:


[BlindVsSightedPositionInteractions.png]


The point Olive and Toole will make from this test is how you can see some distinct changes in ratings between sighted and blinded listening, as well as
how the sighted listening impressions seem to also overwhelm the sonic changes even in changing the speaker position. All well and good. But is that the ONLY thing the data seem to show? While, yes it shows variation from sighted to blind ratings - anyone with any familiarity with psychology would predict that! - does it completely trash sighted perception? For speaker position 1, there is real variability. But for loudspeaker position 2...much less so. For speaker position 2, some general trends hold from sighted to blind listening:

For Speaker Position 2, in both sighted and blinded conditions:

1. G and D are rated higher than S and T
2. D rated very best
3. S rating is pretty consistent
4. T shows the most variability, though still rated below G and D


So, loudspeaker position 2 seems to have some effect on the reliability of the ratings - general trends similar in sighted to blind listening are there. I'm curious: What would a bunch more studies with speaker in position 2 show? Would we see more of these sighted listening trends carry over to blinded conditions? (If that was followed up, I'm all ears).

Another graph is, I presume (and would like to be corrected if wrong) a summary of the trends found in the test (presumably including both speaker positions?)





If I'm interpreting the graph correctly, it tells us that, despite some differences, everything taken together the trend is that G and D were consistently rated best, and with a similar difference in rating, S suffered the least correlation, T a relatively strong correlation. (Again, please correct my understanding if wrong).

If so, even this study, often cited to trash sighted listening still suggests some correlation between sighted impressions and "sound only" impressions.
Which wouldn't be a total surprise: our hearing evolved in sighted conditions, so there has to be SOME level of reliability in sighted conditions.

I have also on this forum cited numerous times where the sonic impressions of audiophiles and audio writers tracked well with the measurements of a speaker.

Has left me wondering about sighted listening and especially those done different places and different times removed from each session. I've also mentioned that I'm pretty confident I could identify the bass of a large panel vs box speakers using this method having lived with panels for years. It is so hard however, you get stronger feelings about what is a good sound vs bad when sighted than when blinded. I've done what everyone else does out of a lack of alternatives. Go listen and see what I think about the sound of speakers. Maybe a cautionary phrase since I instinctively typed "see" what I think.

I found all early Thiele speakers to sound brightish. The designer was shooting for flat measured response (not the downward slope) which makes sense. I found early Vandersteen speakers to sound a little slow, lacking detail and soft. The designer also was shooting for flat measured response. What limited data there is indicates the Vandy's were probably much less peaky up and down off axis. Thiele speakers pretty much nailed the perfect Q on the woofer, and Vandy speakers had that Acoustic Coupler that sort of added low end. Maybe those explain the difference or not. Both could do a pretty good square wave though not as good as a Quad ESL63.

I had read so many good things about the Quad ESL63 without hearing them. A friend purchased some and said I needed to come listen to them. I already owned some Acoustat Twos. When I heard the ESL63 it seemed well beyond any speaker I'd heard. I said right then I had to have some. Within a few months I did. I used them happily for 12 years until one malfunctioned. Were all of these opinions close to useless due to bias that became more reinforcing the more I listened? I don't think any of us want to answer that question with a yes. The big scary question is whether the correct answer is yes.

It's always *possible* that we can be imagining things. Even Big Things. As I've often said on other forums: People have imagined they are probed by aliens. You don't think an audiophile can imagine a reduction of "midrange glare?" when he tries a new AC cable? :)

But hyper-skepticism would be paralyzing and untenable. We could set up countless tests that show our perception of sound to be very reliable. If I test whether you can tell the difference between a dog bark and a cat meow, or between the voices of two people you know well, you'd be likely, like most people, to reliably score %100 correct. Your blinded test will track perfectly with your sighted tests. Similarly, when a pair of speakers diverge *enough* in sound - say in the piercing quality of the high frequencies, or the depth/pounch of the low frequencies, then in all likelihood people would be just as accurate identifying the differences under blind conditions, as they heard in sighted conditions. The question becomes where do we approach this middle ground were it is reasonable to say "Yeah, I probably am identifying sonic characteristics between A and B, even in sighted conditions."


Obviously some sound is so far from the realm we can say so without blind comparisons.

You beat me to it!

You would think living with a speaker would work too. How far can bias cover up for differences in sound? Not far enough to like what most sound systems in bars and concert arenas sound like. Not enough to like what lots of rap fans want their car stereo to sound like. Maybe much further than we'd like to think with home stereo however. It is almost impossible to answer isn't it?

Yes, lots of gray area. That's why at some point we have to acknowledge when it's reasonable to come to conclusions even though we haven't scientifically controlled for biases. "Yeah, it's possible what I'm perceiving is skewed via a bias effect, but it's also plausible enough for me to proceed on the conclusion the differences are real."
 

kemmler3D

Major Contributor
Forum Donor
Joined
Aug 25, 2022
Messages
3,015
Likes
5,617
Location
San Francisco
The WHY..... is because right now, it is too expensive - the level of processing power required, the cost of those processors...

we are a couple of orders of magnitude away from the technology needed being viable - we would need Audyssey or Dirac to work in real time - adjusting in sub millisecond timeframes.

Not really, we have the processing power to do realtime corrections / processing of this type. With a modern GPU you can do many 16K-tap FFT filters in realtime. ANC headphones do realtime correction with a chip that would not win any awards for FLOPS.


I think the problem is that you need to correct the sound before it leaves the speaker cone, or within a small fraction of a single waveform after the speaker cone moves. This more or less precludes a servo-style correction at high frequencies. This means we need an ADC running at 192khz or so, but luckily there is nothing fancy about that. However, you also need a measurement of the tweeter's motion that's more accurate than your tweeter... good luck with that part... maybe some kind of fancy laser like on the Klippel, what do those go for these days? And, if you have all that, you do need to complete the correction in about 0.1ms, so latency is a challenge for sure.

e: such lasers do exist: https://www.micro-epsilon.com/displacement-position-sensors/confocal-sensor/ here's one with 70khz speed and nanometer resolution. Good enough for my ears. However, it's 'call for pricing' which means it costs more than your whole setup. But I guess in theory we can do full-range servo control in 2022 if we have the time, money, and inclination. Would love to see one of those $150K setups go this high-tech.

A feedback system working at greater distance (like with a mic) wouldn't work for the simple reason that you can't correct a waveform after it's left the speaker. So for other styles of realtime correction, you need predictive models of behavior.

Realtime ANC in headphones works well because the diaphragm is closer to your ear than the sound you're trying to eliminate. You have time to capture and process it before it reaches your ear. Not so sitting far away from loudspeakers.

I think removing room influence at a single listening position is possible today, using ANC techniques adapted for a larger space. However, it's not even remotely possible to do for a whole space. How do you capture and fully model a 3D sound field with a handful of 2D microphones? I am not sure if anyone knows yet.

You could measure the room with LIDAR, raytrace to find intensity of reflections at a given listening position, and subtract that signal from the speaker's output. Perhaps supplement this with mic input to adjust in realtime. I think here we are talking about something that would tax consumer GPUs /CPUs pretty badly, but should be possible with high-end consumer hardware. There is a new 3D printer on the market that uses LIDAR and only costs $1500, it's not very exotic anymore.

Removing distortion might also be possible with a very high resolution model of how the speaker distorts, but keep in mind distortion is nonlinear, so it's not as simple as standard ANC or EQ. You need a different high resolution FIR filter for each sample AND a full set for each gain level. Probably a sweep at ... 60 different gain levels would be necessary, if not 100+? This might really tax a GPU, maybe here we reach some practical limits of computing power.
 
Last edited:

YSC

Major Contributor
Joined
Dec 31, 2019
Messages
3,194
Likes
2,570
We know that bias is hovering over virtually everything we perceive. (Folks on other audiophile forums get annoyed by my bringing up the problem of sighted bias all the time!). The issue is a need to keep perspective: how far is it reasonable to push skepticism on what can be ascertained under uncontrolled conditions?

At some point along the continuum between "totally unreliable" and "very reliable" we will find some middle ground of "reliable enough" to come to pragmatic (even if tentative) conclusions. We do this all day long. As I keep pointing out, the only reason my efforts "work" in my job is that there is some level of predictability in how other human beings will perceive the sound I am creating. The general success of the work would be inexplicable on the hypothesis that sighted listening conditions so derange perception that it becomes too unreliable to predict. (Much of my job is manipulating perception...in a reliable manner).
Similarly, if you took two neutral speakers and equ'd a lift between 2 and 4K, at some point as you keep increasing the difference, it is highly likely it will be reliably identified in both sighted and blind conditions. One being reliably identified (by audiophiles who can still hear those frequencies!) as brighter than the other. Again, if this weren't the case, all the sonic manipulation mixers and engineers do - including EQ and mastering - would be irrational and uncesseful.

Now, when it comes to the specific study of speakers, in which subjects preferences changed from sighted to blind conditions, the one I keep seeing cited is this one, discussed in Sean Olive's blog:


It's the original 1994 test set up by Olive et al at Harmon. Admittedly, while I have seen a lot of Toole's writing, watched his talks etc, I haven't read his book yet, so I don't know what else he has in there in terms of specific blind vs sighted tests like the one described above. I note in his terrific, much watched talk on youtube, he cites the same study as Olive above when talking about the problem of sighted listening.

So looking at the results:


[BlindVsSightedPositionInteractions.png]


The point Olive and Toole will make from this test is how you can see some distinct changes in ratings between sighted and blinded listening, as well as
how the sighted listening impressions seem to also overwhelm the sonic changes even in changing the speaker position. All well and good. But is that the ONLY thing the data seem to show? While, yes it shows variation from sighted to blind ratings - anyone with any familiarity with psychology would predict that! - does it completely trash sighted perception? For speaker position 1, there is real variability. But for loudspeaker position 2...much less so. For speaker position 2, some general trends hold from sighted to blind listening:

For Speaker Position 2, in both sighted and blinded conditions:

1. G and D are rated higher than S and T
2. D rated very best
3. S rating is pretty consistent
4. T shows the most variability, though still rated below G and D


So, loudspeaker position 2 seems to have some effect on the reliability of the ratings - general trends similar in sighted to blind listening are there. I'm curious: What would a bunch more studies with speaker in position 2 show? Would we see more of these sighted listening trends carry over to blinded conditions? (If that was followed up, I'm all ears).

Another graph is, I presume (and would like to be corrected if wrong) a summary of the trends found in the test (presumably including both speaker positions?)





If I'm interpreting the graph correctly, it tells us that, despite some differences, everything taken together the trend is that G and D were consistently rated best, and with a similar difference in rating, S suffered the least correlation, T a relatively strong correlation. (Again, please correct my understanding if wrong).

If so, even this study, often cited to trash sighted listening still suggests some correlation between sighted impressions and "sound only" impressions.
Which wouldn't be a total surprise: our hearing evolved in sighted conditions, so there has to be SOME level of reliability in sighted conditions.

I have also on this forum cited numerous times where the sonic impressions of audiophiles and audio writers tracked well with the measurements of a speaker.



It's always *possible* that we can be imagining things. Even Big Things. As I've often said on other forums: People have imagined they are probed by aliens. You don't think an audiophile can imagine a reduction of "midrange glare?" when he tries a new AC cable? :)

But hyper-skepticism would be paralyzing and untenable. We could set up countless tests that show our perception of sound to be very reliable. If I test whether you can tell the difference between a dog bark and a cat meow, or between the voices of two people you know well, you'd be likely, like most people, to reliably score %100 correct. Your blinded test will track perfectly with your sighted tests. Similarly, when a pair of speakers diverge *enough* in sound - say in the piercing quality of the high frequencies, or the depth/pounch of the low frequencies, then in all likelihood people would be just as accurate identifying the differences under blind conditions, as they heard in sighted conditions. The question becomes where do we approach this middle ground were it is reasonable to say "Yeah, I probably am identifying sonic characteristics between A and B, even in sighted conditions."




You beat me to it!



Yes, lots of gray area. That's why at some point we have to acknowledge when it's reasonable to come to conclusions even though we haven't scientifically controlled for biases. "Yeah, it's possible what I'm perceiving is skewed via a bias effect, but it's also plausible enough for me to proceed on the conclusion the differences are real."
The mean loudspeaker ratings and 95% confidence intervals are plotted in Figure 1 for both sighted and blind tests. The sighted tests produced a significant increase in preference ratings for the larger, more expensive loudspeakers G and D. (note: G and D were identical loudspeakers except with different cross-overs, voiced ostensibly for differences in German and Northern European tastes, respectively. The negligible perceptual differences between loudspeakers G and D found in this test resulted in the creation of a single loudspeaker SKU for all of Europe, and the demise of an engineer who specialized in the lost art of German speaker voicing).



Brand biases and employee loyalty to Harman products were also a factor in the sighted tests, since three of the four products (G,D, and S) were Harman branded. Loudspeaker T was a large, expensive ($3.6k) competitor's speaker that had received critical acclaim in the audiophile press for its sound quality. However, not even Harman brand loyalty could overpower listeners' prejudices associated with the relatively small size, low price, and plastic materials of loudspeaker S; in the sighted test, it was less preferred to Loudspeaker T, in contrast to the blind test where it was slightly preferred over loudspeaker T.

The above are the conclusions from the page which makes perfect sense of the results, and this, is actually a test when he first joined Harman, the guys who give the listening test are the trained, very experienced Harman employees which I would assume are more "pro" than most ppl here. and even they are wildly affected by the sighted bias, this alone to me says everything
 
Last edited:

MarkS

Major Contributor
Joined
Apr 3, 2021
Messages
1,062
Likes
1,502
This is all great news! We can just make fake boxes that look like expensive speakers, and put one crappy 4" full-range driver in them, and they will all sound just the same! No one will be able to hear a difference!
 

Holmz

Major Contributor
Joined
Oct 3, 2021
Messages
2,018
Likes
1,241
Location
Australia


Thanks @MattHooper for a stimulating set of questions.

I've had a thought - we manage to drive down distortion in amplifiers through the use of negative feedback (and before anyone declares a dislike for feedback, I can recommend Bruno Putzeys' "The F Word".

Can we do something similar with loudspeakers? Everyone here using PEQ and REW etc to control some room nodes or to crossover to a sub is doing "manual negative feedback". But it requires a lot of work and is quite coarse. And when someone walks in the room, the settings should be tweaked, or if the curtains are opened or closed, we need to change it again.

But why can't we do this in real-time? Why can't we create a setup that is always correcting for speaker errors, driving down distortion (by subtracting the speaker distortion from the source) and handling errors in real time
The WHY..... is because right now, it is too expensive - the level of processing power required, the cost of those processors...

we are a couple of orders of magnitude away from the technology needed being viable - we would need Audyssey or Dirac to work in real time - adjusting in sub millisecond timeframes.

I am not so sure.
Most of this could done “open loop”, and subwoofer frequency are so low that it can almost be done on a abacus.

Thanks @MattHooper for a stimulating set of questions.

I've had a thought - we manage to drive down distortion in amplifiers through the use of negative feedback (and before anyone declares a dislike for feedback, I can recommend Bruno Putzeys' "The F Word".

Can we do something similar with loudspeakers? Everyone here using PEQ and REW etc to control some room nodes or to crossover to a sub is doing "manual negative feedback". But it requires a lot of work and is quite coarse. And when someone walks in the room, the settings should be tweaked, or if the curtains are opened or closed, we need to change it again.

PEQ and REQ are open loop, there is no negative or positive feedback, and no feedback at all.



But why can't we do this in real-time? Why can't we create a setup that is always correcting for speaker errors, driving down distortion (by subtracting the speaker distortion from the source) and handling errors in real time

We could do it in real-time.
  1. We need to know the cause(s) of the distortion… Before…
  2. We can know what to do to drive down the distortion.
  3. This is something where one assumes that it is NOT magic and we do NOT want apply some AI to understand it. The distortions are manfiested by real things that are largely understood.
For instance; take the Purifi work.
They pretty much addressed the causal mechanisms for the distortion, and went about ameliorating them.

One could just know that the surround causes the effective area to not be constant, and drive the motor in a non linear way to account for it.
Maybe that gets the unit to have 10dB lower distoriton.

Is it better or worse than a better and more consistent surround?

It is better if one is make 10M units, and can save coins.
 

dlaloum

Major Contributor
Joined
Oct 4, 2021
Messages
3,092
Likes
2,353
I am not so sure.
Most of this could done “open loop”, and subwoofer frequency are so low that it can almost be done on a abacus.




PEQ and REQ are open loop, there is no negative or positive feedback, and no feedback at all.




We could do it in real-time.
  1. We need to know the cause(s) of the distortion… Before…
  2. We can know what to do to drive down the distortion.
  3. This is something where one assumes that it is NOT magic and we do NOT want apply some AI to understand it. The distortions are manfiested by real things that are largely understood.
For instance; take the Purifi work.
They pretty much addressed the causal mechanisms for the distortion, and went about ameliorating them.

One could just know that the surround causes the effective area to not be constant, and drive the motor in a non linear way to account for it.
Maybe that gets the unit to have 10dB lower distoriton.

Is it better or worse than a better and more consistent surround?

It is better if one is make 10M units, and can save coins.
I am thinking more along the lines of analysing the frequencies up to circa 2KHz (although in a perfect world we want 20kHz)- similar to what Dirac is proposing with SRC - but doing it in real time. (I am setting the 2khz limitation based on what Dirac has published with regards to their SRC system, showing that the benefits - as the system currently stand, mostly are achieved below 2kHz)

Real time analysis of not just the speakers direct signal, but the reflections as well, and then real time correction of those sound flaws both direct and reflected through active cancellation - so the end result is closer to the theoretically targeted sound field.

That would ultimately be the path to Nirvana - the ability to simulate the soundfield we want within any listening space (given sufficient speakers, and processing power).
 

olegtern

Member
Joined
Sep 23, 2020
Messages
84
Likes
98
Which wouldn't be a total surprise: our hearing evolved in sighted conditions, so there has to be SOME level of reliability in sighted conditions.
We can go deeper down the rabbit hole. Somewhat speculative ideas, but at least some new notes in the discussion :)

In short. Modern neuroscience proves that the brain is not a reflective organ but a modeling organ. The brain does not reflect reality, it constantly creates it, adjusting its forecasts with incoming sensory information. (Karl John Friston, Thomas Metzinger etc). We are constantly modeling reality a little ahead of time, and adjusting perceptions retroactively. And all this seems to us a continuous perception of reality, although this is not so.

We perceive not what we hear. What we hear corrects our expectations and refines the model. Just like what we see. And all this affects the virtual reality that we create for ourselves.

In common we assume that what we heard in the blind experiment "is true", and the prejudiced perception "is false". We discuss bias from what we see as something that pollutes, degrades the quality of what we hear. But this is at least not an accurate concept.

The brain models the perceived sound based on expectations and adjusts based on all available information. The tactile sensation of the bass affects the perception of space and presence, it is not hearing, but it affects the quality of what is heard. In the same way, vision affects the "rendering quality" of the sound image in our internal virtual reality.

This distorts perception, but it does not necessarily "worse" it, at least there are situations where bias can enhance some aspects of perception. And no matter, if we talking about the ability to assess the quality of the sound (which seems to be an objective reality), or about our general impressions of the sound (which seems to be a subjective reality) - in both cases we are dealing with a subjective virtual reality.

For example, the idea that people would prefer fancy-looking speakers is not always true. At any audio-show, there are examples of expensive and visually impressive speakers which get very poor audience ratings. And even trained listeners on such shows don't necessarily choose the prettiest or most expensive speakers.

Or on the example of the Tools experiment cited above. Visual perception corrected listeners' estimates, but did it make them less accurate? Or (at least in some cases) more accurate? If we are talking about trained listeners, will the ability to see the system worsens or improves the accuracy of the assessment? Our hearing is not technically accurate, we are constantly filling in the gaps in what we hear or the errors of the senses, creating a virtual perception. But if we add visual information (for example, the location of the speakers in the room, or speakers build), do we impair or improve the ability to assess? At least this is an interesting question.

As for subjective impressions. We love eating in a beautiful restaurants more than in a messy kitchen, listening to opera in beautiful concert halls (built as a place for celebration, not for critical listening)), and to watch movies on the big screens. Visual experience enhances our ability to immerse ourselves in virtual reality. But this is not just a simple psychological trick, it's like a ticket to a better performance, when the brain starts to create a better rendering.

Do we want all speakers to look the same? :p
 
Top Bottom