Master Thread: Are measurements Everything or Nothing?

[Galliardist]

Of course *knowing* that our equipment has been selected for transparency and/or fidelity to the linear on axis/even dispersion standard also creates a positive bias in our listening experience.

Does it? Evidence?
(Sorry. I'm in that sort of mood at the moment)

 

[goat76]

I think the whole point of the science behind the spinorama measurements, is to identify which measurements are highly aligned with almost everyone's preferences, and how to interpret the measurements for best alignment.

The "almost everyone's preferences" doesn't have to be the most important thing when you want to know what works best for your personal preferences.

One loudspeaker can have an outstandingly flat frequency response while the distortion is sky-high but some listeners may not be bothered by it at all, while another speaker may have some deviations in the frequency response but has a wider dispersion that perfectly suits another listener. The list of things that can make a listener prefer a certain loudspeaker over another can be many different things, even if most other listeners may prefer another speaker.

You can't just say 'who knows, maybe it isn't': that is a bit like hand-waving away all the work (and evidence) that went into this.

No, it's not. Why wouldn't you choose the loudspeaker you preferred in a blind listening test, isn't that the ultimate test to determine what suits your taste no matter the outcome?

If you are saying, "maybe I am an exception", then the work has established that probability to be very high if you have significant hearing damage beyond normal age-related decline, and the probability is very low if you don't.

Yes, there have been many people who have participated in the blind tests Toole has set up, who didn't fall right in the middle of the preference curve. Their taste in sound did deviate more or less from the average Joe's taste, and one of those people could be me or you, we don't know before we have done the blind testing ourselves.

So, if one's hearing is okay-ish, one can proceed with confidence to correlate measurements to subjective excellence in the sound waves themselves.

cheers

I don't think deviations in the taste of sound necessarily have to do with hearing loss. Which listeners have the most problems with their hearing, the fan of the Genelec sound or the Neumann sound?

 

[MattHooper]

Isn't it just replacing one intangible, "there is something in the sound that isn't being measured" with another, "there is something in your head that isn't being measured".

Let's step back a bit, and drop the "why". Is there a study or proper citable evidence that simply demonstrates, going back to your earlier post, that the sighted listening effect is

You are treading around some of the types of questions that I have raised before, that some people around here don’t seem to like much.
It gets into the weeds, the messiness of real life, and some prefer to cling to certainties.

The issue I have raised, and as far as I can tell, you are also hinting at, is that any true, coherent picture of our perception is going to have to account not just for the fallibility on one side, but also on the reliability and success at the other end. No fully sceptical account of our perception as too fallible and untrustworthy could possibly make sense of all the success we have.

Therefore, our account of our perception is going to have high unreliability to one side, and high reliability to the other side, and then we are going to have to account for the
“ mushy middle” in between. And since most of our time is actually spent in that mushy middle, it is worthwhile to have some account as to what kind of beliefs we can justify while in that space. Because we can’t be doing science all the time, and yet we have to justify plenty of perceptual, beliefs, and actions.

There are some here who want to cling to the certainty of the scientific studies that point to our perceptual fallibility. And they become uncomfortable or highly suspicious, if someone starts trying to talk about the big picture, where we get into areas of uncertainty.

As an analogy, imagine somebody keeps telling us that our visual system is simply too fallible to trust at all, and unless one has measurements or results using scientific controls, any claim that assumes the reliability of our vision is to be rejected.

What is this based on? “The science!”
Which science? “ You know, all those studies about optical illusions! The optical illusions studies where people’s perception is reliably fooled, demonstrates the inherent unreliability of our vision for actually perceiving reality!”

Wait though! If you’re conclusion is a fully sceptical one about our visual perception, that it is wholly unreliable, do you realize that hypothesis is going to also have to account to the huge amount of success and reliability we seem to have with our vision? How does your hypothesis explain the fact, people pass eye exams, all the time? How does it explain people driving successfully? How does it explain every day success using our vision?
I work as a graphic designer . How does your thesis explain my success?

“Don’t be ridiculous, those questions aren’t pertinent, I’m talking about what has been scientifically demonstrated! Look like I can even demonstrate it to you. Here is the checkerboard illusion. See how you were perceiving the squares labelled A as significantly darker than B? OK now let’s isolate just the squares A and B from the rest of the image. And now you can see that you were fooled! Now you can see that ANB are precisely the same shade. This clearly demonstrates the inaccuracy of your vision!”

Wait, hold on. Do not see what you just did there? There’s a contradiction hiding in your example! You first showed a condition under which my vision was inaccurate - when seeing the full illusion - but in showing me next the squares were the same shade, you’ve assumed the reliability of my vision at that point! So the demonstration you are appealing to contains not only Scepticism about the reliability of our vision, it simultaneously acknowledges it can be reliable!

“ What is this deceitful sophistry you are bringing? You are just trying to trick me with these Questions. Look, I’m appealing to scientific studies! The science is therefore, obviously on my side, so I’m clearly right about our visual system being too fallible to trust on its own! Therefore you are JUST BEING ANTI-SCIENCE!!”

What’s going on here? This person simply isn’t grasping the problems, the legitimate issues that have been brought up numerous times to his sceptical thesis. Either that or he has intuited his claims, those boundaries of certainty, may be starting to unravel somewhat in a way that makes him uncomfortable, and he just continually retreats to the “ certainty” he thinks he has, He’ll just continue to cite the studies as if that made his position certain, and ignore unpleasant complexities or problems that attend his position.

Not only that, you absolutely know that as soon as this person has stopped arguing for the unreliability of our vision, he’s going to walk out the door and go through the day assuming the reliability of his vision, and will have plenty of success in doing so! This is the “ arguing in a bubble” problem I brought up before. Thinking you have some airtight position “ based on the science” while arguing in an audio forum, without having explored the implications in any wider context to see how it holds up.

This is what it’s been like trying to raise these issues sometimes on the forum, especially when you’ve got folks like Newman.

When we reference the fallibility of our perception, of the type clearly demonstrated in blind versus sighted audio tests, It remains a valid and important question to ask how that fits in with the wider picture of our perception. Because that wider picture is going to include the obvious amount of evidence we have for the success and reliability of our auditory perception. There’s lots of noise in the system, but if evolution hadn’t created a system that is able to rise above that noise… if not perfectly, but reliably enough… then it wouldn’t be a survival feature.

If like the person relying on optical illusion studies, you were going to emphasize scepticism about our perceptual system, a full account is going to have to include our success. And we are going to have to have some account for how it is reasonable to operate in the “ mushy middle” where we don’t have scientific controls available. What kind of inferences from our perception are JUSTIFIED in that area? Quite important since we tend to mostly operate in that area most of the day.

And just as there was an internal contradiction hidden in the optical illusion guys examples - in which the reliability of sight was not only questioned in one moment, yet reliability assumed in the next moment, I’ve raising a similar concern found in the appeals to blind testing and measurements:

Blind testing results, such as the results from the Harman Kardon studies, are used to bolster the sceptical case against sighted listening.

And yet, the measurements denoting “ better sound” in the blind test are used to design speakers to “ sound better” in sighted conditions as well! It therefore clearly assumed that the preferable Sonic characteristics heard under blind conditions will also be perceived under sighted conditions! Otherwise it makes total nonsense of Harman Kardon’s project.
Just like the checkerboard illusion demonstration, you have to acknowledge both sides of the equation: That the message includes both an acknowledgement of our fallibility AND acknowledgement of our sighted perceptual reliability! You can’t have it both ways and look only at the sceptical part of the equation.

That goes for the relevance of speaker measurements on ASR. It is advised to purchase speakers that measure well, in the sense of measurements that have been preferred under blind, listening conditions.
Except of course, we will be using our speakers in sighted conditions! If the measurements do not translate to what we will perceive in sighted conditions as well - which assumes some reliability of our sighted impressions! - then they may as well be nonsense for our actual use case.

And, just like the optical illusion guy who clearly assumes the reliability of his vision as soon as he stops talking about optical illusions, it’s very clear that people here are looking to the speaker measures to predict what they will perceive from those loudspeakers, should they buy them and use them in their sighted conditions! Which again assumes some level of reliability in sighted listening. And all the fuss members here go to, integrating subwoofers, ironing out the sound with room correction, it’s for sided listening conditions, generally assuming the results will translate to sided perception of the sound.

None of this is some form of “ gotcha.”

It is simply delving into the implications of the science we know, how far inferences from that science can be taken, and what type of reasoning people are basing off of that science, and how that can fit into the big picture of our perception. There is no arguing and a vacuum about this. Claims about fallible perception leak out implications far and wide.

Nor is it “ rejecting the science.” Precisely the opposite. It is accepting the results of what science we have, but trying to put that in the larger picture which would include the success of our perception, to see what we can or cannot justify In the large non-scientific realm in which we are usually operating day to day. What kind of conclusions from sighted listening can be justified, within the context of acknowledging that bias could be playing a role?”

If you say “ none, it’s all too untrustworthy”…
Then you invite all the type of problems I’ve been raising for the internal and external coherence of that stance.
It’s not going to work very well when you extend the implications to the rest of the real world.

 

[MattHooper]

Sorry. Too much time on my hands.

 

[Doodski]

Sorry. Too much time on my hands.

I'm seeing a person that in all probability aced English Composition studies.

 

[MattHooper]

I'm seeing a person that in all probability aced English Composition studies.

Try that shit on an iPhone dictation!
Not exactly helpful for maintaining writing skills.

 

[JustJones]

That was the unabridged version. I read the unabridged version of The Hunchback of Notre Dame once, for some crazy reason. GOTHIC Architecture

 

[Doodski]

Try that shit on an iPhone dictation!
Not exactly helpful for maintaining writing skills.

Hahaha... Even more impressive..LoL... Dictation and organizing one's thoughts in a readable order is tricky at best.

 

[Galliardist]

You are treading around some of the types of questions that I have raised before, that some people around here don’t seem to like much.
It gets into the weeds, the messiness of real life, and some prefer to cling to certainties.

The issue I have raised, and as far as I can tell, you are also hinting at, is that any true, coherent picture of our perception is going to have to account not just for the fallibility on one side, but also on the reliability and success at the other end. No fully sceptical account of our perception as too fallible and untrustworthy could possibly make sense of all the success we have.

Therefore, our account of our perception is going to have high unreliability to one side, and high reliability to the other side, and then we are going to have to account for the
“ mushy middle” in between. And since most of our time is actually spent in that mushy middle, it is worthwhile to have some account as to what kind of beliefs we can justify while in that space. Because we can’t be doing science all the time, and yet we have to justify plenty of perceptual, beliefs, and actions.

There are some here who want to cling to the certainty of the scientific studies that point to our perceptual fallibility. And they become uncomfortable or highly suspicious, if someone starts trying to talk about the big picture, where we get into areas of uncertainty.

As an analogy, imagine somebody keeps telling us that our visual system is simply too fallible to trust at all, and unless one has measurements or results using scientific controls, any claim that assumes the reliability of our vision is to be rejected.

What is this based on? “The science!”
Which science? “ You know, all those studies about optical illusions! The optical illusions studies where people’s perception is reliably fooled, demonstrates the inherent unreliability of our vision for actually perceiving reality!”

Wait though! If you’re conclusion is a fully sceptical one about our visual perception, that it is wholly unreliable, do you realize that hypothesis is going to also have to account to the huge amount of success and reliability we seem to have with our vision? How does your hypothesis explain the fact, people pass eye exams, all the time? How does it explain people driving successfully? How does it explain every day success using our vision?
I work as a graphic designer . How does your thesis explain my success?

“Don’t be ridiculous, those questions aren’t pertinent, I’m talking about what has been scientifically demonstrated! Look like I can even demonstrate it to you. Here is the checkerboard illusion. See how you were perceiving the squares labelled A as significantly darker than B? OK now let’s isolate just the squares A and B from the rest of the image. And now you can see that you were fooled! Now you can see that ANB are precisely the same shade. This clearly demonstrates the inaccuracy of your vision!”

Wait, hold on. Do not see what you just did there? There’s a contradiction hiding in your example! You first showed a condition under which my vision was inaccurate - when seeing the full illusion - but in showing me next the squares were the same shade, you’ve assumed the reliability of my vision at that point! So the demonstration you are appealing to contains not only Scepticism about the reliability of our vision, it simultaneously acknowledges it can be reliable!

“ What is this deceitful sophistry you are bringing? You are just trying to trick me with these Questions. Look, I’m appealing to scientific studies! The science is therefore, obviously on my side, so I’m clearly right about our visual system being too fallible to trust on its own! Therefore you are JUST BEING ANTI-SCIENCE!!”

What’s going on here? This person simply isn’t grasping the problems, the legitimate issues that have been brought up numerous times to his sceptical thesis. Either that or he has intuited his claims, those boundaries of certainty, may be starting to unravel somewhat in a way that makes him uncomfortable, and he just continually retreats to the “ certainty” he thinks he has, He’ll just continue to cite the studies as if that made his position certain, and ignore unpleasant complexities or problems that attend his position.

Not only that, you absolutely know that as soon as this person has stopped arguing for the unreliability of our vision, he’s going to walk out the door and go through the day assuming the reliability of his vision, and will have plenty of success in doing so! This is the “ arguing in a bubble” problem I brought up before. Thinking you have some airtight position “ based on the science” while arguing in an audio forum, without having explored the implications in any wider context to see how it holds up.

This is what it’s been like trying to raise these issues sometimes on the forum, especially when you’ve got folks like Newman.

When we reference the fallibility of our perception, of the type clearly demonstrated in blind versus sighted audio tests, It remains a valid and important question to ask how that fits in with the wider picture of our perception. Because that wider picture is going to include the obvious amount of evidence we have for the success and reliability of our auditory perception. There’s lots of noise in the system, but if evolution hadn’t created a system that is able to rise above that noise… if not perfectly, but reliably enough… then it wouldn’t be a survival feature.

If like the person relying on optical illusion studies, you were going to emphasize scepticism about our perceptual system, a full account is going to have to include our success. And we are going to have to have some account for how it is reasonable to operate in the “ mushy middle” where we don’t have scientific controls available. What kind of inferences from our perception are JUSTIFIED in that area? Quite important since we tend to mostly operate in that area most of the day.

And just as there was an internal contradiction hidden in the optical illusion guys examples - in which the reliability of sight was not only questioned in one moment, yet reliability assumed in the next moment, I’ve raising a similar concern found in the appeals to blind testing and measurements:

Blind testing results, such as the results from the Harman Kardon studies, are used to bolster the sceptical case against sighted listening.

And yet, the measurements denoting “ better sound” in the blind test are used to design speakers to “ sound better” in sighted conditions as well! It therefore clearly assumed that the preferable Sonic characteristics heard under blind conditions will also be perceived under sighted conditions! Otherwise it makes total nonsense of Harman Kardon’s project.
Just like the checkerboard illusion demonstration, you have to acknowledge both sides of the equation: That the message includes both an acknowledgement of our fallibility AND acknowledgement of our perceptual reliability! You can’t have it both ways and look only at the sceptical part of the equation.

That goes for the relevance of speaker measurements on ASR. It is advised to purchase speakers that measure well, in the sense of measurements that have been preferred under blind, listening conditions.
Except of course, we will be using our speakers in sighted conditions! If the measurements do not translate to what we will perceive in sighted conditions as well - which assumes some reliability of our sighted impressions! - then they may as well be nonsense for our actual use case.

And, just like the optical illusion guy who clearly assumes the reliability of his vision as soon as he stops talking about optical illusions, it’s very clear that people here are looking to the speaker measures to predict what they will perceive from those loudspeakers, should they buy them and use them in their sighted conditions! Which again assumes some level of reliability in sighted listening. And all the fuss members here go to, integrating subwoofers, ironing out the sound with room correction, it’s for sided listening conditions, generally assuming the results will translate to sided perception of the sound.

None of this is some form of “ gotcha.”

It is simply delving into the implications of the science we know, how far inferences from that science can be taken, and what type of reasoning people are basing off of that science, and how that can fit into the big picture of our perception. There is no arguing and a vacuum about this. Claims about fallible perception leak out implications far and wide.

Nor is it “ rejecting the science.” Precisely the opposite. It is accepting the results of what science we have, but trying to put that in the larger picture which would include the success of our perception, to see what we can or cannot justify In the large non-scientific realm in which we are usually operating day to day. What kind of conclusions from sighted listening can be justified, within the context of acknowledging that bias could be playing a role?”

If you say “ none, it’s all too untrustworthy”…
Then you invite all the type of problems I’ve been raising for the internal and external coherence of that stance.
It’s not going to work very well when you extend the implications to the rest of the real world.

It's actually worse than this, since we don't have definitive knowledge of what we should expect people to discern in every circumstance. There's an article on this site, but it doesn't contain proper references. I've seen research regarding limits in certain and some key areas. I use that article, at least here, because it is here - and I use the "lenient" limits in that article mostly.

But - not only do we not know when sighted listening may fail, we don't fully understand what someone could discern in sighted listening either. Some of us may, of course, and be able to explain. Thankfully there are online tests and tools that can help an individual discover some aspects of what differences they can hear, and usually hearing turns out to be somewhat worse than even most here would expect.

I do know that sighted comparisons do fail. I also know that people including myself have managed when testing with others to induce others to hear differences where we can be certain none exist. I know that blind testing makes differences go away, pretty much every time, where we can state that a difference is not in the sound waves with a degree of certainty. So I'm on board with the mainstream ASR tropes, as it were.

But, for example, my first posts here were in a thread where I could see somebody describing speakers that were not being properly driven in the bass. Checking, I found a combination of a speaker with very low impedance and a phase shift in the bass region, and amplification with low gain. Yet several people were telling the newcomer that he was imagining it. So that wasn't a great introduction, and it tallies with something I suspect, which is that serious problems in sound will normally be heard.

(cf.. for esample a series of reviews I read in the mid 1980s in the UK, where a sample Nagaoka cartridge got glowing reviews, until someone measured it and found that it was internally incorrectly wired - something that I would expect those reviewers at that time to notice. It was out of phase!)

Having said that, most of what "ASR knows" is, I suspect, still correct. Rather than our hearing always being fooled- we just don't know when our hearing is being fooled. It could be comparatively rare and it could be most of the time.

And we have far less chance of knowing if someone else's hearing is being fooled. That is a different proposition, but a lot of the same conclusions can be drawn.

Of course, in ASR land, we get lots of people turning up where they have been fooled (cables sounding different that aren't, for example) which reinforces the lesson again and again, and possibly makes things seem worse than they are.

Particularly, though, the lesson not to trust reviewers remains, for sound quality at least. I presume that where sufficient measurements exist, we could score subjective comments against reviews for a sample, and see if Toole's conclusion holds, and who gets closest to what might be expected from the measurements.

As for the poor people who wander into this place and find themselves "requested" to run their own blind test on some fancy cable or DAC, the onus should be on the industry to do those tests properly before any "remarkable products" even reach them. It shouldn't be on customers.

I'm sure @Newman has to hand the results Toole had for different groups of listeners and how well they performed in blind testing. Time for that to be posted again, it is at least something to chew on.

 

[Newman]

Let's step back a bit, and drop the "why". Is there a study or proper citable evidence that simply demonstrates, going back to your earlier post, that the sighted listening effect is :-
Newman said: more pervasive than almost anyone would believe when they look back on their own sighted listening experiences
Note, I'm letting you off "stronger" and "calibrate" with this version of the question.
Consider again the raw numbers of people. All the people on forums and working subjectively in the audio industry, all those people who come here with those unusual claims, are a tiny fraction of a fraction of one per cent of the population of the planet that can afford and and are in the market buying audio products, and they in turn are a fraction of humanity. As I see it, without a proper study I only have anecdote to support the claim.

Toole, F.E., and Olive, S.E. (1994). “Hearing Is Believing vs. Believing Is Hearing: Blind vs. Sighted Listening Tests and Other Interesting Things”

Relevant parts of the paper's conclusions:-

• "... when listeners knew what they were listening to, the opinions were dictated more by the product identity than by the sound"

• "In summary, in listening tests where the audible differences between products were not difficult to hear, knowledge of product identity while listening had profound effects on listener opinions. In some instances, altered listener preferences resulted from listeners being less responsive to audible differences in the sighted tests than they were in the blind tests. For example: (a) they were less responsive to differences caused by loudspeaker location in the room, and (b) they were less responsive to differences associated with program material. Overall, though, it was clear that the psychological factor of simply revealing the identities of the products altered the preference ratings by amounts that were comparable with any physical factor examined in these tests, including the differences between the products themselves. That an effect of this kind should be observed is not remarkable, nor is it unexpected."
• "What is surprising is that the effect is so strong, and that it applies about equally to experienced and inexperienced listeners." (my emphasis)

cheers

 

[Newman]

The "almost everyone's preferences" doesn't have to be the most important thing when you want to know what works best for your personal preferences.

One loudspeaker can have an outstandingly flat frequency response while the distortion is sky-high but some listeners may not be bothered by it at all, while another speaker may have some deviations in the frequency response but has a wider dispersion that perfectly suits another listener. The list of things that can make a listener prefer a certain loudspeaker over another can be many different things, even if most other listeners may prefer another speaker.

The Spinorama picks up your example involving dispersion. Nobody is saying that flat FR is all you need and everything else be damned. As for distortion, how many decent loudspeakers have "a perfectly flat FR and sky-high distortion" when used as intended? Too few to make a fuss about, is my first thought.

I don't think we want to go into the exceptions by extremes argument of your "a perfectly flat FR and sky-high distortion" hypothetical. If we have a perfectly usable guideline that makes a difficult task easier, then its general usefulness does not swing on whether it applies equally well to hypothetical extremes.

Why wouldn't you choose the loudspeaker you preferred in a blind listening test, isn't that the ultimate test to determine what suits your taste no matter the outcome?

You absolutely would, I agree. It's ideal.

And I bet you haven't done it. I bet Joe Blow next to you hasn't done it. I bet practically everyone hasn't done it, for the same reason I know I haven't done it: it's too hard.

That's a major objective of all this research: to find a shortcut that reliably leads us to the same choice of loudspeaker as if we had done blind testing ourselves.

And all the evidence is that it works! Hooray! (Not Boo!)

Here is another thing to ponder: even if you do go and do that rarest of things, ie conduct blind listening tests on yourself to determine which sound waves you actually prefer, your expertise as an experimenter comes into play. If you are a modest individual, your confidence that your blind testing protocol is of the highest standard and has given a true result, should be lower than the confidence level that the Spinorama does indeed apply to you, and you are not one of the exceptions.

So, it's kind of a no-brainer.

Yes, there have been many people who have participated in the blind tests Toole has set up, who didn't fall right in the middle of the preference curve. Their taste in sound did deviate more or less from the average Joe's taste, and one of those people could be me or you, we don't know before we have done the blind testing ourselves.

I think you are making an ambit claim. You hope there are many, you expect there are many, but there are not actually many. It is a surprising result, I know.

I don't think deviations in the taste of sound necessarily have to do with hearing loss. Which listeners have the most problems with their hearing, the fan of the Genelec sound or the Neumann sound?

Hello Sighted Listening Effect!

 

[Galliardist]

Toole, F.E., and Olive, S.E. (1994). “Hearing Is Believing vs. Believing Is Hearing: Blind vs. Sighted Listening Tests and Other Interesting Things”

Relevant parts of the paper's conclusions:-

• "... when listeners knew what they were listening to, the opinions were dictated more by the product identity than by the sound"

• "In summary, in listening tests where the audible differences between products were not difficult to hear, knowledge of product identity while listening had profound effects on listener opinions. In some instances, altered listener preferences resulted from listeners being less responsive to audible differences in the sighted tests than they were in the blind tests. For example: (a) they were less responsive to differences caused by loudspeaker location in the room, and (b) they were less responsive to differences associated with program material. Overall, though, it was clear that the psychological factor of simply revealing the identities of the products altered the preference ratings by amounts that were comparable with any physical factor examined in these tests, including the differences between the products themselves. That an effect of this kind should be observed is not remarkable, nor is it unexpected."
• "What is surprising is that the effect is so strong, and that it applies about equally to experienced and inexperienced listeners." (my emphasis)

cheers

Thanks.

 

[krabapple]

That's why certain individuals who keep on doing exactly that are a kind of white noise to the site. They just think they are posting signal content.

As if there weren't a dozen or two subjectivist audio sh*tholes on the web where they could pleasure themselves that way.

And yet they choose here. Go figure.

 

[MattHooper]

This was a good post by Newman. Quite sensible and I agree with pretty much all of it.
I have many times agreed that choosing loudspeakers in the way Newman describes here makes plenty of sense.

However, the conundrum I keep pointing out, raises its head again…

And I bet you haven't done it. I bet Joe Blow next to you hasn't done it. I bet practically everyone hasn't done it, for the same reason I know I haven't done it: it's too hard.

That's a major objective of all this research: to find a shortcut that reliably leads us to the same choice of loudspeaker as if we had done blind testing ourselves.

And all the evidence is that it works! Hooray!

I would ask: what does that last sentence mean?

What is “ working” and what is the
“ evidence” for this?

Does this claim mean that those who have bought speakers based on good measurements (derived from blind preference ratings) have found the loud speakers to sound good, just as the measurements predicted?

If so, the obvious implication there is that the Characteristics determined via blind testing have also been confirmed in sighted listening! (by the owners of such speakers)

But wait, we aren’t supposed to trust sighted listening, right?

Just last page Newman wrote

“The critical aspect I am hard on, however, is people trying to give more credence than reality dictates to the notion that sighted listening can be insightful to what we think of the sound waves themselves. It simply isn't.”

So how could the sighted listening experience of, say, an owner of a Revel or Genelec speaker possibly be appealed to any form of confirming evidence?

Or is this some broader claim that owners of “ properly measuring loudspeakers” report more satisfaction or more long-term satisfaction, than those who did not buy speakers on such measurements?

If so, I would ask: where is the evidence for that? Let alone that “ all the evidence” is in favour of such a claim.

I haven’t seen any such studies, establishing this evidence.

And anecdotally, as I’ve pointed out here before observing threads in various forms of what gear people own and for how long, doesn’t seem to be any obvious trend that audiophiles choosing equipment based on certain measurements are more satisfied, or satisfied longer, then the more “ subjectivist” crowd.

This does seem to be an underlying assumption in this forum. But if we are going to be making claims about “ all the evidence” pointing towards something, it’s reasonable to ask: evidence for what exactly and where is this evidence?

 

[ahofer]

And yet they choose here. Go figure.

 

[MattHooper]

Having said that, most of what "ASR knows" is, I suspect, still correct.

Agreed. Though again, It’s entirely reasonable to explore the wider implications of what ASR “knows.”

Rather than our hearing always being fooled- we just don't know when our hearing is being fooled.

Agreed.

How are we to think about the results of controlled blind testing for loud speakers? (and I stick with loudspeakers because that is to stay in the realm of differences well-known to be audible).

If the inference is “ Our default should be that our perception is wrong in any sighted conditions” it leads to the type of problems I’ve been getting at. And some here seem to default to something like that “ whatever a subjective reviewer or audiophile reports under sighted conditions is to be assumed bullshit. It cannot be of any use.”

It’s possible to push the scepticism too far.

So it seems to me, and I presume lots of people here as well, that the more reasonable inference is to talk about uncertainty over
“ what we can know“ given the variable of sighted bias. It always has to be acknowledged as a possible confounding factor.

However since we are not omniscient virtually none of our empirical conclusions are Absolutely Certain. So we are always operating within the realm of uncertainty, yet justifying conclusions based on reasonable confidence levels. If there were no way to justify our conclusions through the day, virtually all of which are made without scientific controls, then we’d be mired in irrationality. So we understood the pragmatism of allowing many justified conclusions without meeting scientific confidence levels.

And that is the “ mushy middle” I referenced in my previous post and which we are often operating in which I am exploring.

If I’m cooking and I add more salt, and then the dish taste more salty, I haven’t done so under scientific controls. In that sense, I can’t have a scientific level of confidence that it’s merely a biased effect of knowing I added more salt. On the other hand, adding salt is certainly well known to make something taste saltier. So my perception the dish is saltier tasting is plausible and reasonable. Again, I could be wrong, but few would consider inferences made under such circumstances to be unreasonable and totally unjustified.

Likewise, if I pushed my speakers out much wider so that each of them is now placed beside hard reflective tile in my room, and I perceive the sound as having become brighter or more harsh and the upper frequencies, lacking scientific controls I can’t have a scientific level of confidence in the apparent Sonic change. But certainly it seems reasonable, on a lower confidence level, given some knowledge of acoustics, that what I’m hearing is real. Always of course within the context of “ I could be wrong.”

But, for example, my first posts here were in a thread where I could see somebody describing speakers that were not being properly driven in the bass. Checking, I found a combination of a speaker with very low impedance and a phase shift in the bass region, and amplification with low gain. Yet several people were telling the newcomer that he was imagining it. So that wasn't a great introduction, and it tallies with something I suspect, which is that serious problems in sound will normally be heard.

Yeah, you do get that happening here, a default to “ It’s your imagination” in the context of sided, listening being worthless.

There was a ASR thread in which some were bending backwards to not acknowledge that Herb Reichert clearly identified a colouration in a speaker review that showed up in the measurements. But it had to be all junk and useless because it was a sighted review!

It’s quite possible for somebody to develop some skills as a listener. I’ve worked with quite a number of film mixers, who are absolutely wicked with EQ and other processing. If some of us identify an issue in the mix “ her line sounded a bit muffled” “ His voice sounds too thick or boomy” … the mixers recognize the characteristics as well, directly target that sound with EQ or whatever, and “ boom” the problem is fixed.
This happens constantly throughout any mix, quite a reliable amount of success, not requiring scientific controls and under sighted conditions.

Or you can look at an audio reviewer like Erin from Erin’s audio corner. He seems to have a terrific ear. He listens first, gives a subjective account of the sound, and most of what he points from his sighted listening out shows up in the measurements!

Even someone like stereophiles Michael Fremer, typically disparaged and mocked in places like this, has been surprisingly accurate in many of his loudspeaker reviews.
I’ve gone back and looked at quite a few of his reviews and usually his description of the general tonal character, for instance matches pretty well with the measurements. He says the speaker had a generally flat character in the mids up, but sounded somewhat lean in the bass? There it is in the measurements. He says another speaker it plays quite low and powerful and has a bit of a pear shape slightly exaggerating the mid base, but giving a somewhat rich sound…. There it is in the measurements. he says of another speaker that has a somewhat mellow downward, tilt and frequency response. There is the measurements. And Fremer has a nice facility for describing the consequences. These characteristics have for music played through the speakers. (I’ve heard plenty of speakers that he’s reviewed and his descriptions usually seem very accurate to what I perceive as well).

I listened to some PMC loudspeakers at my pal‘s place, and though they sounded spacious energetic, Clean with good resolution and somewhat forward highs. But I was bothered by a general “ coolness” and lack of richness to the sound, especially in the lower mid range/upper bass. Later, I saw Kal Rubinsons review and his description was bang onto what I heard, including the fact that Cal was bothered by the very same hollow or lack of richness. And…. It was there in the measurements.

Along those lines, my friend who reviews audio gear has me over to hear whatever loudspeakers he has in for review. And he usually just sits me down and after some music ask for my impressions before he tells me what he’s been hearing. And the vast majority of the time are perception and descriptions coincide.

A more recent example, he had some floor standing loud speakers in for review and after about a week of his listening, he had me come over for a listen, didn’t tell me his impressions, but asked me mine.

It took only a short bit of listening for me to turn to him and say “ there’s something wrong here. These things sound way too lean and favouring the high frequency. I closed my eyes while listening, and I swear to God they sounded more like a small monitor than a big floor stander. Like there is no bass. “. He said that was EXACTLY what he was hearing and that he’d been struggling with it the whole week. They were sent off to be measured, and a speaker designer looked at the measurements and said “ Yep, these are going to sound quite lean… you’ve got to move them towards the corners of the room.” And sure enough, That was the ticket. Once they were moved in the directions of the room corners, we got more base in the sound became richer and more balanced. Now they were quite pleasant to listen to.

So all of the above is talking about work done and inferences drawn under sighted conditions. Was there possible bias effects? Absolutely, always possible! But is that always the most plausible explanation? I don’t think so.

Cynical responses like “ luck” or “ a broken clock can be right once a day” would come off as quite hand wavy.

There seems to be some decent justification for saying “ yeah people were hearing real sonic attributes in such examples.” Could be wrong? Yes. But totally unjustified only because it was “ under sided conditions?” No. You don’t have scientific levels of confidence, but there can be some level of justifiable inference making.

Now it seems to be entirely reasonable for any ASR member to say “ Sighted listening reports are simply not good enough for me.
Too many variables. I’m seeking scientific levels of confidence. So I’m going to buy my loudspeakers based on measurements alone, which have scientific support for sound quality.”

But that’s a different thing from claiming that nobody else can have any justification for any Sonic impressions formed under sighted conditions. That’s going to be problematic.

And somebody can say “ well OK at least some of those instances were confirmed with objective measurements! And also if I’m going to set up speakers in my room, I’m going to use room correction I want to see objective evidence for their sound profile! “

Cool.

But…why? What does the objective evidence do for you? You’re going to be listening under sighted conditions, right?
That means, presumably that all your measurements are aiming at achieving a sonic result that you will actually appreciate and correctly perceive when listening to your speakers. Otherwise, what’s the point?
But that of course, is right back to assuming that sighted listening is not simply useless.

This conundrum is going to remain there so long as we are talking about making inferences from the relevance of measurements to whatever results we expect to apprehend and appreciate in our sighted listening.

I'm sure @Newman has to hand the results Toole had for different groups of listeners and how well they performed in blind testing. Time for that to be posted again, it is at least something to chew on.

Yup. Good for Newman for posting it. I had actually reread that study before making my previous post.

It’s just a type of study often cited, from which the issues and conundrums I’m pointing to arise.

 

[MattHooper]

As if there weren't a dozen or two subjectivist audio sh*tholes on the web where they could pleasure themselves that way.

And yet they choose here. Go figure.

Newman is usually taking potshots at me with those type of comments, and I’m not sure if you were including me as well.

If so, I would point out that, even though I may veer too prolix, I have received a great amount of feedback from ASR members who have appreciated the perspective I bring in my posts. I doubt I would’ve accumulated as many “likes” if that weren’t the case.

It’s a pretty big tent here, but those who see my posts as a blight to their eyes can always just skip them or put me on ignore.

Of course, if you weren’t including me in there…as Emily Litella would say ..Never mind…

Cheers

 

[MattHooper]

In another thread, a poster said ..."Unfortunately measurement is very unreliable at predicting what people actually like."
I replied ... "You've got it backwards. What people actually like is fortunately very reliable at predicting measurements."

I think we were both correct.

If you mean the first quote entails “ under sighted conditions” and yours means “ under blind conditions” then I see your point.

When one advocates sighted listening as useful, it begs the question, "How useful?"

Yes, exactly. I think if you are going to suggest that sighted listening can be useful, you want to be able to talk about specific examples. And so that’s what I do.

By the same token, when someone says that sighted listening is useless, it begs the question, "How useless?"

Yes, exactly. Take it too far and things get a bit wacky.

ASR has an enormous undercurrent of subjectivists. There are many people who resent that science is not as "malleable" as they would like, and would like to see the effectiveness of ASR reduced.

I don’t know about that. I don’t see an
“ enormous undercurrent of subjectivists”
reflected in the posting here. So that leaves it as conjecture that there is some great swelling of subjectivist members who aren’t speaking up. But I don’t know how you would demonstrate that. Anecdotally I’ve received plenty of likes and also personal messages of support from lots of the
“ regulars” (which include mods ) who I wouldn’t think fit the
“ subjectivist” description.

And of course it’s also going to depend on what your description of subjectivist is in the first place. Some like to put me in that category, the same as any, you might pluck out of a Steve Hoffman forum or audiogon, which always cracks me up since on those forums the dogmatists there see me as the Objectivist AntChrist.

Please don't take the number of "likes" a person has as a sign of their clarity of thought or of proven authority. I don't.

I don’t. I wasn’t. I wouldn’t.

My point was only that while certain members find my posting unpalatable or even against their idea of this forum, they certainly aren’t speaking for everybody.

As to the quality of my arguments, of course they stand or fall on their own, under scrutiny, and not on “ likes.”

Back to the actual substance of what we’ve been discussing…

 

[NTK]

Hasn't Dr Toole already answered Hooper's question?

omnidirectional loudspeakers = best design available

Thanks Floyd. QED I hope you are joking. (Hard to tell for sure on the interwebs).

www.audiosciencereview.com

 

[MattHooper]

Specific examples are only relevant to one person .... you.

I disagree. First, I should point out that if the examples were only useful to one person, then they are still useful.

But if you look at the examples I gave you can see they are not relevant to only one person. I referenced reviewers who under sighted conditions accurately perceived characteristics shown to be real by measurements (Herb,Erin,Fremer, Kal) which are public demonstrations not just “ to me.”

The proficiency of studio mixers working under sighted conditions, quickly identifying Sonic characteristics and fixing them, is something endless numbers of clients and technicians observe every day, and if you want to see an action, you could probably arrange it.

In the example I gave of the loudspeakers at my friends place that were too lean sounding in his room, that was also identified by other listeners, including another visiting reviewer, and the
Measurements are available to the public
They show why the sighted impressions a number of us had and described were quite plausible, including the soundstage review description.

Further, if the results of sighted listening was only pertinent to any particular reviewer and nothing more, I certainly wouldn’t find anything useful in that. But the whole point is that, ideally, they are able to perceive true characteristics of the loudspeaker and describe those accurately, or give a good impression as to what they sound like.
And I have found plenty of subjective reviews do just that, which has been also very useful for some of my own purchases. That’s the same for any other audiophiles.

In other audiophile forums I have had long thread describing my impressions of many different loudspeakers I’ve auditioned and owned. There are people who read the descriptions and found they corresponded very well to their own impressions of the same speakers, which gave them some level of trust in my descriptions. And this has led to others making successful purchases based on what they read from me. For instance, somebody who owned Harbeth speakers was very curious about Joseph audio speakers and asked if I could compare the two brands. As it happened, I owned the same Harbeth speaker as well as the Joseph audio speakers he was interested in. So I did a detailed comparison of their strengths and weaknesses and overall character difference.
This person ended up ordering the Joseph speakers based on the comparison, and he was happy as a clam declaring that, yes, all my descriptions were bang on. (Quote: “
let’s just say that the Perspectives have met exceeded all my expectations, and then some more…You were spot on with your analysis!”)

So my own experience has proven useful to others over the years, just as others have proven useful to me.

Scientific levels of confidence? Nope. Useful?
In plenty of cases, sure.

This is not something I advocate to anybody here. Anyone here buying purely on measurements…. perfectly reasonable.

But if someone is going to claim that alternative methods based on sighted listening are useless, I’m going to explain why I do not find that to be the case and why I still can get value out of sighted listening descriptions. (not all of course…. But carefully chosen).