Master Thread: Are measurements Everything or Nothing?

[SIY]

Agreed. I assumed that was covered in my saying:

“To that end, it seems to me we would allow any claims to be made about gear, but those claims will be subjected to scrutiny informed by engineering knowledge and a more scientific mindset.”

Mostly, scientists have zero time or patience for cranks. Every one of us has received single-spaced screeds explaining with handwaves why relativity/quantum theory/whatever is incorrect with pleas of, "I can't get this work published because my concepts are revolutionary but I need help putting this into math terms." They get treated with rapid trips to the round file. If one of these cranks makes their way into a scientific meeting or forum, they're quickly escorted out. They deserve no consideration, respect, or response whatsoever.

 

[MattHooper]

Mostly, scientists have zero time or patience for cranks. Every one of us has received single-spaced screeds explaining with handwaves why relativity/quantum theory/whatever is incorrect with pleas of, "I can't get this work published because my concepts are revolutionary but I need help putting this into math terms." They get treated with rapid trips to the round file. If one of these cranks makes their way into a scientific meeting or forum, they're quickly escorted out. They deserve no consideration, respect, or response whatsoever.

Indeed, I’m certainly assuming A level of discretion and not just snapping at every troll sounding remark from a drive-by subjectivist or something. One can always decide where one puts their efforts. I think some claims by subjectivists here may warrant more attention than others. Cheers.

 

[SIY]

Indeed, I’m certainly assuming A level of discretion and not just snapping at every troll sounding remark from a drive-by subjectivist or something. One can always decide where one puts their efforts. I think some claims by subjectivists here may warrant more attention than others. Cheers.

My first question is always a simple, "Evidence?" When invariably such is not forthcoming but there's a wall of handwaving text, my terse dismissal is, "OK, so no evidence." When there's a sincere desire to learn something, that's a different matter, but being lectured to by confident ignoramuses is not a productive use of forum space.

 

[MattHooper]

Surely that depends on the nature of the upmixing? If it follows a defined process from science/research to generate, say, an improved stereo image, is it still "pleasing yourself"?

Yes, that’s a good point: in principle, you could set up surround upmixing that would be indistinguishable from stereo. In fact, I’ve done that myself, in experiments in my system.

But that’s generally not the type of upmixing I’m talking about, nor I think the type that is very common, nor I think, even the type that someone like Floyd Toole has extolled. (Floyd tool has written in the past that he upmixes most of his stereo content to surround to enjoy the added spaciousness. Auro-3D)

In most cases at the very least upmixing is used to extract ambiance and reverb cues to the surround speakers, creating a more spacious and enveloping presentation.

It’s the very departure from the limitations of stereo - the way in which it sounds different than simply listening to normal stereo - that is usually extolled as the benefit. And to the degree it IMPROVES the immersion, gives you obviously greater immersion and spaciousness, that speaks to the departure from accuracy to the original stereo signal and the colouration involved.

And then of course it’s also pretty common for up mixing matrixes to extract discrete sounds to place in the surrounds as well.
I actually often use such up mixing for stereo to surround. Especially for electronic music I quite enjoy the effect.

 

[Galliardist]

Fair suck of the sauce bottle, mate! Can you distill this barrage of questions into something more manageable please.

cheers

I'll put the sauce bottle away and try again.

I think the underlying questions are:
Is there a tiny percentage of the population whose preference in controlled tests for speakers and electronics vary from the accurate: and if so, are they actually a very large number of individuals? How does a science oriented site, dealing with preference based experimental science, actually cope with such a group?

and

What is the state of science around non-sonic preference in uncontrolled sighted testing? Is it worth examining that preference?

 

[MattHooper]

What is the state of science around non-sonic preference in uncontrolled sighted testing? Is it worth examining that preference?

That reminds me of the question I have asked in various ways, the sort of elephant in the room:

To what degree do blind listening preferences predict listener preferences and satisfaction scores in sighted listening conditions?

If there is no good correlation, one has to wonder what companies like Harman Kardon were doing wasting all that time, money and resources designing loudspeakers via blind listening preferences, given their products are going to be listened to in sighted conditions.

 

[MattHooper]

Perhaps you have asked the the right question the wrong way. A better way might be ...

"If listener preferences and satisfaction in sighted listening conditions are valid, then how is it possible that people give different answers in blind tests?"

That’s the usual way sighted versus blind listing is discussed.

I have flipped the question on its head to look at the type of implications that people don’t seem to get in to normally.

Think about what it means to actually answer the question as I formed it.

People are subject to many different factors that they use to form a list of priorities. Appearance is one priority. Performance is another. I believe that performance is a higher priority than appearance, even though I know appearance is a powerful factor.

But the issue I’m getting at is hidden right in what you’ve written. What good is
“ performance” that you would never perceive?

Which brings us right back to the central question as I formed it: when you buy a loudspeaker that was designed based on blind listening preferences, are you going to perceive that better performance in the conditions you’ll actually be using it, sighted?

Seems like an incredibly obvious question, and it’s fascinating that I rarely see it asked or investigated in an overt way.

It’s actually generally assumed that the answer is going to be “ yes.” That’s why you find loudspeakers with the right measurements being recommended here all the time.

The implications of that are interesting though…

 

[Newman]

My first question is always a simple, "Evidence?" When invariably such is not forthcoming but there's a wall of handwaving text, my terse dismissal is, "OK, so no evidence."

Some people think that their sighted listening experience is "Evidence". They think this is a 'gotcha' reply to the audio science proponent, because the connection between sighted listening and sound waves is obviously not zero. So they see it as a grey zone that they can take advantage of.

It takes a bit of skill to spot when this type of deceptive argumentation is happening. A skilled writer can make it look like they are doing it in good faith when in fact they are doing it in bad faith. There are such people on ASR. Eventually, as you say, 'terse dismissal' is the right course of action, even though it makes you look like the bad guy (another win for the bad-faith writers with skill, which they will not hesitate to take advantage of).

The science is clear, it is not unclear: the sighted listening effect is (1) stronger and more pervasive than almost anyone would believe when they look back on their own sighted listening experiences, and (2) utterly unreliable to the point that, even if one were to 'calibrate' oneself with sighted vs blinded listening, one cannot trust that calibration to hold for even one more day. It might hold for a while, it might hold for a long while, but it might not. And there is only one way to be sure.... LOL.

"Ok, so no evidence." Like you say!

cheers

 

[MattHooper]

Some people think that their sighted listening experience is "Evidence". They think this is a 'gotcha' reply to the audio science proponent, because the connection between sighted listening and sound waves is obviously not zero. So they see it as a grey zone that they can take advantage of.

It takes a bit of skill to spot when this type of deceptive argumentation is happening. A skilled writer can make it look like they are doing it in good faith when in fact they are doing it in bad faith. There are such people on ASR. Eventually, as you say, 'terse dismissal' is the right course of action, even though it makes you look like the bad guy (another win for the bad-faith writers with skill, which they will not hesitate to take advantage of).

The science is clear, it is not unclear: the sighted listening effect is (1) stronger and more pervasive than almost anyone would believe when they look back on their own sighted listening experiences, and (2) utterly unreliable to the point that, even if one were to 'calibrate' oneself with sighted vs blinded listening, one cannot trust that calibration to hold for even one more day. It might hold for a while, it might hold for a long while, but it might not. And there is only one way to be sure.... LOL.

"Ok, so no evidence." Like you say!

cheers

More paranoia and strawman, which is nothing new. (and you long ago lost the highroad from which to accuse anyone else of bad faith. The examples of your misrepresentations and strawman could fill a separate thread).

But then, you have

STRUGGLED

With these questions

BEFORE

 

[MattHooper]

I think you were typing a bit fast.

Dictated. See my signature line for the apology.

The answer is, "Yes." Please refer back to my example of the cars. Will the sighted bias (which I admitted is powerful) in favor of the show car with no motor overcome the realization that the drab car satisfies a much higher priority? I don't believe so.

OK, thanks for the answer. That’s my suspicion too.

People aren't as dumb as they are sometimes made out to be. If they were, ASR wouldn't have between one and three thousand visitors, every hour of every day, all year long. I don't believe for a second that the majority - or even a significant minority - of those visitors are in unbreakable thrall to sighted bias.

Interesting. Newman’s post suggests perhaps you are being naïve:

The science is clear, it is not unclear: the sighted listening effect is (1) stronger and more pervasive than almost anyone would believe when they look back on their own sighted listening experiences, and (2) utterly unreliable to the point that, even if one were to 'calibrate' oneself with sighted vs blinded listening, one cannot trust that calibration to hold for even one more day. It might hold for a while, it might hold for a long while, but it might not. And there is only one way to be sure.... LOL.

I hope you aren’t a subjectivist in hiding Jim! If so, prepare for an inquisition.

 

[Old_School_Brad]

All organizations benefit from being regularly pressure tested - not only to maintain focus, but also because, one day, something new might emerge. It's important to create space for this new idea to be examined and explored, even if it doesn't entirely align with our current understanding.

That's why I believe everyone should have the opportunity to express their thoughts and beliefs.

 

[Blumlein 88]

Some people think that their sighted listening experience is "Evidence". They think this is a 'gotcha' reply to the audio science proponent, because the connection between sighted listening and sound waves is obviously not zero. So they see it as a grey zone that they can take advantage of.

It takes a bit of skill to spot when this type of deceptive argumentation is happening. A skilled writer can make it look like they are doing it in good faith when in fact they are doing it in bad faith. There are such people on ASR. Eventually, as you say, 'terse dismissal' is the right course of action, even though it makes you look like the bad guy (another win for the bad-faith writers with skill, which they will not hesitate to take advantage of).

The science is clear, it is not unclear: the sighted listening effect is (1) stronger and more pervasive than almost anyone would believe when they look back on their own sighted listening experiences, and (2) utterly unreliable to the point that, even if one were to 'calibrate' oneself with sighted vs blinded listening, one cannot trust that calibration to hold for even one more day. It might hold for a while, it might hold for a long while, but it might not. And there is only one way to be sure.... LOL.

"Ok, so no evidence." Like you say!

cheers

I agree with you.

But this music thing is a hedonistic experience. So prone to sighted bias, and stories of gear. In day to day listening you always listen sighted. So one can do the best to make choices on rational basis. On a scientific basis. So hard however not to be influenced a little with how something looks, how it is styled and even how it is reputed to work. To have a bit of FOMO when so many are so sure something special is going on. And if it works, if it gives you a different experience due to that bias, one that you enjoy a little more, then how can you be too hard on that? Sometimes people are wrong for all the right reasons and really enjoy something extra because of it. It really is a completely different category of experience.

 

[Newman]

That's right, I can't be too hard on that. I have written numerous posts, probably dozens, to that exact point. Saying much as you say.

The critical aspect I am hard on, however, is people trying to give more credence than reality dictates to the notion that sighted listening can be insightful to what we think of the sound waves themselves. It simply isn't.

 

[Galliardist]

Some people think that their sighted listening experience is "Evidence". They think this is a 'gotcha' reply to the audio science proponent, because the connection between sighted listening and sound waves is obviously not zero. So they see it as a grey zone that they can take advantage of.

It takes a bit of skill to spot when this type of deceptive argumentation is happening. A skilled writer can make it look like they are doing it in good faith when in fact they are doing it in bad faith. There are such people on ASR. Eventually, as you say, 'terse dismissal' is the right course of action, even though it makes you look like the bad guy (another win for the bad-faith writers with skill, which they will not hesitate to take advantage of).

The science is clear, it is not unclear: the sighted listening effect is (1) stronger and more pervasive than almost anyone would believe when they look back on their own sighted listening experiences, and (2) utterly unreliable to the point that, even if one were to 'calibrate' oneself with sighted vs blinded listening, one cannot trust that calibration to hold for even one more day. It might hold for a while, it might hold for a long while, but it might not. And there is only one way to be sure.... LOL.

"Ok, so no evidence." Like you say!

cheers

If the science is clear, I presume you can cite it? I have never seen a convincing reason for why the sighted listening effect is so pervasive, or indeed evidence as to how pervasive it is.

 

[Newman]

If the science is clear, I presume you can cite it? I have never seen a convincing reason for why the sighted listening effect is so pervasive, or indeed evidence as to how pervasive it is.

I think I know what you are getting at, but I think you are going to struggle to find experimental evidence directed at calibrating sighted listening. It is akin to asking a scientist to calibrate imagination.

The reasons why the sighted listening effect is so pervasive, are the same reasons why illusionists can be so confounding: we are hard-wired to believe that sensory cognition is 'direct', but it isn't.

The same reasons why trance states exist.

The same reasons why, with a little directed suggestion, one can bite into an onion and actually taste apple. (Now apply this to the sound of a loudspeaker after 'a little directed suggestion' regarding its engineering, or componentry, or legacy/backstory, or the opinions of people you respect.)

The scientific reason for it lies deep in the science of human sensory processing and cognitive processing. Even though I am a very shallow wader in that pool of science, it soon becomes obvious that we are not actually directly observing the objective world through our senses at all... but that is exactly what we are hard-wired to believe in, and such hard-wiring is critical to our survival as individuals and as a species, and that is exactly why it is so difficult to persuade anyone who hasn't looked into these things that the sighted listening effect via cognitive distortions is as strong as it is. And how individually random it is (with a few cultural norms of commonality, such as bias in favour of size, weight or price, etc).

Floyd Toole has touched on a few of these 'common cultural norm' biases, in his books where he mentions blind vs sighted listening test outcomes, for speakers that sound distinctly different, swinging wildly with attributes like size, price, and country of origin. But that hardly counts as the sort of calibration that you asked for. Calibrating cognitive misattribution bears similarity to calibrating imagination.

cheers

 

[goat76]

Even if you base your loudspeaker choice only on looking at the measurements, that is also a sort of sighted evaluation. Who knows, maybe you would have choosen another speaker with worse measurements if the evaluation were made in a blind listening test.

So, how many of you have made the evaluation of your choice of speakers listening blind?

 

[goat76]

I commented on the traffic to this site ... thousands of visitors every hour. What's your personal opinion ... why do you think they come here? What are they looking for?
I'd be interested in your viewpoint.

Besides the great measurements made by Amir, I think people come here for the fairly open discussions that taken place as in this thread. If the discussion where less open, I believe the traffic of visitors would go down.

 

[Newman]

Even if you base your loudspeaker choice only on looking at the measurements, that is also a sort of sighted evaluation. Who knows, maybe you would have choosen another speaker with worse measurements if the evaluation were made in a blind listening test.

So, how many of you have made the evaluation of your choice of speakers listening blind?

I think the whole point of the science behind the spinorama measurements, is to identify which measurements are highly aligned with almost everyone's preferences, and how to interpret the measurements for best alignment.

You can't just say 'who knows, maybe it isn't': that is a bit like hand-waving away all the work (and evidence) that went into this.

If you are saying, "maybe I am an exception", then the work has established that probability to be very high if you have significant hearing damage beyond normal age-related decline, and the probability is very low if you don't.

So, if one's hearing is okay-ish, one can proceed with confidence to correlate measurements to subjective excellence in the sound waves themselves.

cheers

 

[Galliardist]

I think I know what you are getting at, but I think you are going to struggle to find experimental evidence directed at calibrating sighted listening. It is akin to asking a scientist to calibrate imagination.

The reasons why the sighted listening effect is so pervasive, are the same reasons why illusionists can be so confounding: we are hard-wired to believe that sensory cognition is 'direct', but it isn't.

The same reasons why trance states exist.

The same reasons why, with a little directed suggestion, one can bite into an onion and actually taste apple. (Now apply this to the sound of a loudspeaker after 'a little directed suggestion' regarding its engineering, or componentry, or legacy/backstory, or the opinions of people you respect.)

The scientific reason for it lies deep in the science of human sensory processing and cognitive processing. Even though I am a very shallow wader in that pool of science, it soon becomes obvious that we are not actually directly observing the objective world through our senses at all... but that is exactly what we are hard-wired to believe in, and such hard-wiring is critical to our survival as individuals and as a species, and that is exactly why it is so difficult to persuade anyone who hasn't looked into these things that the sighted listening effect via cognitive distortions is as strong as it is. And how individually random it is (with a few cultural norms of commonality, such as bias in favour of size, weight or price, etc).

Floyd Toole has touched on a few of these 'common cultural norm' biases, in his books where he mentions blind vs sighted listening test outcomes, for speakers that sound distinctly different, swinging wildly with attributes like size, price, and country of origin. But that hardly counts as the sort of calibration that you asked for. Calibrating cognitive misattribution bears similarity to calibrating imagination.

cheers

Does this work as an argument?

Isn't it just replacing one intangible, "there is something in the sound that isn't being measured" with another, "there is something in your head that isn't being measured".

Let's step back a bit, and drop the "why". Is there a study or proper citable evidence that simply demonstrates, going back to your earlier post, that the sighted listening effect is

more pervasive than almost anyone would believe when they look back on their own sighted listening experiences

Note, I'm letting you off "stronger" and "calibrate" with this version of the question.

Consider again the raw numbers of people. All the people on forums and working subjectively in the audio industry, all those people who come here with those unusual claims, are a tiny fraction of a fraction of one per cent of the population of the planet that can afford and and are in the market buying audio products, and they in turn are a fraction of humanity. As I see it, without a proper study I only have anecdote to support the claim.

 

[ahofer]

Of course *knowing* that our equipment has been selected for transparency and/or fidelity to the linear on axis/even dispersion standard also creates a positive bias in our listening experience.