Master Thread: Are Measurements Everything or Nothing?

[UprightCitizen]

By

So, then name a single case where there was an audible difference that was not explained by measurements.

This task is to open for interpretation to be of use. It can be read two ways, or even three if one one really want to get into the weeds:

1. Audibility was found but the measurements that would explain it has not been done.

2. Audibility was found, and there are measurements that would deem this impossible.

3. Audibility was found and the cause can not be measured.

The way I see it: 1 is very common, not strange at all. 2 demands very rigorous definitions of hearing thresholds and mapping to test scenarios. Comes a long way but one have to be open that we can end up in 1, or that the mapping is not done well enough. Number 3 I do not believe in, but is a common subjectivist confusion.

 

[voodooless]

The way I see it: 1 is very common, not strange at all.

It might be common, but also renders the test totally pointless, as is clear by your Swedish example.

Number 3 I do not believe in

Well good, at least on that we agree then!

 

[UprightCitizen]

It might be common, but also renders the test totally pointless, as is clear by your Swedish example.

Well good, at least on that we agree then!

I don't follow your reasoning. The valid critique of the LTS test is that they cut it short and only did a blind and not a double blind test. Should they have gone the full mile, that test would of course be more useful than any measurements of harmonic distortion, or what have you, when trying to determine absolute audibility.

 

[Yorkshire Mouth]

That's "The Science is Settled" attitude that's so prevalent... And so anti-science.

It really isn't.

Look, from an absolutist point of view, I take your point. There may indeed by some discovery which we've not yet thought of, and we must keep an open mind, in as far as that goes.

But 'science is settled' doesn't contradict that. It says that, based on all the knowledge we currently have, it's settled. And that's as true for audio science as it is for how your car's engine works.

Deciding to have an opinion which is fundamentally opposed to all the science we currently have, in the 'hope' that science may discover something new, and that this new discovery will coincidentally support your current thinking, appears wilfully perverse. And anti-science.

 

[UprightCitizen]

That's "The Science is Settled" attitude that's so prevalent... And so anti-science.

I don't believe I agree with you, but I see a lot of overconfidence out there

 

[boxerfan88]

Interesting & insightful YT video talk. Featuring Andrew Jones.
(warning - it is a very lengthy video)

Video = L1J6-P25NQg

Interesting points to me (some taken from the transcript):

• Measurements are important. Listening is equally important.
• Klippel measurement has it's weaknessess. Klippel is supposed to measure the response of a speaker in any environment and it removes the environment and it does that by processing spherical harmonic stuff. A lot of mathematics. And you can see in every curve pretty much there's a lot of fuzz in the mid-range and that's the room still coming through. It has not fully eliminated the room.
• ABX is stressful and tiring and you're repeating over and over again (I totally agree after trying out a few ABX tests)
• In hi-fi we're always talking about accuracy. Closest approach to the original sound. Great concept. Totally flawed concept. Not one of us in almost any recording we listen to have a a real clue as to what was done to produce the sound on that recording.
• Whatever that target is, speaker measurement is never going to match perfectly. And so it's got to be inaccurate. That inaccuracy gives it a sound character.
• Floor bounce - we live in a world where we're on a floor, right? We don't float in the air. We're not birds. Everybody hears sounds bouncing off the floor. So everything we hear has got floor bounce and we can kind of tune that out. But it does partly explain why speakers sound different when you put them on different height stands. It's one of the factors.

 

[voodooless]

I don't follow your reasoning. The valid critique of the LTS test is that they cut it short and only did a blind and not a double blind test. Should they have gone the full mile, that test would of course be more useful than any measurements of harmonic distortion, or what have you, when trying to determine absolute audibility.

That’s only half of the critique, though. The other half is that they didn’t try to find out what was going on. If the difference was so obvious, it must be something super trivial. For all we know, some setting was missed and some processing was applied.

That's "The Science is Settled" attitude that's so prevalent... And so anti-science.

That’s not the attitude. The attitude is: if you have something, show the evidence! If the evidence is good, we can try to figure out wat is going on, and use that information for future predictions. That is exactly the scientific method!

 

[UprightCitizen]

That’s only half of the critique, though. The other half is that they didn’t try to find out what was going on. If the difference was so obvious, it must be something super trivial. For all we know, some setting was missed and some processing was applied.

I do agree that details regarding methodology is too sparse in this case, but I don't agree with the notion that a controlled listening test is worthless just because you don't have an audio analyzer lying around. As long as they have turned the eq and sub crossover off, all (or at least most) other settings I could think of should be considered a valid use case, and what you write would actually just strengthen my point with all of this: be careful to make blanket statements of 100% proved no-debate absolute inaudibility and have a sh***y attidude about it.

 

[Yuhasz01]

First measurement of significance is your own hearing ability and capacity performed by an audiologist.

 

[voodooless]

I don't agree with the notion that a controlled listening test is worthless just because you don't have an audio analyzer lying around.

It’s not if you know beforehand what is going on. Randomly picking equipment and testing for audibility might be fun, but without further investigation, I think it’s a rather pointless exercise. Without a hypothesis, there is little it learn. Moreover, often this leads to unverified speculation about the possible causes, which in turn get transformed into persistent miss-information and myth.

 

[JiiPee]

Interesting & insightful YT video talk. Featuring Andrew Jones.
(warning - it is a very lengthy video)

Video = L1J6-P25NQg

Interesting points to me (some taken from the transcript):

• Measurements are important. Listening is equally important.
• Klippel measurement has it's weaknessess. Klippel is supposed to measure the response of a speaker in any environment and it removes the environment and it does that by processing spherical harmonic stuff. A lot of mathematics. And you can see in every curve pretty much there's a lot of fuzz in the mid-range and that's the room still coming through. It has not fully eliminated the room.
• ABX is stressful and tiring and you're repeating over and over again (I totally agree after trying out a few ABX tests)
• In hi-fi we're always talking about accuracy. Closest approach to the original sound. Great concept. Totally flawed concept. Not one of us in almost any recording we listen to have a a real clue as to what was done to produce the sound on that recording.
• Whatever that target is, speaker measurement is never going to match perfectly. And so it's got to be inaccurate. That inaccuracy gives it a sound character.
• Floor bounce - we live in a world where we're on a floor, right? We don't float in the air. We're not birds. Everybody hears sounds bouncing off the floor. So everything we hear has got floor bounce and we can kind of tune that out. But it does partly explain why speakers sound different when you put them on different height stands. It's one of the factors.

Sounds like an audiophile salesman trying to belittle objective measuring of equipment in order to create more room for subjective BS based sales talk.

About Klippel not being perfect - Nothing is perfect, but somebody please point out a better practical way of finding out actual speaker performance.

About ABX being stressful - I guess taking a lie detector can be stressful to some...

About accuracy - Trying to refer to recording process is totally lame. HiFi is about faithful reproduction of the *result of the recording process* regardless of what was done during that process.

About speaker measurements not going to match perfectly - I'm not sure exactly what he means by this, but again, just because something is not perfect, does not mean that it could not be better than other available methods.

About floor bounce - Seriously, is this meant to be some kind of big revelation? Yes, room (floor included) acoustics is a significant factor. So what?

 

[IPunchCholla]

About ABX being stressful - I guess taking a lie detector can be stressful to some...

I don’t find it stressful, but I do find ABX tiring. The funny thing is people want to succeed at tests. They (meaning me) get upset about test results. On the Klipple distortion test on only got -42 dB on pure tones and -15 on “fast car”. I could only get -15 if I was fast switching frequently and really trying to listen to only one particular part of the spectrum. I hate that song now. And I was disappointed about my scores. Even though logically, I shoukd be relieved as that means I’m less sensitive to distortion. Why would I want to be sensitive to that?

So I can’t say I like doing it, but having heard huge differences when there can technically be no difference, it’s still the only way to know for sure if a difference is audible to me.

 

[ahofer]

I don’t find it stressful, but I do find ABX tiring. The funny thing is people want to succeed at tests. They (meaning me) get upset about test results. On the Klipple distortion test on only got -42 dB on pure tones and -15 on “fast car”. I could only get -15 if I was fast switching frequently and really trying to listen to only one particular part of the spectrum. I hate that song now. And I was disappointed about my scores. Even though logically, I shoukd be relieved as that means I’m less sensitive to distortion. Why would I want to be sensitive to that?

So I can’t say I like doing it, but having heard huge differences when there can technically be no difference, it’s still the only way to know for sure if a difference is audible to me.

As Popper would say, if you can’t falsify it, it’s faith.

 

[UprightCitizen]

It’s not if you know beforehand what is going on. Randomly picking equipment and testing for audibility might be fun, but without further investigation, I think it’s a rather pointless exercise. Without a hypothesis, there is little it learn. Moreover, often this leads to unverified speculation about the possible causes, which in turn get transformed into persistent miss-information and myth.

Well that depends. Person X is strict about having a transparent signal chain in their HIFI system. Person X finds a controlled listening test that says the device was not fully transparent when used the way Person X would have used it. Person X does not work at the R&D department of the company that made the device and decide that it's better to look elsewhere. Person X valued the knowledge that was gained reading about the test.

 

[JustJones]

Doesn't do person Y and Z a lick of good. If this " controlled test" ever finds its way from behind a paywall where it can be observed in the light of day then it might provide something worthwhile. Until then I'm sure everyone is thrilled for person X.

 

[UprightCitizen]

Doesn't do person Y and Z a lick of good. If this " controlled test" ever finds its way from behind a paywall where it can be observed in the light of day then it might provide something worthwhile. Until then I'm sure everyone is thrilled for person X.

I'm not discussing a particular test anymore, but the general filosophy. Anyways, I think I'm done now, it's starting to become a distraction.

 

[JustJones]

I'm not discussing a particular test anymore, but the general filosophy. Anyways, I think I'm done now, it's starting to become a distraction.

I thought distraction was the point? We're told about a controlled double blind test that wasn't completed because it was apparent the Ultra isn't up to snuff but we don't really know much about this test other than knowledgeable people did it. From there we're off to the races. Has all the earmarks of distraction.
So back to the Ultra from all the measurements I've seen it looks audibly transparent to homosapiens.

 

[UprightCitizen]

I thought distraction was the point? We're told about a controlled double blind test that wasn't completed because it was apparent the Ultra isn't up to snuff but we don't really know much about this test other than knowledgeable people did it. From there we're off to the races. Has all the earmarks of distraction.
So back to the Ultra from all the measurements I've seen it looks audibly transparent to homosapiens.

What I meant was, it started to become distracting having my phone buzz with notifications I was just tired of the snarky attitudes I see, so I teased that this test existed. A naive thought that it might introduce at least the tiniest of doubt with at least one or two. And I'm sure there are a couple LTS members here, they might find it an interesting read. Or not. If in the end the said test reflect reality or not is not really the point.

And I agree that from all test results I have seen, it looks to my best knowledge like it should be audibly transparent.

 

[antcollinet]

This task is to open for interpretation to be of use. It can be read two ways, or even three if one one really want to get into the weeds:

1. Audibility was found but the measurements that would explain it has not been done.

2. Audibility was found, and there are measurements that would deem this impossible.

3. Audibility was found and the cause can not be measured.

The way I see it: 1 is very common, not strange at all. 2 demands very rigorous definitions of hearing thresholds and mapping to test scenarios. Comes a long way but one have to be open that we can end up in 1, or that the mapping is not done well enough. Number 3 I do not believe in, but is a common subjectivist confusion.

What you were asked for is an example of 2: Do you have one?


Specifically - a test where it has been demonstrated that there is an audible difference between two devices which both meet the strict limits defined here.

F

'Audibility thresholds of amp and DAC measurements'

Dec 17, 2018

Introduction
A recurring theme on ASR is whether or not the various measured qualities of the devices are audible. In this post, I'll present some clear and visual thresholds for when those imperfections can be considered a potentially audible concern. I will not explain the basics of amp/DAC measurements.
Feel free to comment if you have any issues with the content of this post, be it technical, grammatical, or maybe just something that's poorly communicated.

Orange marks strict limits while green marks lenient limits.
Strict thresholds guarantee that anything...

• flipflop
• Replies: 989
• Forum: Psychoacoustics: Science of How We Hear

• 

TBF - Id even be impressed if you were able to come up with a test that showed audibility of difference between devices that only meet the lenient limits.

 

[Yorkshire Mouth]

Not quite as perverse as the mindset that denies that people can HEAR differences in equipment...

You were talking about science. There's a very simple way to discover whether or not people can tell the difference between two DACs which are both measured as transparent. It's been conducted. They can't. If you disagree and just say "I don't care, I can hear a difference", then we've just discovered where the anti-science is located.