Master Thread: Are measurements Everything or Nothing?

[storing]

If a subjective reviewer is tested and it is found their opinions seem to chime with the objective data, then surely their opinions have value, to some extent. The correlation would prove it isn't purely a feat of imagination on their part.

Theoretically in terms of logic this is actually not correct. It's still merely correlation, not causal, in that it can still be imagination but just imagination which happens to confirm the objective data. I also have this impression my current DAC of which I know measures better than the previous one, produces 'better' sound. Until I do a blind listening test of both, I cannot be sure there's really something to hear there, or just imagination.

Since you mention correlation, this one is always fun: https://tylervigen.com/spurious-correlations

 

[alitomr1979]

the problem is: for those guys everything which is not measureable is placebo, thats the real problem here, they are in a massive deny about everything they cant "explain", i bet this has a way greater effect then placebo
also because, atleast me (and i bet many "true" audiophiles too) go to a subjective listening test way more objective than the "science guys" here, because they have a mindset of just denying things that cant be measured right from the start, and/or dont have the right gear to evaluate those things, if i cant hear them i would also believe they dont matter

This is interesting. I have been working in strategic planning for about 14 years now. Not full time but occasionally I work that subject. My experience somehow is similar to what you are saying when it comes to measuring things.

We humans apparently have a tendency to measure (and do) regardless of the significance/value of those measures. This experience that people want to measure something and assume it is, not only of value, but the absolute thing from which we should infer value, is so present everywhere in teams that I even use an analogy in my first presentation to management teams where I say that we as people tend to look for missing keys where it is comfortable, where there is light, where we don’t have to bend too much and there is not too much heat, instead of the dark, hot, dangerous parking lot where it is likely that we lost it. I think that’s some kind of bias we have, because my experience is modest, but reading from the likes of McKinsey and Company, Boston Consulting Group, Bain and Company, they mention this issue as something omniscient.

That is why when we set out to set indicators we need to be really careful and do our best effort, because they tend to become almost religion. This is known in strategic management. And we try to design processes to prevent this serious problem: we need to accept that often there is more than what we can measure or do measure, and be alert and trying to find those things. I see a variation of that here: “all we measure is all there is to know and if you think there’s more you are delusional, stupid, or a combo of both”. LOL.

Terrible. We can do better.

 

[Vintagear73]

They're your words, not mine:



So, which devices have perfect specifications on paper but end up sounding bad?

You really don’t understand the point of this whole story.

I will try to simplify the situation for you.

Based on your looks, you may think you are the best lover in the world, but someone else can confirm or deny this based on practical experience with you.

So theory and practice do not always have to be one.

 

[tifune]

You really don’t understand the point of this whole story.

I will try to simplify the situation for you.

Based on your looks, you may think you are the best lover in the world, but someone else can confirm or deny this based on practical experience with you.

So theory and practice do not always have to be one.

I tried to Google that model # but I got a lot of weird results, do you know who manufactured it?

 

[pkane]

This is interesting. I have been working in strategic planning for about 14 years now. Not full time but occasionally I work that subject. My experience somehow is similar to what you are saying when it comes to measuring things.

We humans apparently have a tendency to measure (and do) regardless of the significance/value of those measures. This experience that people want to measure something and assume it is, not only of value, but the absolute thing from which we should infer value, is so present everywhere in teams that I even use an analogy in my first presentation to management teams where I say that we as people tend to look for missing keys where it is comfortable, where there is light, where we don’t have to bend too much and there is not too much heat, instead of the dark, hot, dangerous parking lot where it is likely that we lost it. I think that’s some kind of bias we have, because my experience is modest, but reading from the likes of McKinsey and Company, Boston Consulting Group, Bain and Company, they mention this issue as something omniscient.

That is why when we set out to set indicators we need to be really careful and do our best effort, because they tend to become almost religion. This is known in strategic management. And we try to design processes to prevent this serious problem: we need to accept that often there is more than what we can measure or do measure, and be alert and trying to find those things. I see a variation of that here: “all we measure is all there is to know and if you think there’s more you are delusional, stupid, or a combo of both”. LOL.

Terrible. We can do better.

You're confusing opinion and science. You're also making some uncalled-for generalizations and accusations about this forum: "all we measure is all there is to know and if you think there’s more you are delusional, stupid, or a combo of both". Really?

In science, measurements are an objective mechanism for determining observable facts. Measurements don't come into existence by someone just declaring that they are valid. This is done through study, experimentation, and controlled, repeatable, falsifiable scientific testing process. Measurements that have excellent correlation to observable facts are used for further analysis, and are always subject to further testing to confirm their validity, but any objections must be supported by evidence. Just because one is ignorant of the process of how and why measurements are used doesn't make them any less valid. If you have specific objections to a measurement or a collection of measurements, present your evidence. Presenting your (overly general) opinion as fact, based on some unknown "strategic planning experience" means absolutely nothing to anyone.

 

[BDWoody]

Yeah. Anyone who don’t agree with you and what you think you know Boggs you down. Very nice.

No, just those who come here to make claims, then when asked to provide anything more than 'because I say so' as a response, basically admit they just can't be bothered because they are too lazy, and would rather just double down on more claims. It was a few posts later where you decided calling one of our members an imbecile was the appropriate way to bridge the divide (we delete that kind of garbage generally). THAT is EXACTLY what boggs us down. People who aren't here in good faith using up as much bandwidth as they can, but who are too lazy to provide any reason to take them seriously.

 

[audio2design]

@alitomr1979

You repeatedly make posts that are totally off base and just shows you don't understand either the science of measurement in audio nor the interpretation of the science which is not to say that there is not truth in what you wrote as it applies to audio, but it assumes audio science is stagnant while having no understanding of it.

In business, often, the wrong measures are used to drive behavior. I had a v.p. who was obsessed with the number of customer engagements and raw value of engagements as a metric. It was a horrible and ultimately failed metric which fortunately I ignored.

Technically a similar situation existed in the 70s with an obsession with a simple THD measurement.

Well that vp was ultimately fired and it's not the 70s any more. As well, unlike in business where it may take years to see the effect of a bad decision, with blind testing I can arrive at highly accurate conclusions much quicker. Being a scientific endeavour I can build on both previous direct field knowledge and related field knowledge. That direct field knowledge provides well founded basis of say what is audible, and general EE and physics knowledge gives me the ability to say quantify cable differences. Put those two together and I know that most cable claims are nonsense. Not all. But most.

Most the value of ASR is just that. It's not about saying what "sounds best". It is about identifying what is BS and what is not.

 

[Deleted member 23982]

This post says far more about you as an individual than I could ever say by refuting your "arguments".

nocebo is a thing too

It is about identifying what is BS and what is not.

while thinking about it i think blindlistening might help with placebo but probably not so much with nocebo

 

[tvrgeek]

Gee, I was hoping in the last three pages, someone might make some suggestions on what we are not measuring or not interpreting in a meaningful way. Sigh, no progress.

There is a lot of value in the last line Audio2 said. I can pretty much believe I would not like the "subjective" sound of those products that are on the right.

 

[audio2design]

Didn't someone bring up Doppler distortion in speakers? I personally feel like that person does that the jury is not totally out on that subject as it applies to a wide swath of speaker drivers. Then again that may be totally a different thread

 

[alitomr1979]

@audio2design No, I am not assuming audio science is stagnant. I understand your points, but I don’t think they respond to the fact that there is reductionism in audio, not in the scientists working in acoustics, I am talking about the measures people use to infer the quality of audio equipment.

@BDWoody you are are either lying if you are referring to the thread you quoted, or just didn’t understand what I wrote. I didn’t claim I was able to tell the difference. I said literally that “I think” I could tell the difference. I did wrong calling an imbecile to anybody who didn’t understand the difference and started attacking. That was my bad. You are also lying when you say I was asked to provide anything. I said I think I can notice and also said I was too busy and lazy to get to do it. The lazy part was mostly a comment, because I actually don’t have proper equipment to make the AB tests. The fact that you are misrepresenting what I said suggest to me that you really don’t like when people disagrees with you, which is sad even though is so common.

@pkane when I say I see something in the forums, and we are discussing gradients of where people stand in certain continuum… it is not a generalization. Specifically, when I say I see a lot of people thinking that “all we measure is all there is to know and if you think there’s more you are delusional, stupid, or a combo of both” is certain people, not a generalization.

It is obvious I appreciate the forums, specially the great effort and contributions @amirm does with his tests. I think this is a great source of information and we have a lot of very knowledgeable people about audio. My participation in this thread has more to do with how we manage our differences than the specifics of audio. Sadly some of us are not very good at it, and some have this terrible urgency to establish others are wrong. Why that is like that is an interest I have. That’s all.

 

[voodooless]

Gee, I was hoping in the last three pages, someone might make some suggestions on what we are not measuring or not interpreting in a meaningful way. Sigh, no progress.

What for? If you make a claim, you provide the evidence. If there is any we can see why, not the other way around.

 

[Vintagear73]

I tried to Google that model # but I got a lot of weird results, do you know who manufactured it?

But I found something on Google to help you.

7 reading strategies you can use to improve your comprehension skills

1. Improve your vocabulary.
2. Come up with questions about the text you are reading.
3. Use context clues.
4. Look for the main idea.
5. Write a summary of what you read.
6. Break up the reading into smaller sections.
7. Pace yourself.

 

[Digby]

Also, and sorry if I'm misread what you are trying to say, but are you arguing that objectivists (even hard objectivists) are dismissing results of controlled listening tests?

I'm not sure, it is difficult to know exactly. There are people that say that a persons ears can deceive them; if you have 5 people who hear a similar thing, they have the same answer - likely they are being deceived; 50 people, the same answer.

If you stick those 50 people in a controlled, double blind test and the outcome is the same, 50 (or close to) heard what they claimed to have heard outside the test, suddenly this is considered valid and useful information?

I dunno, it seems to me the person who says that ears are easy to fool outside of a controlled background, might not have to high an opinion of using listeners within controlled test environments, although they'd rather not admit it.

Of course, it has to be controlled if the data is to have value from a scientific perspective, but it would be interesting to see the relationship between what people think they hear when in an uncontrolled setting and what they think they hear in a controlled one. I think the relationship between the two may be tighter than some would imagine. Just speculation on my part.

Theoretically in terms of logic this is actually not correct. It's still merely correlation, not causal, in that it can still be imagination but just imagination which happens to confirm the objective data. I also have this impression my current DAC of which I know measures better than the previous one, produces 'better' sound. Until I do a blind listening test of both, I cannot be sure there's really something to hear there, or just imagination.

Since you mention correlation, this one is always fun: https://tylervigen.com/spurious-correlations

Right, I suppose in terms of logic, you are correct. However if a number of tests are done over a period of time and similar results are achieved, it becomes less and less likely that is pure luck/imagination on the part of the person taking the tests. Like you said, doesn't make it causal, but the likelihood of chance alone goes down.

There are quite likely people that have 'golden ears', not just in their own opinion, but in that they can discern bad reproduction from good just through listening. Likely not as well as a machine, but better than a high percentage of others.

I think these people probably could be identified by some kind of test, given a number of times over an extended period.

 

[audio2design]

I'm not sure, it is difficult to know exactly. There are people that say that a persons ears can deceive them; if you have 5 people who hear a similar thing, they have the same answer - likely they are being deceived; 50 people, the same answer.

If you stick those 50 people in a controlled, double blind test and the outcome is the same, 50 (or close to) heard what they claimed to have heard outside the test, suddenly this is considered valid and useful information?

This is the difference between evidence and anecdote.

That 50 audiophiles claimed to hear something to me is relatively meaningless as your sample set ONLY includes people who claimed to hear a difference. What about the 1000 that did not? If you picked 50 at random from a wide breadth of audiophiles maybe, just maybe but even limiting it to audiophiles is flawed.

A blind test on the other hand removes all prior claims and makes them meaningless. In fact using people who made prior claims can yield highly specific results, IE can audiophiles who claim cables are audible reliably identify cables.

 

[pkane]

@pkane when I say I see something in the forums, and we are discussing gradients of where people stand in certain continuum… it is not a generalization. Specifically, when I say I see a lot of people thinking that “all we measure is all there is to know and if you think there’s more you are delusional, stupid, or a combo of both” is certain people, not a generalization.

Like most things involving humans, it's a normal/gaussian distribution with a mean and standard deviation. Outliers will always be there, and on both sides of the mean.

 

[audio2design]

@audio2design No, I am not assuming audio science is stagnant. I understand your points, but I don’t think they respond to the fact that there is reductionism in audio, not in the scientists working in acoustics, I am talking about the measures people use to infer the quality of audio equipment.

This is but a small subset of the community you refer to who equates cleanest signal reproduction must sound better to all people.

I would argue this is a much smaller subset than those who equate their preference with "it must be more accurate".

Both are extreme positions and obviously wrong. That is why another aspect of the science is studying preferences.

 

[Digby]

My point is, just because something is an anecdote, doesn't mean it is a product of imagination. If blind tests are conducted and those who claimed to hear something outside them, hear it again inside the test it likely means something.

I'm relating this to speakers, where there are differences, not cables where there aren't.

I refer to people hearing differences where there can be differences, not where any difference can only be the product of imagination.

 

[voodooless]

That 50 audiophiles claimed to hear something to me is relatively meaningless as your sample set ONLY includes people who claimed to hear a difference. What about the 1000 that did not? If you picked 50 at random from a wide breadth of audiophiles maybe, just maybe but even limiting it to audiophiles is flawed.

Exactly. It’s like asking people using homeopathy if it works. Odds are in favor of a resounding “yes”. Is that evidence that it actually works?

 

[Digby]

I didn't mean a test only with the initial 50, but that they hear within a controlled test what they heard outside of it, sample size of, oh, I don't know...say 500.

How is asking people about homeopathy a controlled/blind test?