• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Catalogue of blind tests

Wes

Major Contributor
Forum Donor
Joined
Dec 5, 2019
Messages
3,843
Likes
3,788
when I drive across a long, narrow bridge I exercise a belief that engineers have learned about transient analysis after "Galloping Gertie"
 

PaulD

Senior Member
Joined
Jul 22, 2018
Messages
453
Likes
1,341
Location
Other
People innately trust daily experience over scientific data - even the scientists. .

That statement is absolutely untrue. Maybe it is true for you, but it is not for me, and not for the vast majority of the people I interact with, ESPECIALLY the scientists.
 

Here2Learn

Active Member
Joined
Jan 7, 2020
Messages
112
Likes
113
That statement is absolutely untrue. Maybe it is true for you, but it is not for me, and not for the vast majority of the people I interact with, ESPECIALLY the scientists.

Perhaps you misunderstand the context of everyday life being the generalization in my post? I am sure that the vast majority of scientists undertake completely objective scientific work, but that does not mean the same scientific rigor applies in how they live everyday life outside of work.

Do you objectively know your car will get you home, or trust it will not break down? (even though you might mitigate risk through regular servicing)

When/if you married, did you perform a polygraph and/or functional MRI on your future spouse's brain to have empirical evidence that the 'love' was there, or was it taken on trust?

Last time you got an a flight, did you ensure the aircraft had been serviced and checked for metal fatigue, or did you trust somebody or some system should be doing such things and it will probably be OK because you trust the statistic that indicates less likelihood of dying in a plane crash than a car crash? (Again, if you got on the plane, you likely had no empirical/objective proof it was safe for flight and exercised a cognitive bias to believe you won't be in the small statistics of persons killed in plane crashes).

Unless you are obtaining the empirical evidence for every daily decision you are faced with making, you inevitably must be exercising some element of trust/faith in some pre-existing belief about others or some societal system. If you do not exercise such, then in that moment, while you are unable to examine or validate any empirical evidence useful to understand the specific situational context, you would be incapable of making a decision leading to an action.

My other point was we can think we're more objective than most, but then that just shows (when we look at the real evidence) that we all lie to ourselves.

The point is everyday life is based mostly on some kind of trust in people and systems (with a stronger bias to trust if we believe those people or systems have agreeably objective science behind them) - we are hard-wired for it. I don't want to derail the thread completely, but there is an abundance of social and evolutionary psychology evidence to re-iterate the point if one chooses to look. Or, you could take it on trust! ;)

But to be completely fair, I have to accept that however the above is informed from my reading of psychology papers (on a casual interest basis), it does lend itself to being a refutable hypothesis. So I am happy to consider proofs we're not exercising trust/faith in everyday decisions and that somehow everything a scientist does in life is always objective and empirically based, which is what a refutation of the above requires.

To make things easier, if we can find one scientist who supports the republican party and another who supports the democrats, and both claim to be objective evidence-led individuals - who is right, and mustn't both be right to refute my point, otherwise at least one of them is trusting something he/she shouldn't be?
 

PaulD

Senior Member
Joined
Jul 22, 2018
Messages
453
Likes
1,341
Location
Other
Perhaps you misunderstand the context of everyday life being the generalization in my post? I am sure that the vast majority of scientists undertake completely objective scientific work, but that does not mean the same scientific rigor applies in how they live everyday life outside of work.

Not at all. I did not misunderstand anything or the generalisations you are making. EVERY scientist I know (many) apply the same principles to their everyday lives. :facepalm:
 

Here2Learn

Active Member
Joined
Jan 7, 2020
Messages
112
Likes
113
Not at all. I did not misunderstand anything or the generalisations you are making. EVERY scientist I know (many) apply the same principles to their everyday lives. :facepalm:

So you are saying they use empirical data at EVERY opportunity in their personal lives, and don't make ANY decisions when there is no evidence to hand? Like for the examples I gave?

EDIT: Actually, I've thought about your concern where I said "People innately trust daily experience over scientific data - even the scientists. ". Taken on its own, it can sound like I meant scientists would trust feeling over data that was to hand. Actually, what I meant was that in daily life (like the other examples in my second post) we often are without such data and go on to act based on prior knowledge/experience/trust/belief, with this representing a large amount of the decision-making we do each day. I was also trying to convey that in such situations, scientists are NOT going to run a series of experiments to yield objective data for everyday casual decision-making, so in that respect, scientists also act as non-scientists do.
 
Last edited:

PaulD

Senior Member
Joined
Jul 22, 2018
Messages
453
Likes
1,341
Location
Other
So you are saying they use empirical data at EVERY opportunity in their personal lives, and don't make ANY decisions when there is no evidence to hand? Like for the examples I gave?
Fundamentally, yes.
 

Here2Learn

Active Member
Joined
Jan 7, 2020
Messages
112
Likes
113
Fundamentally, yes.

I might believe you if you answered the hypothetical questions as you'd approach them with what you think is 'objectivity'. (I'm aware of the psychology definition of 'objectivity': the tendency to base judgments and interpretations on external data rather than on subjective factors, such as personal feelings, beliefs, and experiences ).

Don't misunderstand. I have no issue with anybody claiming to be mostly objective even in everyday decision making. I would only have doubts if somebody claimed absolute objectivity for every decision they make, and never make decisions where there is no data or incomplete data. But that appears to be your claim in answering "Fundamentally, yes".

So if you or your colleagues are married, how was that achieved objectively and which subjective personal feelings, beliefs and experiences were dismissed? I'm sure your wife and/or theirs want to know too.

How does one wholly objectively choose which music to listen to? How is one's choice of audio system wholly objective and devoid of 'personal' biases caused by 'needs' or desired feature sets? How does one live life completely devoid of personal needs, desires and the subjective feelings they cause - is it by a pious life of denial and repression to stick to objectivity at all costs? (We know in psychology how that pans out). If so, how can one even own an audio system that by its very nature is meant to provide subjective enjoyment? You also claimed in another thread "You could stick a kebab in my ear at 20KHz and I'd not notice" - evidence based claim?

Sorry, your claim to total and absolute evidence-based decision making is ridiculous. I don't need a reply - I would believe you're trolling. I'm done too, in case I may also seem a troll.
 

SIY

Grand Contributor
Technical Expert
Joined
Apr 6, 2018
Messages
10,386
Likes
24,749
Location
Alfred, NY
I might believe you if you answered the hypothetical questions as you'd approach them with what you think is 'objectivity'. (I'm aware of the psychology definition of 'objectivity': the tendency to base judgments and interpretations on external data rather than on subjective factors, such as personal feelings, beliefs, and experiences ).

Don't misunderstand. I have no issue with anybody claiming to be mostly objective even in everyday decision making. I would only have doubts if somebody claimed absolute objectivity for every decision they make, and never make decisions where there is no data or incomplete data. But that appears to be your claim in answering "Fundamentally, yes".

So if you or your colleagues are married, how was that achieved objectively and which subjective personal feelings, beliefs and experiences were dismissed? I'm sure your wife and/or theirs want to know too.

How does one wholly objectively choose which music to listen to? How is one's choice of audio system wholly objective and devoid of 'personal' biases caused by 'needs' or desired feature sets? How does one live life completely devoid of personal needs, desires and the subjective feelings they cause - is it by a pious life of denial and repression to stick to objectivity at all costs? (We know in psychology how that pans out). If so, how can one even own an audio system that by its very nature is meant to provide subjective enjoyment? You also claimed in another thread "You could stick a kebab in my ear at 20KHz and I'd not notice" - evidence based claim?

Sorry, your claim to total and absolute evidence-based decision making is ridiculous. I don't need a reply - I would believe you're trolling. I'm done too, in case I may also seem a troll.

Your logical error is going from, "Not all questions are scientific and have testable evidence-based answers" to "...therefore, no questions can be scientific and have testable evidence-based answers."
 

Here2Learn

Active Member
Joined
Jan 7, 2020
Messages
112
Likes
113
Your logical error is going from, "Not all questions are scientific and have testable evidence-based answers" to "...therefore, no questions can be scientific and have testable evidence-based answers."

You have used quotes and I have said neither thing. Certainly, if that is your inference, it was not my intention to convey it.
 

SIY

Grand Contributor
Technical Expert
Joined
Apr 6, 2018
Messages
10,386
Likes
24,749
Location
Alfred, NY
You have used quotes and I have said neither thing. Certainly, if that is your inference, it was not my intention to convey it.

Really? Then why the analogies to "personal feelings, beliefs and experiences"?
 

Here2Learn

Active Member
Joined
Jan 7, 2020
Messages
112
Likes
113
Really? Then why the analogies to "personal feelings, beliefs and experiences"?

A few things...

Paul is intimating that 100% of all his decision making (which would include things like which toothpaste to buy, what food, flavour ice cream, music to listen to etc) are evidence-based decisions. He would never have a "I'm thirsty and fancy a coke" moment, but a "I need 279ml of hydration with X amount of electrolytes" moment if it was all based on objective test evidence.

Do you think such daily things are decided wholly objectively, without personal influence, and if so, how? I merely contend what some psychology articles (which I guess I'll have to start posting) say. Which is we all fundamentally make many daily decisions subconsciously even when we think they are consciously made and reasoned. We rationalize logic around them, but often those decisions are from feelings, beliefs and experiences - the decision comes first subconsciously and then we validate it with what seems like logic. In other words, we often lie to ourselves autonomously - there's no malice involved. Since we all do it, nobody should be adopting an absolutist 'it doesn't apply to me' stance. But that was one of my other points about audiophiles and people in general: we lie to ourselves without even knowing.

Similarly, while one can think boarding a plane is statistically safe, there's no doubt all the things that I mentioned one could check for evidence are not being personal done, such that one could claim to have ALL the evidence to make a 100% objective decision whether to board a plane or not. Some part of it requires trusting institutions, laws, safety systems etc. Those thing can be based on objective science, but if we haven't reviewed it all first hand and simply believe what we were told, taught etc... then we're trusting the accuracy of information, rather than knowing the accuracy of it.

I equate the trust to subjectivity and the knowing to objectivity.
 

SIY

Grand Contributor
Technical Expert
Joined
Apr 6, 2018
Messages
10,386
Likes
24,749
Location
Alfred, NY
Paul is intimating that 100% of all his decision making (which would include things like which toothpaste to buy, what food, flavour ice cream, music to listen to etc) are evidence-based decisions.

Except that's not exactly what he said.
 

Here2Learn

Active Member
Joined
Jan 7, 2020
Messages
112
Likes
113

Koeitje

Major Contributor
Joined
Oct 10, 2019
Messages
2,292
Likes
3,880
What would be great would be to catalog the actual test here like above. Many are hard to find.
You can find the Audiocritic test on page 7 in the attached pdf. Or at least the story.
 

Attachments

  • The_Audio_Critic_24_r.pdf
    1.7 MB · Views: 268
Last edited:
OP
ahofer

ahofer

Major Contributor
Forum Donor
Joined
Jun 3, 2019
Messages
4,952
Likes
8,698
Location
New York City

Here2Learn

Active Member
Joined
Jan 7, 2020
Messages
112
Likes
113

^ was very informative.

I have my own musings after reading, and they are just that, nothing more.

IMHO, Archimago's study didn't quite elucidate as clearly as I would have liked, as to why 0.02% THD was favoured over 0.0000002% THD. Or perhaps I didn't mentally note it reading the article. I think there wasn't enough foray into why the 0.0000002% THD was in some cases less preferable to 0.02% THD. This was somewhat surprising to me.

I felt the conclusion that the threshold between subjectively positive and subjectively negative views of a recording lay between 0.02% THD and 0.03% THD was premature and warrants further study/verification for reasons below.

If he had another recording at 0.01% THD, would it get preferred over 0.02% THD track?

Certainly, there's little doubt any of the replay systems could even approach replay to 0.0000002% THD fidelity. They are introducing distortion themselves in addition to the non-linear distortions added to the tracks via the Distort software. Shouldn't the scientific method dictate all variables are fixed and the only one variable, distortion from the Distort software, is varied under control? The study doesn't know or control the replay system THD of the participants.

So wouldn't the average THD level of the systems used for the study somewhat cloud what could be perceived by the test participants? What use is a recording with 0.0000002% THD if the systems cannot render it without a far worse THD?

Perhaps it was the average threshold of the fidelity of the systems in use which dictated the audibility threshold for the non-linearities added to the recordings. I didn't see that avenue objectively ruled out in the study.

I'm happy to go along with the conclusion that objectively better measuring systems in the context of non-linear distortions would equate to subjectively preferable. I am not convinced the threshold of audibility for such distortions has been correctly identified in this study.
 

Wes

Major Contributor
Forum Donor
Joined
Dec 5, 2019
Messages
3,843
Likes
3,788
Imagination seems to affect our ears moreso than other senses. Here's how it typically works: Mr. Reviewer takes a known recording, often Pink Floyd's Dark Side of the Moon, and usually the alarm clock sequence, and then waxes incredibly about how he's now hearing things (it's an alarm clock, for crying out loud!) he's never heard before with new component "X" or special calbe "Y" in his system. How many times have we read this exact scenario, over and over, in reviews?

Think about how that is. Here's Typical Reviewer Guy listening to a record he's heard a thousand times before in conjunction with numerous pieces of expensive hi-fi gear out on loan, and he's now telling you that he's hearing stuff he's never heard before, with such 'clarity' or 'slam', or whatever it is. Stop and think about that. Does that even sound like something that could happen? In real life? But the people who consume this sort of thing just accept it, and wait for the next time their favorite reviewer inserts a new cable or DAC in their system, and goes on about how he's hearing new things in those alarm clocks that he's never heard before.

You have to laugh out loud. It's really funny.

My new DAC allows me to hear the ticking of Digital alarm clocks on 9 of the 14 DSOM variants.
 
OP
ahofer

ahofer

Major Contributor
Forum Donor
Joined
Jun 3, 2019
Messages
4,952
Likes
8,698
Location
New York City
OP
ahofer

ahofer

Major Contributor
Forum Donor
Joined
Jun 3, 2019
Messages
4,952
Likes
8,698
Location
New York City
Top Bottom