• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Audiophile objections to blind testing - an attempt from a layman

Joined
Apr 12, 2021
Messages
33
Likes
85
Hello everyone. First, a bit of preamble that hopefully will make sense once I come around to the point I'm trying to make at the end.

I am a former subjectivist - read my story here: https://www.audiosciencereview.com/forum/index.php?threads/repentant-subjectivist.22411/

I have recently come back to the hobby after years of absence "just" owning a "poverty entry level" rig of Stax L500 mk2 > 006t > modi 3. Because of better listening conditions in our new home, I am returning and got the 007mk2/L700mk2 > 727II > Eversolo Z8 DAC going. This newfound interest has of course once again subjected me to the classic audiophile discourse. But it has also subjected me to my own bias' and how I interpret the different equipment. I am doing my best to remain rational in a field where I understand very little.

When I changed the 006t amplifier to the 727II amplifier on the L700, I didn't expect to hear any changes. However, I certaintly did. The subjectivist in me wants to say that the sound became a bit brighter, a bit more detailed and a bit more engaging. For better or worse, as the L700 through the 006t was a serene and relaxing listening experience that, although less "resolving" sounded fantastic. Now I'm sure this effect is most likely due to either higher listening volumes or just expectation bias etc.

I EQ all my headphones through roon after oratory1990 settings to Harman target. It is obvious to me that while the headphones post EQ might measure very similar, they don't actually sound similar. I am very confident that I would be able to discern between different headphones EQ'ed to the same curve in a double blind test. If only there was a way to not tell by the sensation of the headphones on my head. Either way, this tells me that surely there must be other measurements than just distortion figures and frequency response that explains the sound difference. Size and shape of the driver, earpad thickness etc surely plays a role.

One of the main reasons for me evening "seeing the light" and understanding that the audiophile hobby is mostly a combination of psychoacoustics and dopamine addiction is the topic of electrostatic amplifiers. When I bought a pair of Stax SR-007mk1 and paired it with a 727II, I kept reading about how this was a bad sounding combination and that I needed a $ 6000 USD third party amplifier to drive the headphones. This to me seemed so ridiculous that I started questioning it, and it led me down the path I am still on to this day. My thinking is that if the SR-007 sounds vastly improved through the Mjolnir Carbon (6k third party amp) compared to the Stax SRM-727II amp, there should be a way to measure it. But as I understand it, the frequency response will actually be the same no matter what amp you run it through. However it is also clear that a severely underpowered electrostatic headphone will sound very bad. My personal experience putting the 007 into the entry level 252s amp sounded terrible - broken almost. I would be confident I could easily discern between the 252s and 727II. This might come down to distortion, but I don't know.

Anyways, my thought has always been that if the difference is as big as the "stax mafia" proclaims - why would they not produce measurements or tests that actually prove this difference? Many of these people own a ridiculous amount of gear, have the necessary knowledge to pull of measurement/tests and would clearly have an interest - sometimes even commercial interest, possibly. So when they don't do that, it makes me think they just don't want to know the results. And that may very well be the case.

So, to the point of blind tests. When I was still a gung ho subjectivist, I remember having some opinions on the limits of blind tests. These opinions weren't very well developed, but there was something in me intuitively that told me that blind testing was very limited. I remember thinking something along the lines of - "well, if I can't "put the sound on a mental coat rack so to speak", then I can't properly interpret it." And I suppose this might be correct, and my opinion on blind tests in a way coming down to me not understand what "objective" even means - just acknowledging the interpretation part should've made me realize that I needed to read up on psychoacoustics.

I'm largely confused about it all, but I saw something I found interesting on TV the other day. Basically, there is a Norwegian christmas advent calendar show on the state channel that has celebrities go through various christmas related challenges. In one of the challenges, they were tasked to identify various food items from a christmas cold table with limited senses.

Task 1 was to identify the food without seeing or otherwise touching it, while being spoon fed it. They were fed "pultost", a matured sour milk cheese.
Task 2 was to identify the food by smell only. The food was pickled herring.
Task 3 was to identify food in a opaque container by shaking the container. The food was rice porridge.
Task 4 was to identify the food by touch only. The food was beetroot salad.
Task 5 was to identify the food by sight only, except it was placed a far distance from them. The food was a slab of pork ribs.
Task 6 was to use their "6th sense" by essentially guessing what hid under a lid without having any way of inspecting what was under. The food was cabaret.

Most if not all of these food items is something adult norwegians will have a close relationship with, having eaten it most every christmas for their entire lives. To my surprise, almost none of the contestants could identify the items. The most interesting test to me was the pickled herring smell test. Pickled herring is extremely popular and has a quite distinct smell which most if not all contestants would be familiar with. Yet only 1 or 2 out of about 14-15 contestants managed to correctly identify it as pickled herring.

This made me think about a claim I have often heard when it comes to blind testing: "If you can't hear a difference in a blind test, then for all intents and purposes there is no difference in actual sound to that person".

I can't remember exactly what the contestants on the show said they believed the smell from the pickled herring was, but there was a lot of different responses from things that certaintly smells nothing like pickled herring. We can assume that since they wrongly identified the pickled herring from these items, they may be unable to differentiate pickled herring from things smelling very different in a blind test. My question then is, would this mean that in terms of smell - there isn't actually a difference because we wouldn't be able to differentiate it between a blind test? Surely not, right?

I can imagine someone knowledgeable reading this and laughing their ass of, since it probably shows a big lack of understanding in basically everything I'm talking about. But my idea here is basically that identifying something from smell looked to become almost impossible when not getting to see the foods. Same with taste and feel. This tells me that our senses of taste and smell is incredibly bad - bordering on useless - without the aid of sight. If this is true to some extent in regards to hearing, wouldn't this mean that blind testing is in fact almost useless because of our reliance on other senses to interpret sound?

Basically, if our senses of hearing is so lost if we isolate it - how useful is it to ascertain that we can't differentiate between gear in a blind test? All it would tell us is essentially that humans sense of hearing sucks. It doesn't seem to say a whole lot about whether or not there actually is a difference in sound, but rather that we can't discern a difference in a blind test. But in the same way I would hesitatet to say that the difference in smell between lets say pickled herring and beetroot salad is just bias and voodoo because a person couldn't smell the difference in a blind test - and thus no difference in smell actually exists - , I wouldn't say that a lack of discernment between equipment in blind test means that there isn't any difference in sound.

Of course where something measures the same, it must also sound the same objectively. But it feels like there are limitations to this, re: same frequency response but different sounds etc.

Okay I guess I'll stop typing and pray that I've made sense to some degree.
 
My question then is, would this mean that in terms of smell - there isn't actually a difference because we wouldn't be able to differentiate it between a blind test? Surely not, right?

No, in a an ABX between pickled herring and pickled cabbage, they'd probably consistently pick correctly, even though guesses to what the two samples could be would be all over the place. A failure of identification does not imply that there's no difference.

An audio ABX is a tool to investigate cause and effect. Identification isn't necessary. Only the difference or the lack thereof.
 
You are confusing 'identification' with 'detection of difference'. They weren't asked to simply discern whether two things were 'different'.

Perceptible difference has to exist first; without it, identification and preference are moot.
 
You are confusing 'identification' with 'detection of difference'. They weren't asked to simply discern whether two things were 'different'.

Perceptible difference has to exist first; without it, identification and preference are moot.
I realize its not 1:1 relationship. But my point isn't to say "this result means that they wouldn't be able to differentiate between pickled herring and >x similar smelling item<, blind tested busted". But rather, that its an example of how bad we are at using our senses in isolation. My thinking is essentially that while its true we can't trust our ears when doing sighted testing because of expectation bias etc., we might also not be able to trust our hearing in blind testing because its garbage and can't differentiate between anything almost.

IDK I'm not really convincing myself with this line of reasoning, so I might just be walking down a dead end lol
 
I realize its not 1:1 relationship. But my point isn't to say "this result means that they wouldn't be able to differentiate between pickled herring and >x similar smelling item<, blind tested busted". But rather, that its an example of how bad we are at using our senses in isolation.

Actually, what confuses us re:sonic difference is NOT using our sense of hearing in isolation. Our hearing isn't intrinsically 'garbage', it's actually very sensitive when it's allowed to be.

It's trivially easy to demonstrate all this. That's why blind controls are standard for demonstrating perceptible sonic difference.

Every so often someone comes along wanting to reinvent this wheel, and I guess this week it's you.
 
Actually, what confuses us re:sonic difference is NOT using our sense of hearing in isolation. Our hearing isn't intrinsically 'garbage', it's actually very sensitive when it's allowed to be.

It's trivially easy to demonstrate all this. That's why blind controls are standard for demonstrating perceptible sonic difference.

Every so often someone comes along wanting to reinvent this wheel, and I guess this week it's you.
Well, I don't really want to reinvent the wheel. However I think its better to subject the ideas that pops into my head to scrutiny instead of just believing them without questioning them, since I'm not able to critically question them myself.
 
In your analogy, a blind test would be asking people to say there is a difference (and perhaps describe the difference) between beets and herring by smell. I think you would agree most people can do that.

In a blind test of audio gear, you get to compare two or more things back to back. You don't have to listen to one song and then say "oh yes, this is a McIntosh amp." This is just as hard with listening to gear as it is with sniffing food, probably harder.

I think a better analogy would be asking people to differentiate two colognes or perfumes based only on smell. Food is a multi-sensory thing by design (smell, appearance, taste, texture) but audio gear and perfumes are single-sense products (hearing, smell.) And in practice people actually can do this pretty easily.


You're correct that lack of audio memory is a difficulty in these tests, but "fast switching" is a solution that works. If you can flip back and forth between two audio devices quickly in a blind test, people can identify even small audible differences.

If you put two devices into that kind of blind test and people STILL can't tell the difference, I think we can say pretty conclusively there is no practical difference between them, except maybe in how they look on your shelf or how big a dent they put in your wallet.

I wouldn't say that a lack of discernment between equipment in blind test means that there isn't any difference in sound.

If we isolate the sense of hearing and compare the sound of two things directly... how else are we going to establish there is or isn't a difference in sound? We could measure the output of the gear, I guess... ;)
 
Last edited:
I'm not certain I understand the reasoning here in the first place. If people have trouble telling things apart in controlled testing, how well do you think they are able to actually perceive a difference in regular use? There's no question that being able to use multiple senses makes it easier to tell things apart. But when we are talking about audio gear, what matters is the audio output. Whether we can differentiate the equipment more readily by using senses other than just our hearing is not really in question.
 
In a blind test of audio gear, you get to compare two or more things back to back. You don't have to listen to one song and then say "oh yes, this is a McIntosh amp." This is just as hard with listening to gear as it is with sniffing food, probably harder.
WHAT?! Fremer does it all the time you plebes!

(/sarcasm)
 
"blind testing is in fact almost useless because of our reliance on other senses to interpret sound?"

Do not confused the task to assess sound quality vs the act of enjoying music, quiet different thing. Occasionally I find my wife humming to music and even dancing, while listening to music through her iPhone speaker! She does not need a high-quality reproduction to enjoy music; but we OCDs, we need reassurance it is indeed high quality, we have created instrument that can assess that way beyond human ear capabilities. double blind testing is a time-consuming process if done scientifically, still has a place, specially to try to determine level of human hearing different thresholds, frequency response tastes in a population.
 
A brilliant Christmas Troll. At present this Thread is leading the competition and may be poised to win the Coveted annual holiday award. :cool:
 
A brilliant Christmas Troll. At present this Thread is leading the competition and may be poised to win the Coveted annual holiday award. :cool:


I guess the fact that the OP was talking about herring rather than hearing was the red flag that this was all a red herring
 
I have two identical potato chips/crisps. In blind testing, they are identical. In non-blind testing, one says “manufactured using slave labor” and the other says “manufactured by your friend’s grandma”. Your experience is different; your enjoyment is different because you cannot avoid the bias/subconscious perception.

Think about the shape of two fruits. One is distorted and one is perfect. Might it affect your experience?

The value of blind testing is to really know what is measurable. The disadvantage of blind testing is that you can make sighted bias a feature not a bug.

What we don’t know yet is if sighted bias is predictable.
We know that cables sound the same in blind testing. However, if you showed someone who NEVER knew anything about speaker cables, one that was made with copper and one that is made with silver, what would the person imagine? Is it “universal” that lighter color cables will be associated with bias toward treble? Is it universal that fat cables generate a bias of dynamics or bass?

In that case, the sighted bias is no different that the use of color theory

Except it’s visual theory.
 
What we don’t know yet is if sighted bias is predictable.
I think you vastly underestimated the knowledge of the marketing departments in this industry.

Is it universal that fat cables generate a bias of dynamics or bass?
It doesn’t have to be. You can tell people it is so. This is what is going on for decades. It’s a self fulfilling prophecy.
 
A brilliant Christmas Troll.
OP surely reads as such...

1734594340316.png



JSmith
 
Back
Top Bottom