• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Catalogue of blind tests

OP
ahofer

ahofer

Major Contributor
Forum Donor
Joined
Jun 3, 2019
Messages
4,947
Likes
8,694
Location
New York City
I think it's important to keep doing it - improving testing methodology, looking to disprove the null hypothesis of no audible difference that cannot be measured within known human sensitivities. It took a while for jitter to enter gear measurements. It might also help R&D focus on speakers, DSP, and room adaptations.

As for changing minds, I've learned the hard way from managing people that they don't have sudden epiphanies. They take in the evidence, and give themselves time to evolve in a face-saving way (it's a managers job to help create that path). It's a lot easier for me to look back and SMH on the money I spent on VTL amps and Analysis Plus cables now than it would have been then (25 years ago), even if that money would have been better put to my sons' college education funds.
 

MattHooper

Master Contributor
Forum Donor
Joined
Jan 27, 2019
Messages
7,194
Likes
11,806
As for changing minds, I've learned the hard way from managing people that they don't have sudden epiphanies. They take in the evidence, and give themselves time to evolve in a face-saving way (it's a managers job to help create that path). It's a lot easier for me to look back and SMH on the money I spent on VTL amps and Analysis Plus cables now than it would have been then (25 years ago), even if that money would have been better put to my sons' college education funds.

It's hard for most people to challenge or disregard their own "experience." Which is what blind testing challenges, and which science demands. That's why IMO it often takes another experience to challenge the first experience; many of us in years past were sure we experienced sonic differences in, say, cables, but when we had an alternative experience, performing blind tests, that was strong enough to make us re-evaluate the first subjective experiences.

Lacking first hand experience in blind testing or the protocols of science, I think it generally takes an attitude of really wanting to know the truth about something to the point of being willing to re-examine one's own assumptions and methods. From that standpoint you don't necessarily have to have done the work yourself, but only recognize that, say, it's inconsistent and relying on personal subjective bias to pretend our audio hobby is magically excepted from all the variables and human biases that need to be controlled for everywhere else when we are trying to be rigorous.
So it's possible to get there without personal experience, but it does take a certain attitude towards the truth, and a switch in epistemology of sorts, which tends to come from someone's constitution....though can be encouraged through education and debate.

As to the money that could have been put in to your son's education, that reminds me of a post I just read, I think on audiogon. from someone who had an expensive vinyl rig and record collection, but who fell in love with the "sound" of an expensive CD player/transport costing $80,000! (Eighty Thousand!). He perceived this as necessary to get the best out of his CD collection and stretched finances, including sacrificing his analog gear/record collection, in order to finance the purchase of that digital front end. I actually felt sad reading that. To think of how much money was sacrificed on the alter of unchallenged subjective inferences, and which could have been saved by just using a few reasonable protocols before buying. Yeesh. BTW, I am not condemning in principle someone spending that amount of money on their digital front end. People are free to spend as they please. If you go in to a purchase advisedly, with a good understanding of what you are really getting for your money, so be it. It's more sad when it is done so out of ignorance or being misled.
 
OP
ahofer

ahofer

Major Contributor
Forum Donor
Joined
Jun 3, 2019
Messages
4,947
Likes
8,694
Location
New York City
I suppose that $80k DAC can make him happy even if it isn’t audibly different. It just seems unlikely. And the level of dismissive anger when you bring up controlled testing is so palpable, I tend to think of it as heightened defense against regret.

I’m an investor for a living. Detaching oneself from all these myriad emotional/subjective complications is the secret to success, but impossible to do completely. Recognizing it is a crucial first step. The process often starts with keeping a journal of your decision process. When you go back and face the *actual* rather than rationalized, reasoning behind your failures, the scales tend to fall fast from the eyes. I think if I weren’t an investor, I’d have owned a bunch more amplifiers.
 
OP
ahofer

ahofer

Major Contributor
Forum Donor
Joined
Jun 3, 2019
Messages
4,947
Likes
8,694
Location
New York City

SIY

Grand Contributor
Technical Expert
Joined
Apr 6, 2018
Messages
10,383
Likes
24,749
Location
Alfred, NY
OP
ahofer

ahofer

Major Contributor
Forum Donor
Joined
Jun 3, 2019
Messages
4,947
Likes
8,694
Location
New York City
OP
ahofer

ahofer

Major Contributor
Forum Donor
Joined
Jun 3, 2019
Messages
4,947
Likes
8,694
Location
New York City
I just delivered this challenge to someone on the Facebook Harbeth Group. Heard a new one:

Andrew P. Hofer I make no claim to have golden ears. I believe that anybody with hearing that is not damaged and has decent training on how to listen to an audio system can differentiate differences in components with ease. Most people in this hobby have damaged hearing, either due to age or other trauma. I am young yet, but old enough to know what to listen for and smart enough not to expose myself to stupid loud sounds, so I would say that my hearing is reasonable enough to make these distinctions.

Having said that I am not willing to put money on this, it is a hobby after all and all it takes is a single bout of sickness to permanently affect hearing in a negative way. Also I believe firmly that travel has a negative impact on hearing. I remember my last vacation to Miami, where it wasn't until day 3 that most of my hearing in the upper frequencies returned (at least to 19khz), so no travel for me.
I wear earplugs when I travel, myself. I didn’t know Seattle was such a hotbed of tropical hearing-impairing diseases.
 

BDWoody

Chief Cat Herder
Moderator
Forum Donor
Joined
Jan 9, 2019
Messages
6,948
Likes
22,625
Location
Mid-Atlantic, USA. (Maryland)
I just delivered this challenge to someone on the Facebook Harbeth Group. Heard a new one:


I wear earplugs when I travel, myself. I didn’t know Seattle was such a hotbed of tropical hearing-impairing diseases.

Gotta be careful out there...
 

minusnine

Member
Joined
Feb 2, 2019
Messages
24
Likes
9
Location
the eastern flatlands
I recognise exactly what several of you above have articulated: that knowledge that if one has to really concentrate hard and put one’s frowny face on to distinguish between two amps, DACs, speakers, cables, bit rates or flavours of scented candle, whether blinded or not, then one isn’t going to perceive that difference when you’re listening to music, which surely is what we actually want to do with this kit in the end. But I think there is another factor that persuades people to keep seeking out, and then ‘hearing’, differences in their systems. It is that, if you become persuaded that you can’t improve your hifi system distinctly audibly, where does the hobby of audiophilia or whatever you want to call it go next?

In a way, I’m a case in point. This year, coming back to a position where I want to and am able to put together a decent hifi system and actually achieving that at a low cost (odac into atom then either to Hd58x or on to minidsp 2x4 for room correcting refurbed kef active speakers and a 2nd hand wharfedale subwoofer - total cost including stands and cables less than £800), now that my spending decisions are made, what is there left to do in hifi? I am interested to see the developments in driving SINAD of DACs ever lower, and finding out if anyone’s ever made a well-measuring AV receiver, but since I found audiosciencereview I can’t bear to look at other hifi forums, and I see no real reason to change anything in my system so my interest in the subject of the kit is falling away.

Now this is great - I can concentrate on actually listening to the new Nick Cave album without worrying whether I need some silver-twisted oxygen-free usb cable to really hear how he feels about losing his son - but it will probably see me visit here less often, has seen me abandon other hifi forums entirely, and certainly means Mr Richer and Mr Sevenoaks won’t be seeing any more of my cash, but for lots of people who’ve come this far or further, they don’t want to give it up (and I lied about not coming here so often- it’s my sanctuary from news about brexit and I think I’ll be needing you all for a while yet). The audioquests and others selling interconnects and daft accessories at £50 a pop are simply creating a way for hobbyists to maintain their interest (and spending), and you have to convince yourself you can hear the difference to keep going. I’m done with it, and I’m very happy to see the increasing tide of opinion in favour of objectivity in assessing hifi kit, but I can see how some people would not want to let go of the fiction.
 

scott wurcer

Major Contributor
Audio Luminary
Technical Expert
Joined
Apr 24, 2019
Messages
1,501
Likes
2,821
Now this is great - I can concentrate on actually listening to the new Nick Cave album without worrying whether I need some silver-twisted oxygen-free usb cable to really hear how he feels about losing his son -

A general comment the range of musical interest here is refreshing, people here actually listen to Conlon Nancarrow, Merzbow, Test Dept., and then Furtwangler's wartime Bruckner. These are all "punk rock" to somebody.
 

minusnine

Member
Joined
Feb 2, 2019
Messages
24
Likes
9
Location
the eastern flatlands
A general comment the range of musical interest here is refreshing, people here actually listen to Conlon Nancarrow, Merzbow, Test Dept., and then Furtwangler's wartime Bruckner. These are all "punk rock" to somebody.
Absolutely- I regularly dip in to the ongoing megathreads of new or old music here, and have found many new favourites.
 

BDWoody

Chief Cat Herder
Moderator
Forum Donor
Joined
Jan 9, 2019
Messages
6,948
Likes
22,625
Location
Mid-Atlantic, USA. (Maryland)
I’m done with it, and I’m very happy to see the increasing tide of opinion in favour of objectivity in assessing hifi kit, but I can see how some people would not want to let go of the fiction.

I think that's why many get into Vinyl. Plenty to tweak and actually hear differences.
Satisfies that need...
 

PaulD

Senior Member
Joined
Jul 22, 2018
Messages
453
Likes
1,340
Location
Other
A general comment the range of musical interest here is refreshing, people here actually listen to Conlon Nancarrow, Merzbow, Test Dept., and then Furtwangler's wartime Bruckner. These are all "punk rock" to somebody.
Agreed! It's very refreshing. :D
 

minusnine

Member
Joined
Feb 2, 2019
Messages
24
Likes
9
Location
the eastern flatlands
I think that's why many get into Vinyl. Plenty to tweak and actually hear differences.
Satisfies that need...
I think that's absolutely true, and could even apply to me if money was no object - the only thing I feel I miss out on through having exclusively streamed/ripped sources is the performance of putting on a record (or even a CD), having that big bit of cover art and reading the sleeve notes. But I'm just that bit too young to have bought LPs and a record player when I first bought music. if I win the lottery...
 
OP
ahofer

ahofer

Major Contributor
Forum Donor
Joined
Jun 3, 2019
Messages
4,947
Likes
8,694
Location
New York City
Amir just posted this cable test elsewhere. I hadn't seen it before:

https://www.avsforum.com/forum/86-u...41184-observations-controlled-cable-test.html

And to cut to the chase, Mike could not identify the Monster from the Opus MM with any accuracy (nor the reverse, which also would have been a positive result if he had been consistently wrong) using our testing methodology. We stopped the test a little less than halfway through, I think we got through 8 A/Bs before we gave up.

Mike apparently has a $500k system.
 

anmpr1

Major Contributor
Forum Donor
Joined
Oct 11, 2018
Messages
3,722
Likes
6,405
Amir just posted this cable test elsewhere. I hadn't seen it before: https://www.avsforum.com/forum/86-u...41184-observations-controlled-cable-test.html
Mike apparently has a $500k system.
Imagination seems to affect our ears moreso than other senses. Here's how it typically works: Mr. Reviewer takes a known recording, often Pink Floyd's Dark Side of the Moon, and usually the alarm clock sequence, and then waxes incredibly about how he's now hearing things (it's an alarm clock, for crying out loud!) he's never heard before with new component "X" or special calbe "Y" in his system. How many times have we read this exact scenario, over and over, in reviews?

Think about how that is. Here's Typical Reviewer Guy listening to a record he's heard a thousand times before in conjunction with numerous pieces of expensive hi-fi gear out on loan, and he's now telling you that he's hearing stuff he's never heard before, with such 'clarity' or 'slam', or whatever it is. Stop and think about that. Does that even sound like something that could happen? In real life? But the people who consume this sort of thing just accept it, and wait for the next time their favorite reviewer inserts a new cable or DAC in their system, and goes on about how he's hearing new things in those alarm clocks that he's never heard before.

You have to laugh out loud. It's really funny.
 

Sgt. Ear Ache

Major Contributor
Joined
Jun 18, 2019
Messages
1,894
Likes
4,150
Location
Winnipeg Canada
It really is quite bewildering. The sad thing is, I do it to myself! Even though I'm fully aware of how fallible these ears are under the influence of all the weird biases we have I still fool myself. A recent example - I generally stream music from my laptop to my system via bluetooth. Yep...crummy old bluetooth. Its really just convenience for me, but I also know that even SBC bluetooth is generally operating at essentially a 320kbps level and since I know I can't really tell any difference at all between 320kb and lossless I don't worry about it. However, I do sometimes run the audio from my laptop to my Topping D10 via USB and then wired into my system via rca cable - so, 16bit/44.1 CD quality flac is maintained. There are times when I'm convinced I am hearing details that weren't there via bluetooth. I get a nice warm fuzzy feeling of bliss that I'm enjoying improved SQ and hearing my music in a much richer, fuller way. Invariably though, if I switch back to bluetooth and listen specifically for those new details I think I'm hearing for the first time, they are actually still there and were all along. If I did a blind A/B comparison I'm sure I'd not be able to tell the difference. The brain wants us to be happy and it does whatever it can to make it so. lol...
 

anmpr1

Major Contributor
Forum Donor
Joined
Oct 11, 2018
Messages
3,722
Likes
6,405
It really is quite bewildering. The sad thing is, I do it to myself! ....
I fool myself all the time. Or at least some of the time. One day my system sounds great. Another time it sounds less great. And you know what? I haven't changed a thing. Maybe it's atmospheric or temperature fluctuation on my inner ear. Maybe it's aliens trying to contact me through the capacitors. Who knows what it is? But whatever it is, I'm convinced that a few feet of new and improved magic cable, or a different DAC, isn't going to really change things for me, add 'air' between the rings of Pink Floyd alarm clocks, or otherwise send me to audio nirvana.

But seriously, I think there could be valid physiological changes that influence hearing in odd ways. I know that with vision, some people get what is called ocular migraines. It's not a headache, but a restriction in blood flow to the retina. Temporary, and not harmful, the condition is responsible for producing geometric patterns when an affected person looks out. Perhaps there are analogous temporary auditory conditions that cause subjective changes in hearing acuity. I never read anything about it, but that sort of thing wouldn't surprise me if it was the case.

On the other hand, I think the cable thing you read about is just pure wishful thinking and imagination.
 

Sgt. Ear Ache

Major Contributor
Joined
Jun 18, 2019
Messages
1,894
Likes
4,150
Location
Winnipeg Canada
I totally agree wrt the notion of physiology playing a part in what we hear. Actually, I really think there's a whole raft of things that influence what we hear in ways far more profound than almost any of the tweaks audiophiles like to employ. Physiological variability, atmospheric changes, ambient sound levels...
 

Here2Learn

Active Member
Joined
Jan 7, 2020
Messages
112
Likes
113
I agree with the notion that most differences are not going to be audible unless swapping a unit out for a "statistical outlier" (as alluded to on page 1 of this thread). I'd go further and suggest much in the audiophile world is basically a form of OCD. Much like how some women will obsess over their figure and self-perception about attractiveness, I think some males in the audiophile world are actually obsessed over high-cost status symbols (at least subconsciously) and it feeds in to their own self-perception and self-worth Since self-worth and esteem is psychologically a foundation for one's perceived security, it's no wonder than any assault throwing down the gauntlet on a personal world view is typically met with scorn.

As ahofer hints at in regards to being a successful investor, the human condition is to lie to ourselves and believe those lies. The other problem is where the exception is people genuinely hearing a difference due to statistical outlier products that are measurably audibly different. For those, the problem is the OCD hankering for 'better' and confusing 'different' for 'better' but always making sideways moves over time and becoming perpetual box-swappers. This latter scenario reinforces the audiophile view that not all DACs, amps etc sound the same, but rather, there are differences to be had and objective science appears wrong. Thus it can entrench beliefs contrary to objective science.

But appearances, as they say, can be deceptive. It's not so much that audiophiles need an education or don't understand the science-based arguments, but that tipping over peoples world-views, self-perception and security in one fell swoop will be resisted by human nature itself. We tend to want to rationalize our perspective as 'right' in our experience, even when we are wrong. It's a rare individual who will simply acknowledge being wrong and do an about face on their beliefs. Good scientists do, but tend to have learnt to trust observed data from a young age, so they don't develop beliefs based on subjective experience the way most people do.

While this website is good for those of us who are either science-led, want to be science-led or have gone through the mire to have our eyes opened, it will not change the human condition quickly. I have friends who know I lean almost exclusively to the scientifically objective side of things and have bought equipment based on specifications alone, yet they still don't want to try ABX tests despite a few of them actually being scientists in illustrious fields - they still will pay significant amounts for things Amir (and many scientific tests over decades) have shown time and again will make zero audible difference.

All this then leads to objective audiophiles and science, vs subjective audiophiles and the "just listen to it" mantra, but...

People innately trust daily experience over scientific data - even the scientists. For example, when you drive across a bridge you likely exercise a belief it won't collapse. It probably doesn't even come to mind, and you do not give a second thought to whether the engineering plans and math on how it handles stressful forces have been undertaken correctly (or at all) so as to objectively know it won't collapse. Therefore, while you can rationalize being an individual who believes in science, the mere fact you believe its been done correctly without checking means you exercise a belief without supporting data. This is much like the sales pitch for the engineering in all manner of products that people fall for. They haven't checked the validity of the claims, but are happy to place a belief in some claims just the same. So whether there's real science or snake oil behind an audio product, or a bridge design, it makes no difference - people's first inclination is to believe in a positive outcome for which they've seen no empirical data. Life would be unlivable if we had to check empirical data first hand for everything.

So the problems of unravelling audiophile thought processes are not going to be won simply by websites like ASR, or forcing unobjective audiophiles to take ABX tests. This is a personal journey for all who love audio about wrestling the faults and frailties of our own human condition and our tendency to lie to ourselves. Like Neo in the Matrix, we either wake up to the real world, or, like Cypher, choose to enjoy the experience of a steak knowing ignorance can be bliss.
 
Top Bottom