Just how legit is this person's blind test results?

[JRS]

Who ever required an "independent observer" for a *perceptual* test? Or for it to be peer reviewed? You can't witness what someone else experiences in their mind, nor check that what they report corresponds to their subjective experience. You also can't peer review people's recorded perceptions. These same misunderstandings or presumptions have been expressed over and over through this thread. It is not inspiring.

The hell you can't. The Journal of the Audio Engineering Society publishes papers where exactly that is the grist for the mill. Unfortunately, much is behind a paywall. Occasionally a paper gets discussed here. Last I recall was supposed proof that cable differences were real. It wasn't very well peer reviewed as there were gaping holes in the methodology. But it depended on perceptual differences. A good example of solid research was done by Harmon and established the preference curve used here quite a bit, maybe too much but its a solid beginning.

 

[Keith_W]

Your very first sentence is an ad hominem. And emphasised too! Your third paragraph is an amazing expression of bias. He didn't perform a sighted listening test yet you condemn him for doing so. That is neither objective nor rational. This kind of reaction maybe plays well but it could hardly be less credible.

I agree. It is putting the cart before the horse. Now I understand why so many people are so upset by the result of this DBT. They have an almost religious belief that there can not possibly be any difference, and when someone consistently picks the correct result in a blind test, the result is denial, obfuscation, moving goalposts, and so on. Plenty of that in this thread. This is neither rational nor objective. It is an emotional response. Now I don't deny that scientists can get emotional, but come on guys ...

 

[Galliardist]

That question was clear and obvious from the start. Can a person in a blind test distinguish a discrete headphone amp from an op-amp headphone amp?

I disagree. The purpose of the test was to prove that the tester could hear differences that another researcher couldn't. If we are talking about the test in terms of some kind of serious research, that doesn't cut it. From the description of the test, no less:

Amir is hostile to the fact that people can actually have the ability to hear differences with amps that have well over 102+ SINAD (2V into 300-ohms 20kHz bandwidth), so perhaps this affects his hearing ability?

 

[JustJones]

Now I understand why so many people are so upset by the result of this DBT.

First off It wasn't a DBT and I'm in no way upset, just amused.

I doubt anyone could hear differences between DUT with the numbers these amps have but it would be interesting if someone could doing proper controlled testing not family affair tests. Good lord how many people here have had family members swap things and heard no difference? Does that prove anything? Is it worth posting and having, what now 16 pages of hand waving?

 

[Galliardist]

I agree. It is putting the cart before the horse. Now I understand why so many people are so upset by the result of this DBT. They have an almost religious belief that there can not possibly be any difference, and when someone consistently picks the correct result in a blind test, the result is denial, obfuscation, moving goalposts, and so on. Plenty of that in this thread. This is neither rational nor objective. It is an emotional response. Now I don't deny that scientists can get emotional, but come on guys ...

There's no need to get emotional. No real scientist (I'm not one, and I guess nor are most of the commenters here, but my BSc suggests I know a little) would accept that what was described to us here, or generally for listening tests reported in generalist forums, provides a formal proof of anything. Your reaction assumes that this was a scientific experiment of import, and that we are all scientists here. This was not a DBT, and it was not good science. Those two sentences are cold hard facts.

I presume that we can accept that he heard what he heard. That gets us nowhere, because we actually have no idea why he may have heard it.

 

[julian_hughes]

The hell you can't. The Journal of the Audio Engineering Society publishes papers where exactly that is the grist for the mill. Unfortunately, much is behind a paywall. Occasionally a paper gets discussed here. Last I recall was supposed proof that cable differences were real. It wasn't very well peer reviewed as there were gaping holes in the methodology. But it depended on perceptual differences. A good example of solid research was done by Harmon and established the preference curve used here quite a bit, maybe too much but its a solid beginning.

The hell I can't what? We're supposed to defer to you because you use the word "hell" and then appeal to papers which you can't cite and we can't see? This is not counterpoint but just reaction and appeals to a hidden authority. So weak.

 

[solderdude]

You mean Schiit talking ?

 

[JRS]

The hell I can't what? We're supposed to defer to you because you use the word "hell" and then appeal to papers which you can't cite and we can't see? This is not counterpoint but just reaction and appeals to a hidden authority. So weak.

I simply took issue with your claim that perception was somehow outside the realm of science. Which is pure poppycock. Indeed there is a branch of human physiology which studies exactly this topic. Its called psychophysics and no less than Helmholtz and Ernest Mach were pioneers. Or perhaps you have heard of the Fletcher-Munson curve? Or there's this unit of measure called the decibel. To say nothing of all the work on information theory that has its roots in the intelligibility of human speech. I can't help that you are unfamiliar with the JAES or the origins of the Harmon curve. But some of this background is actually useful and well worth having at least a passing familiarity before you come here ranting that we are not taking some internet dude's claims with sufficient open mindedness.
From these studies scientists have determined such interesting phenomenon as the human eye, sufficiently dark adapted, can detect with confidence a blip of five photons and even a single photon more than chance would have it. Likewise in Microsoft's soundproof chamber, hearing well below 0 dB is possible and one can clearly hear his bones creak and heart beat--usually to a maddening extent that forces termination of the experience.
So please refrain from attacking other members when they question your assumptions.

That my friend is part and parcel of science, and should you find such scrutiny uncomfortable, do us all a favor and take a hike.

Addendum:
Oh and this took exactly five seconds to find: https://secure.aes.org/forum/pubs/journal/?ID=2
Only in abstract form (note author and company): https://www.aes.org/e-lib/browse.cfm?elib=3839
Here's a gem: https://www.aes.org/e-lib/browse.cfm?elib=5549 From it, the following quote:

Experience from many years of double-blind listening tests of audio equipment is summarized. The results are generally consistent with threshold estimates from psychoacoustic literature, that is, listeners often fail to prove they can hear a difference after non-controlled listening suggested that there was one. However, the fantasy of audible differences continues despite the fact of audibility thresholds.

(Sorry I don't have 33.00 per article for you to read).

 

[AdamG]

OK Guys, let’s step back from making personal insults. Stick to the ideas, concepts and science. Any further insults, slights and/or diss, will be removed and possibly issued further sanctions. Treat others with dignity, respect and compassion or expect none in return.

Thank you for your understanding and support.

Quoting myself because apparently some missed or ignored the message here. Asking nicely time is over. Warning issued. Remain Civil or you won’t be permitted to remain.

For those of you who behaved like adults. I thank you kindly!

 

[Mr. Widget]

Addendum:
Oh and this took exactly five seconds to find: https://secure.aes.org/forum/pubs/journal/?ID=2
Only in abstract form (note author and company): https://www.aes.org/e-lib/browse.cfm?elib=3839
Here's a gem: https://www.aes.org/e-lib/browse.cfm?elib=5549 From it, the following quote:

Experience from many years of double-blind listening tests of audio equipment is summarized. The results are generally consistent with threshold estimates from psychoacoustic literature, that is, listeners often fail to prove they can hear a difference after non-controlled listening suggested that there was one. However, the fantasy of audible differences continues despite the fact of audibility thresholds.

Thank you! I should have invested the five seconds myself. I will follow up with a deeper dive.

Very interesting reading. This is precisely the background info I have been looking for. I particularly appreciated the long term "audiophile" vs. the ABX test.

Spoiler alert: The audiophiles could not reliably detect distortion in the long term listening test as reliably as the second group using an ABX comparator.

 

[Blumlein 88]

Thank you! I should have invested the five seconds myself. I will follow up with a deeper dive.

Very interesting reading. This is precisely the background info I have been looking for. I particularly appreciated the long term "audiophile" vs. the ABX test.

Spoiler alert: The audiophiles could not reliably detect distortion in the long term listening test as reliably as the second group using an ABX comparator.

In case you've never heard about the Richard Clark Amplifier Challenge offering $10,000 to anyone successful, you might wish to read this description of it. NOTE: Not to be confused with David L. Clark.

Richard Clark $10,000 Amplifier Challenge

Has anyone read this? Basically Richard is say that as long as the amplifier is operating with in its range, distortion free that no one can make an audible difference. Sure you can have better quality components in the amp but states that power is power. Link below http://tom-morrow-land.com/tes...

www.stevemeadedesigns.com

This is an Amir post about one of the David Clark papers. The 3rd link from JRS.

AES Paper Digest: Sensitivity and Reliability of ABX Blind Testing

This is abbreviated synopsis of the paper, Ten Years of ABX Testing, by David Clark: http://www.aes.org/e-lib/browse.cfm?elib=5549 ABX testing as you may know is a double blind testing methodology aimed at determining if someone can reliably identify A from B. Both A and B are presented to the...

www.audiosciencereview.com

 

[Mr. Widget]

In case you've never heard about the Richard Clark Amplifier Challenge offering $10,000 to anyone successful, you might wish to read this description of it. NOTE: Not to be confused with David L. Clark.

Nope... I have been blissfully unaware of most of this craziness and only recently got sucked into this pit of quicksand. For the most part I use my audio time listening to and enjoying music.
But this has been an interesting diversion.

 

[ahofer]

OK, disbelieve every person who doesn't see things the same way as you and place your trust in every person who even superficially seems to agree with you. That's a real recipe for success. Come on, we're not children.

You've strawmanned me, as I did not suggest that. My point is we have to approach this with some sort of reasonable epistemology. Other people may be right or wrong, but we need an accepted way of settling that. Rigorous, scientific method is that way, nobody has found better. That is why I laid out what would cause me to revise my beliefs, and was enthusiastic about a cooperative trial.

I really don't know what you are getting agitated about, but I'd encourage you to read more carefully.

 

[ahofer]

They have an almost religious belief that there can not possibly be any difference, and when someone consistently picks the correct result in a blind test, the result is denial, obfuscation, moving goalposts, and so on.

I have a high degree of skepticism based on a mountain of prior evidence, but I've stated what my conditions would be for revising my belief. I would say most of the reactions reflect same.

Proving something wrong against a ton of scientific evidence is hard. No mystery.

 

[BDWoody]

A video of a repeat test would show them, so best to drop that idea unless an adult can substitute for them.

I think making this an adult video is an excellent idea.

On a separate note, Archimago has another test up.

Internet Blind Test: 24-Bit vs. Dithered 16-Bit Part Deux - Daft Punk Edition! And the honest desire to seek truth as audiophiles.

A blog for audiophiles about more objective topics. Measurements of audio gear. Reasonable, realistic, no snakeoil assessment of sound, and equipment.

archimago.blogspot.com

 

[Keith_W]

I have a high degree of skepticism based on a mountain of prior evidence, but I've stated what my conditions would be for revising my belief. I would say most of the reactions reflect same.

Proving something wrong against a ton of scientific evidence is hard. No mystery.

Scientific evidence that all amps sound the same? Please show me. If you don't have it, then I take it that this is your religious belief that you have arrived at as an article of faith, or dogma, and not science. You are only applying scientific lipstick to dress up what is in essence an unsupported belief. In the meantime, here is another blind test showing that amplifiers can sound different. And here is another. And here is a nice discussion about how and why they can sound different.

 

[delta76]

Scientific evidence that all amps sound the same? Please show me. If you don't have it, then I take it that this is your religious belief that you have arrived at as an article of faith, or dogma, and not science. You are only applying scientific lipstick to dress up what is in essence an unsupported belief. In the meantime, here is another blind test showing that amplifiers can sound different. And here is another. And here is a nice discussion about how and why they can sound different.

Do you even read the link you posted?

 

[MarkS]

Someone says he reliably hears a difference which might be actually true but we would like to know what is responsible for the successful ABX. It could be hidden/false clues, it could be even cheating (or the whole story being made up) but it could also be a surprise and then things could become interesting.

No, this is not possible. Whatever the cause, it is simple and prosaic. Our understanding of amplification circuits is far too thorough and complete for there to be any possible surprises here.

This requires professionalism etc from both sides which is not what we are seeing. With an us-versus-them attitude (from both sides, mind you) no progress will ever be made, rather a honest joint effort would be needed.

Professionalism requires understanding what has come before, and weighing new claims against existing knowledge. In this case, the existing knowledge is vast and overwhelming. Not recognizing that is the unprofessional act.

It's not "us-versus-them", it's long and hard acquired knowledge versus new claims with insufficient support and documentation to be taken seriously.

 

[krabapple]

OK, disbelieve every person who doesn't see things the same way as you and place your trust in every person who even superficially seems to agree with you.

Confronted with results that can a priori be labelled as 'remarkable', the most kindly stance one of a scientific inclination could take is 'trust but verify'.

IOW, don't trust completely. Validate.

You seem to have issues with all that.

 

[krabapple]

Paper cash money only. How much you got? That's how much I need. Both Objectively & Subjectively. I aim to please.

Send me your bank account # and login, I'll transfer as much $$ as you need.