• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Just how legit is this person's blind test results?

krabapple

Major Contributor
Forum Donor
Joined
Apr 15, 2016
Messages
3,169
Likes
3,717
I agree. It is putting the cart before the horse. Now I understand why so many people are so upset by the result of this DBT. They have an almost religious belief that there can not possibly be any difference, and when someone consistently picks the correct result in a blind test, the result is denial, obfuscation, moving goalposts, and so on. Plenty of that in this thread. This is neither rational nor objective. It is an emotional response. Now I don't deny that scientists can get emotional, but come on guys ...

I judge the worth of posts by who 'likes' them*.

This one of yours , not looking so hot.




*lying**


**sometimes
 

krabapple

Major Contributor
Forum Donor
Joined
Apr 15, 2016
Messages
3,169
Likes
3,717
Scientific evidence that all amps sound the same? Please show me. If you don't have it, then I take it that this is your religious belief that you have arrived at as an article of faith, or dogma, and not science. You are only applying scientific lipstick to dress up what is in essence an unsupported belief. In the meantime, here is another blind test showing that amplifiers can sound different. And here is another. And here is a nice discussion about how and why they can sound different.

Amplifiers, how do they work? Do we have any idea?
Human hearing, its limits, do we have any idea what they are?

Hey, I saw an ultraviolet two-headed horse in my living room!

Now, everything known about human vision says we don't see in the ultraviolet range (without help). Everything we know about horses says two heads are...rare.

But unless you show me a paper testing whether someone can see two headed ultraviolet horses, there is no scientific evidence to refute my claim!
 

rdenney

Major Contributor
Forum Donor
Joined
Dec 30, 2020
Messages
2,235
Likes
3,857
The objective of every amplifier is--should be--to linearly enlarge the incoming signal with the least distortion (for me, distortion is any departure from linearity). Any amplifier meeting that objective should sound the same as any other amplifier meeting that objective, no matter what the amplification technology.

There is a threshold of "non-linear" where it becomes detectable. That threshold varies based on the type of distortion and the hearing skills of the listener. We know quite a lot about that. My own threshold for relatively undetectable harmonic distortion is very low, perhaps under 40 dB.

All these challenges were limited to amplifiers operated within their linear range. The question is: At what point, and under what load, does a given amplifier lose linearity? Did it happen during the test? Does it happen in any stated use case? Any test of amplifiers should evaluate this, it seems to me. There was one article from the deep past suggesting that most amplifier evaluations were made with distortion levels elevated by pushing the amps into their nonlinear range. I think the suggestion of that article (which I cannot now recall the citation for) was 1%--40 dB. Most distortions at that level will be audible by most people by then, particularly in the way high-power transients are being amplified (or not).

Of course, not all amplifiers are trying to be linear. Some apply a coloration very much on purpose. But what grinds my gears is when their creators then claim that they are linear.

That is what the gentleman should have explored, given his conviction that the differences were "ridiculously easy" to hear. If there were differences that were easy to hear, then one of the amps, at least, was nowhere close to being linear. I doubt the manufacturer would be willing to admit that.

Rick "thinking high-power transients happen a lot more often than people realize" Denney
 

Newman

Major Contributor
Joined
Jan 6, 2017
Messages
3,454
Likes
4,216
Scientific evidence that all amps sound the same? Please show me.
Dude, you only have to change the volume of one of them to make them sound different. Or deliberately make an amp to have non-transparency. Everyone knows that. Even scientists.

I think you need to clarify what exactly is the claim you disagree with.
If you don't have it, then I take it that this is your religious belief that you have arrived at as an article of faith, or dogma, and not science. You are only applying scientific lipstick to dress up what is in essence an unsupported belief. In the meantime, here is another blind test showing that amplifiers can sound different. And here is another. And here is a nice discussion about how and why they can sound different.
Do you even read the link you posted?
LOL, great point. Especially his link that concludes:-
“One system was made up of the very latest SOTA components and connectors. The other system was comprised of very “pedestrian” equipment, such as a 1970s-vintage Heathkit power amplifier, a garden-variety CD player, etc. Both systems were connected via a switcher to the same pair of speakers, so unit-to-unit speaker differences and speaker placement issues didn’t enter the picture at all. …

His results showed that as long as neither system was pushed into distortion there was no statistically-significant preference shown for either system by the group, using a very extensive and varied selection of program material. His listening panel included novice listeners, mild enthusiasts and self-professed “golden eared” audiophiles.”


Thanks, Keith! For a link that backs up the argument with which you are in dispute…
 

ahofer

Major Contributor
Forum Donor
Joined
Jun 3, 2019
Messages
4,952
Likes
8,698
Location
New York City
Scientific evidence that all amps sound the same? Please show me. If you don't have it, then I take it that this is your religious belief that you have arrived at as an article of faith, or dogma, and not science. You are only applying scientific lipstick to dress up what is in essence an unsupported belief. In the meantime, here is another blind test showing that amplifiers can sound different. And here is another. And here is a nice discussion about how and why they can sound different.
A) My claim is that all amps that measure similarly within audible thresholds will sound similar*, don’t invent new and broader claims
B) https://audiosciencereview.com/forum/index.php?threads/catalogue-of-blind-tests.8675/page-7

I rather like the Audioholics link, as it starts with a blind test with the usual null result between a cheapo set and ‘SOTA’ unless driven to clipping. It goes on to describe a bunch of measurable ways amps could sound different. Unfortunately, no tests of those hypotheses, but certainly the clipping characteristics and impedance mismatch are good candidates. But this isn’t particularly relevant, as the test claimed in this thread is operating within spec (no clipping) and, I understand, very similarly-measuring amplifiers.

The 1987 Stereo Review test that I keep mentioning is my fav, because a) it was in the halcyon days when objectives and subjectivists could get together and test a hypothesis and b) I remember my audiophile self bargaining hard to disbelieve it. One can’t prove the point, but it certainly suggests that differences between even very dissimilar amp designs are pretty hard to hear. So much for “night and day”.

*EDIT: I should add that means so similar that they will not be distinguishable in a level-matched blind test.
 
Last edited:

Axo1989

Major Contributor
Joined
Jan 9, 2022
Messages
2,820
Likes
2,816
Location
Sydney
We are all capable of tribalist reflexes at times. Here's one from earlier in the thread:

Humour note: I see the following in the header block of the Audio Science forum on SBAF: “Excessive insistence on only very difficult to implement double blind testing results in an automatic one week ban or worse.

So, real science is banned from their science forum.

Now that poster certainly doesn't lack raw intelligence or reading comprehension ability (so no offence intended) but does arc up from time to time and here chooses to stumble (perhaps rhetorically) over the English words "excessive insistence". Of course SBAF doesn't ban "real" science via that policy.

You could try the corollary here. Subjective description of audio gear isn't banned on ASR, but "excessive insistence" can get certainly you a holiday "or worse". A lot of things are ok if you don't decide to be a boring git about them. That's really all those words mean.

I don't have an issue with humour personally (apply the usual caveats about abusive, racist and -phobic stuff, obviously) so jibes about AmirNADs or SBAF's GONAD dashboard are pretty funny tbh. There's plenty of snark coming from ASR as well as toward it. But we can all take a breather every now and then and not take the audio thing "excessively" seriously.

As for listening to differences between DACS, not my thing, I don't really hear them.
 
Last edited:

Galliardist

Major Contributor
Joined
Jun 26, 2021
Messages
2,558
Likes
3,273
Location
Sydney. NSW, Australia
I've been digging in SBAF. There are some measurements in other threads of the Heresy and 3+ and potentially these are the items in the test.



From what he gives, they don't look so different from the viewpoint of audiblilty. I'm not sure how much weight to give these, so I'll throw the references in for discussion.

In the first thread, I'm going to cherry pick a quote from PurrIn about his measurements (post 6 there):

You guys wanna know what I did different to get the "bad" result? (And really, it's not that bad to be of any concern - last time I checked, I couldn't hear -110db down or 0.0003% distortion). I used plugged the AC wall wart into a power strip that was closer to the AverLAB and kept the power cord tied up with a wire-tie. Note that it would be easy for me to intentionally bork the measurement of a vendor that I disliked.

In light of what he claimed in the blind test, this is quite a confession. I'm surprised that he doesn't show an interest even in a contradiction between this controversial test result and his own previous listening ability. If he's trying to tell us that other things than SINAD can influence the sound, he's actually preaching to the converted here. Maybe part three will be more useful.

It will probably benefit us if we ignore the comments about Doug Self and NwAvGuy and stick to the information relative to the subject at hand.
 

kemmler3D

Major Contributor
Forum Donor
Joined
Aug 25, 2022
Messages
3,022
Likes
5,635
Location
San Francisco
Scientific evidence that all amps sound the same? Please show me. If you don't have it, then I take it that this is your religious belief that you have arrived at as an article of faith, or dogma, and not science. You are only applying scientific lipstick to dress up what is in essence an unsupported belief. In the meantime, here is another blind test showing that amplifiers can sound different. And here is another. And here is a nice discussion about how and why they can sound different.
"technically correct, the best kind of correct".

The first link is a classic "exception that proves the rule", probably the second also. They compared a 10w tube amp to a 200WPC solid state amp on electrostatic speakers.

"All amps sound the same" is a trusim that isn't always repeated with the necessary qualifications - amps that are operating within their design envelope (not clipping or overdriven). A 10w tube amp running electrostats is... probably not that.

In both cases I think they heard differences in "weird" setups / loads, which is expected due to the known characteristics of the amps.

The whole reason that this blind test is supposedly interesting is Pur1n claims to have heard differences that don't fall into the "weird" category, at least, unless he was using some weird headphones, which I am still not sure about.
 

KSTR

Major Contributor
Joined
Sep 6, 2018
Messages
2,690
Likes
6,013
Location
Berlin, Germany
My claim is that all amps that measure similarly within audible thresholds will sound similar*, don’t invent new and broader claims
You see the circular and self-fulfilling properties of this, do you?
Audible thresholds for certain measured properties are established by checking what amount of change it takes to sound different.

The typical set of measurements are not guaranteed to be complete for fully characterizing a device.

Let's again take the example of polarity inversion in a DAC or amplifier. Depending on whom you ask and at what time, you'll likely hear that there is a "mountain of evidence" that polarity flip is inaudible, therefore an inverting amp may be considered fully transparent (and does not show anything in measurements, obviously, as polarity check is routinely omitted -- well, because "we all know it doesn't matter", including the electronic design engineer who did the amp). One's horizon mainly depends on what information bubble one lives in

Plus, we can't prove a negative. One single instance of successful detection of polarity flip is enough. Actually, audibility of polarity flip with normal music signals (so we have some real-life conditions) is well and widely established these days, I can (and did, just some days ago in a private context) provide a quickly selected test track where pretty much anybody should be able to detect polarity flip in an ABX.
Therefore it is completely possible for two "transparently measuring" amps to sound different, transparently measuring depending on which schools their designers and technical reviewers are from.
 

MaxwellsEq

Major Contributor
Joined
Aug 18, 2020
Messages
1,628
Likes
2,427
Amplifiers, how do they work? Do we have any idea?
Human hearing, its limits, do we have any idea what they are?

Hey, I saw an ultraviolet two-headed horse in my living room!

Now, everything known about human vision says we don't see in the ultraviolet range (without help). Everything we know about horses says two heads are...rare.

But unless you show me a paper testing whether someone can see two headed ultraviolet horses, there is no scientific evidence to refute my claim!
1) it is generally accepted that it's hard to prove a negative
2) there is Black Swan theory. Europeans believed all swans to be white, and so swans were defined as always white, until black swans were discovered. There was no physical reason why a swan could not be black.
 
OP
nyxnyxnyx

nyxnyxnyx

Addicted to Fun and Learning
Joined
May 22, 2019
Messages
506
Likes
474
I think there are still uncertain variables in audio and acoustic for one reason or another, so the best practice would be having an open mind for hypotheses and possibilities, even if they seem far-fetched at the moment.
 

SIY

Grand Contributor
Technical Expert
Joined
Apr 6, 2018
Messages
10,386
Likes
24,752
Location
Alfred, NY
I think there are still uncertain variables in audio and acoustic for one reason or another, so the best practice would be having an open mind for hypotheses and possibilities, even if they seem far-fetched at the moment.
Open minds are fine when there's actual evidence.
 

delta76

Major Contributor
Joined
Nov 27, 2021
Messages
1,615
Likes
2,460
I think there are still uncertain variables in audio and acoustic for one reason or another, so the best practice would be having an open mind for hypotheses and possibilities, even if they seem far-fetched at the moment.
give me a peer-reviewed paper published on a reputable journal, I will change my mind in an instant.
 

julian_hughes

Addicted to Fun and Learning
Joined
Aug 23, 2020
Messages
657
Likes
901
I simply took issue with your claim that perception was somehow outside the realm of science.
At no point did I say that or anything like it. When you misrepresent someone else in order to refute an argument they didn't make then that is just boring and dishonest.
 

Newman

Major Contributor
Joined
Jan 6, 2017
Messages
3,454
Likes
4,216
give me a peer-reviewed paper published on a reputable journal, I will change my mind in an instant.
Even then, if the paper contradicts the working hypothesis of the best audio minds, which in turn are based on data that very likely included similar peer reviewed papers that supported the working hypothesis, a single paper that contradicts the working hypothesis will have some more hurdles to cross before the doyens decide the state of the art’s knowledge has been advanced. This happens from time to time.
 
  • Like
Reactions: JRS
OP
nyxnyxnyx

nyxnyxnyx

Addicted to Fun and Learning
Joined
May 22, 2019
Messages
506
Likes
474
Open minds are fine when there's actual evidence.
What's your take about this topic SIY? It seems like you prefer the 'Either crystal-clear irrefutable evidence or discard all hypotheses and theories' school of thought.
give me a peer-reviewed paper published on a reputable journal, I will change my mind in an instant.
I want one too. I hope individuals with sufficient knowledge and resources can make it happen. We are way more resourceful and easier to communicate now than back then.
 

ahofer

Major Contributor
Forum Donor
Joined
Jun 3, 2019
Messages
4,952
Likes
8,698
Location
New York City
You see the circular and self-fulfilling properties of this, do you?
Well, yes. That's one of the reasons I'm skeptical of the result. We have a lot of science around audible thresholds and measurements. The measurement suite was not designed in a vacuum, it has developed around our knowledge of what's audible. This circularity is a feature, not a bug, of my argument.

I don't really understand the relevance of your example - where an amp is setup incorrectly and you can hear the difference. What does that bring to the table? EDIT - Oh I see, you mean absolute polarity flip, not out of phase. Still, that's a measurable difference that can be corrected/adjusted, right?

But your larger point - that our measurement suite may be incomplete - that's quite possible IMO. Which is why it is worth trying to replicate positive results between transparently measured amps. And it is much more likely (again, IMO) that (1)our measurement suite needs to be improved than there is (2) something immeasurable yet audible, which is where the subjectivist often seem to want to extrapolate positive test results. I'm also skeptical of the in-between hypothesis (3) that the amplifier interaction with other components is so chaotic that we can't make predictions of audibility from measurements. Which one of those three represents your view?
 
Last edited:

delta76

Major Contributor
Joined
Nov 27, 2021
Messages
1,615
Likes
2,460
Even then, if the paper contradicts the working hypothesis of the best audio minds, which in turn are based on data that very likely included similar peer reviewed papers that supported the working hypothesis, a single paper that contradicts the working hypothesis will have some more hurdles to cross before the doyens decide the state of the art’s knowledge has been advanced. This happens from time to time.
I meant in this case it's only the audible threshold of amp. previous researches said 85 dB SINAD (IIRC). if new paper says 90 dB, fine, 90dB it is.

of course only if the new paper itself is highly reputable (reputable researchers, reputable journal), not a random test on the internet :)
 

solderdude

Grand Contributor
Joined
Jul 21, 2018
Messages
15,891
Likes
35,912
Location
The Neitherlands
When one knows that different components all have a 'sound'.
When one knows that certain designers have specific knowledge others may not poses.
When one knows that brands have a 'house sound' and this is created by know-how of designers.
When one knows the hearing is much more discriminate than measurements.
When one trusts certain very knowledgeable and experienced people.
Then it is very easy to be certain that all devices sound different simply because they all measure differently and can be designed to do so by competent designers.

This is then further enforced by others stating they can also hear differences.

And even further enforced when one does some comparisons at home with their own ears and gears and also hears differences.

It is not strange that armed with such knowledge it is very comfortable to know for certain that EVERYTHING matters even if so called 'experts' say otherwise.

So.... as long as one knows for certain (not just thinks, believes or suspects) in the certainties as stated above and does also not fully comprehend the science about all things audio related (but knows they know enough about it) then it is extremely easy to agree that audible differences can exist and do exist and that there are always nay-sayers and arm chair experts around that think they know everything just because they know something about electronics or other aspects but trust their measurements more than their own ears.

So... knowing about things and having actual and factual knowledge are not exactly the same thing but one thing they have in common which is knowing they are right.
 
Last edited:
Top Bottom