• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Double Blind tests *did* show amplifiers to sound different

SIY

Grand Contributor
Technical Expert
Joined
Apr 6, 2018
Messages
10,386
Likes
24,749
Location
Alfred, NY
I'm sure opinions are all over the place on almost any issue. I had a colleague that did very extensive study on dissapation factor which was measurement driven. Unfortunately he passed away before it could be published.

These are my data, not my unsupported opinions.

DF is a different matter than DA, but equally unimportant for coupling caps.
 

MakeMineVinyl

Major Contributor
Joined
Jun 5, 2020
Messages
3,558
Likes
5,871
Location
Santa Fe, NM
These are my data, not my unsupported opinions.

DF is a different matter than DA, but equally unimportant for coupling caps.

I also meant DF.

More to the point, as a manufacturer, you have to realize that to some customers, component choice matters (within reason), and sales could be lost to people who believe that's important. But you're never going to lose sales because someone says "you use components which are too good, so I'm not going to buy"! Real world and all that ;)
 

SIY

Grand Contributor
Technical Expert
Joined
Apr 6, 2018
Messages
10,386
Likes
24,749
Location
Alfred, NY
I also meant DF.

More to the point, as a manufacturer, you have to realize that to some customers, component choice matters (within reason), and sales could be lost to people who believe that's important. But you're never going to lose sales because someone says "you use components which are too good, so I'm not going to buy"! Real world and all that ;)

Yeah, I mentioned in my blogs that there's a lot of sociology behind this. It's disappointing, but you're right, it's still real.
 

sergeauckland

Major Contributor
Forum Donor
Joined
Mar 16, 2016
Messages
3,440
Likes
9,100
Location
Suffolk UK
I also meant DF.

More to the point, as a manufacturer, you have to realize that to some customers, component choice matters (within reason), and sales could be lost to people who believe that's important. But you're never going to lose sales because someone says "you use components which are too good, so I'm not going to buy"! Real world and all that ;)
Quad in the days of Peter Walker was very much like that. PW only ever used components that met the specification required, and was criticised in the HiFi rags for not using the flavour-of-the-month capacitor or resistor. A whole industry sprung up, largely still going, modifying Quad amps to put back all the goodness PW left out. :facepalm:

Much the same thing happens with loudspeakers where a specific marketing point is made of the capacitors used in the crossover, and providing bi-wire terminals.

S.
 

MakeMineVinyl

Major Contributor
Joined
Jun 5, 2020
Messages
3,558
Likes
5,871
Location
Santa Fe, NM
Quad in the days of Peter Walker was very much like that. PW only ever used components that met the specification required, and was criticised in the HiFi rags for not using the flavour-of-the-month capacitor or resistor. A whole industry sprung up, largely still going, modifying Quad amps to put back all the goodness PW left out. :facepalm:

Much the same thing happens with loudspeakers where a specific marketing point is made of the capacitors used in the crossover, and providing bi-wire terminals.

S.
There's a point of balance. $100 capacitors are beyond The pale!
 

Koeitje

Major Contributor
Joined
Oct 10, 2019
Messages
2,292
Likes
3,880
And if you can't hear a difference then you're not an audiophile, i.e. you don't have good ears or the right speakers or the a good enough source and blah blah blah.......
Yeah, I ran into this argument a couple of days ago. Some dude literally started asking what kind of speakers and gear I had and that he has no problem spending thousands for a small improvement instead of just responding to an argument....

For a lot of people this hobby is more like a dick measuring contest than anything else.
 

Sal1950

Grand Contributor
The Chicago Crusher
Forum Donor
Joined
Mar 1, 2016
Messages
14,073
Likes
16,609
Location
Central Fl
ATI does not have an ABX switching device because we're not trying to "prove" anything beyond being confident that amplifiers which go out the door sound right. Subjective? Off course. But to not go thorough this step would be a disservice to the purchaser.
I'm sure you've also done detailed measurements of the final product.
If so, you then can be confident that the product will "sound right" when it arrives in the customers hands.
More to the point, as a manufacturer, you have to realize that to some customers, component choice matters (within reason), and sales could be lost to people who believe that's important.
Sure, right along the lines of the anti-MQA DAC designer who includes MQA so as not to lose any potential sales.
 

MakeMineVinyl

Major Contributor
Joined
Jun 5, 2020
Messages
3,558
Likes
5,871
Location
Santa Fe, NM
I'm sure you've also done detailed measurements of the final product.
If so, you then can be confident that the product will "sound right" when it arrives in the customers hands.

Sure, right along the lines of the anti-MQA DAC designer who includes MQA so as not to lose any potential sales.
Like a new car design; do the math and measurements, then do rigorous test driving to prove it out. The alternative is to not do that......
 

b1daly

Active Member
Forum Donor
Joined
Dec 15, 2018
Messages
210
Likes
358
It is a bit more nuanced than can or cannot hear a difference and not as definitive one way or the other.

There are two major aspects that can differentiate amps - distortion products and tonal balance. Taking extremes of these is audibly different to anybody but the deaf. So, the question is when do those differences become hard to detect. There are some minor factors like damping factors, slew rate, etc., whose effects are harder to measure and predict the effect of.

If distortion products are measured with the kind of measurements Amir is doing and two amps are both well below hearing thresholds, then it would be reasonable to conclude that anything you hear from them will not be different for distortion product reasons. But this comes with a strong caveat which is that the distortion products will not vary too much from the measured values when a real-life load of a speaker is introduced. AFAIK, the jury is still out on whether that caveat is a given. But there are strong opinions on it. Also, a clean amp in that fashion does not necessarily imply one will subjectively like the sound it produces.

If the distortion products differ in their dispersion and get higher than the absolutely inaudible range (and there is a large grey area there especially combined with the caveat above), then the case for saying they are not audibly different becomes less convincing. Some amps suppress high order harmonics but let the second and third get high. Some keep the second and third harmonic not get too big but have a lot of higher order harmonics. Yes, all of these will get counted in the distortion numbers but distortion numbers is not correlated necessarily with SQ. Two amps with the same distortion numbers may have different distortion characteristics that are audibly different and hence differentiable. It could happen that subjective preferences prefer one over the other within such differences.

Tonal balances also work the same way. A single frequency sweep across the audible bandwidth and beyond can establish the behavior across frequencies and IMD tests allow limited inferences when multiple tones exist. Even in this case, if two amps show performance without any effects that are in the audible range, then it supports the likelihood that they are indistinguishable again with caveat above on how well this models real music on real speaker loads.

Some of the amps through history did not aim for complete and clean transparency. They made decisions for getting a certain "sound" either via their approach in shaping the distortion dispersion or affecting the tonal balance. Will these measure very well in these tests, most likely not. But will people like what they hear and pay a lot of money for some of them sure. Eventually, the world is divided into two camps.

Those that like to hear the sound the way they enjoy it viscerally and those that cannot enjoy the sound unless it measures clean.

This gets into debates somewhat like the Bogleheads groups on active vs passive investing. There are Bogleheads who believe an active fund just cannot beat an index if the test for performances and portfolios were "properly designed". But any such test would, in practice, constrain the active manager to buy exactly from the set of stocks that the index fund has and so would not show any consistent out-performance.

Purists here will not accept any claims of difference unless it is done under conditions that are in practice very difficult to set up and do. So, it maintains the divide with no resolution.
I don't think many commercially available amps have been voiced to deviate from flat frequency response and low distortion. Some of the boutique tube amp makers claim this, but I've never heard of a solid state amp that aimed to deliberately introduce distortion.
 

b1daly

Active Member
Forum Donor
Joined
Dec 15, 2018
Messages
210
Likes
358
I refreshingly straightforward and common sense approach.

As for blind ABX testing, for a start it's not what audiophiles (or hi-fi enthusiasts) typically do to evaluate equipment. So what? many will say. However after that, the more blind ABX is constrained to ensure the repeatability and legitimate scientific rigors, less like the real listening experience it is -- i.e. the test results are valid only under the specific conditions of the test and cannot be generalized to all circumstances.
I think you are confused. The ABX test is the easy environment in which to distinguish one amp from another. If this test is failed it defies common sense that there will be actual perceptible differences that will be discovered in everyday use. The latter presents profound obstacles to making any kind of comparison between audio gear that has similar performance specs.

--------------
There are well established reasons for why you perceive that amps sound different. In almost all real world playback environments two amps will never sound exactly the same.

What people don't acknowledge is that the perceptual biases that cause us to hear audio differences when none truly exist is that they actually do make things sound different. The perception of difference is not an 'illusion' or an 'imagined perception' it is a real, actual perceptual difference.

Even playing back the exact same passage on the exact same system will lead to different perceptions of sound. Our minds are constantly adjusting our perception and focus.

Couple this with the many objective reasons why comparing two amps in the real-world will lead to the conclusion that they sound different:
- Unless the amps have some kind of super calibrated gain structure, playbacks levels between amps will be actually different
- the problem of level matching is compounded by the different gain structures and resolution of the volume knobs.
- one amp might not be operating to spec
- tube amps or vintage amps can have a non-flat frequency response
- if tone controls are available on either amp, this can seriously compound efforts to understand how the amp sounds.

-----------------

This discussion is the 'vampire debate' of audio and it's about as settled as anything can be that competently designed amps are sonically in-distinguishable under specified conditions. This has been proved over and over with ABX testing.

To decide whether the results of such ABX testing are meaningful to us, potential user of an amplifier, requires interpretation and an analysis of how the conditions for these ABX tests relate to 'real-world' listening concerns.

Richard Clark ran a relatively famous $10k challenge for anyone who can distinguish two amps in a blind ABX test. Nobody ever claimed the prize after 100s of iterations. (If you've never read about this, it is truly a fascinating experiment worth learning about https://www.stevemeadedesigns.com/board/topic/193850-richard-clark-10000-amplifier-challenge/)

He had some strict conditions (strict in the sense that they must be met, not that they imposed unrealistic expectations on the amps or listeners)

- amps must be commercially available from reputable companies
- amps must be working (functioning at spec)
- test is performed on dynamic speakers (as opposed to something like an electrostatic design)
- amps level must be kept under either clipping or 2% THD 20Hz to 10kHz
- must be level matched to .05 db (on both speakers)
- absolute polarity and L/R are consistent between amps
- if the amps have a different frequency response then one must have some EQ applied to make them the same
- amplifiers must be brand name, standard production, linear voltage amplifiers
- program material must be commercially released music

Within these constraints are a wide range of possibilities and parameters under which the test could be conducted!

Contenders could:

- pick any two amps to compare (including car amps, tube amps)
- use any speakers
- use any music
- control the level of the playback
- can take as long as they want and switch between amp A and B as many times as they want
- if EQ is applied to equalize frequency response they can pick which amp gets the EQ

----------------------

The question for a potential user of an amp is: do the conditions and results of a test like the Richard Clark $10k Amp Challenge provide meaningful guidance on the selection of an amp suitable for purpose?

I argue that this provides a ton of useful, practical information for a consumer. It eliminates the worry about aspects of amp performance that are very difficult to impossible to evaluate in normal listening environment.
 

Vasr

Major Contributor
Joined
Jun 27, 2020
Messages
1,409
Likes
1,923
I don't think many commercially available amps have been voiced to deviate from flat frequency response and low distortion. Some of the boutique tube amp makers claim this, but I've never heard of a solid state amp that aimed to deliberately introduce distortion.

Parasound Halo Series, Pass Labs, PS Audio, Nu Prime, ...

But these aren't deliberate distortion, gross FR like you may be thinking. The sound shaping is more subtle. Squashing certain harmonics, slight rolling off at the high end, things like that.
 

Sal1950

Grand Contributor
The Chicago Crusher
Forum Donor
Joined
Mar 1, 2016
Messages
14,073
Likes
16,609
Location
Central Fl
Like a new car design; do the math and measurements, then do rigorous test driving to prove it out. The alternative is to not do that......
A very PC, good marketing type answer. ;)
 

March Audio

Master Contributor
Audio Company
Joined
Mar 1, 2016
Messages
6,378
Likes
9,317
Location
Albany Western Australia
Yes, agreed, out perceptions can and often will be influenced by our presumptions and prejudices. (That may well apply to objectivists whose prejudice is that there are no sound differences and, accordingly, they don't hear any).

I don't claim to be free of my own presumptions, but they don't explain how often I am surprised by the difference I hear comparing equipment that are fairly often they are different from what I expected.
I dont see how "being surprised' is any mitigation for the issue of cognitive bias.
 

March Audio

Master Contributor
Audio Company
Joined
Mar 1, 2016
Messages
6,378
Likes
9,317
Location
Albany Western Australia
Thank you for reiterating you dogmatic opinion, SIY.

As I said in my reply to March Audio, it would be easier for me to believe that my perceptions were entirely due to presumptions and biases if my perceptions didn't differ so often from what I anticipated.

See, you are doing precisely what I described above. You are choosing to ignore proven science because it doesnt fit in with your personal views.
 

March Audio

Master Contributor
Audio Company
Joined
Mar 1, 2016
Messages
6,378
Likes
9,317
Location
Albany Western Australia
I refreshingly straightforward and common sense approach.

As for blind ABX testing, for a start it's not what audiophiles (or hi-fi enthusiasts) typically do to evaluate equipment. So what? many will say. However after that, the more blind ABX is constrained to ensure the repeatability and legitimate scientific rigors, less like the real listening experience it is -- i.e. the test results are valid only under the specific conditions of the test and cannot be generalized to all circumstances.
Indeed, so what?

Im afraid these are all the typical audiophile excuses to justify your personal view.

If you do the 2 most basic things - accurately match volume and not know what kit is playing then you are a long way there. Nothing thats too difficult to do at all.
 
Last edited:

March Audio

Master Contributor
Audio Company
Joined
Mar 1, 2016
Messages
6,378
Likes
9,317
Location
Albany Western Australia
Thank you for reiterating you dogmatic opinion, SIY.

As I said in my reply to March Audio, it would be easier for me to believe that my perceptions were entirely due to presumptions and biases if my perceptions didn't differ so often from what I anticipated.
Its not dogmatic, its proven science.

I have tested many audiophiles and the moment you introduce the most basic of controls, such as accurately matching volume and not knowing which kit is playing, and the allegedly obvious differences they could hear when sighted evaporate. Thats not to say everything sounds the same, it doesnt, but more often than not differences can disappear into insignificance.
 

Casey Leedom

Member
Joined
Dec 7, 2019
Messages
69
Likes
31
Location
Palo Alto, CA, USA
So where does "Damping Factor" play into all of this? [[ Note: This is a real question from someone who doesn't know the answer. I.e. I'm not trying to "Lead The Witness" into a desired answer, etc. ]] Say, given two Solid State Amplifiers which were identical in all other measured parameters but had radically different Damping Factor, would that be audible with speakers that had a lot of Bass EMF?

Casey
 

Vasr

Major Contributor
Joined
Jun 27, 2020
Messages
1,409
Likes
1,923
So where does "Damping Factor" play into all of this? [[ Note: This is a real question from someone who doesn't know the answer. I.e. I'm not trying to "Lead The Witness" into a desired answer, etc. ]] Say, given two Solid State Amplifiers which were identical in all other measured parameters but had radically different Damping Factor, would that be audible with speakers that had a lot of Bass EMF?

Casey

The Benchmark tech guy had a great blog on this topic in one of the threads. Search for it. My takeaway from that was that certain minimum damping factor was necessary to compensate for the impedance of speaker wires (from long lengths or thin guage) and beyond that it didn't male much of a difference. The theory is that poor damping capability can muddy the sound as continuing vibrations from one note merges into another rather than dying off quickly. I may have the understanding wrong or over-simplifying it.
 

SIY

Grand Contributor
Technical Expert
Joined
Apr 6, 2018
Messages
10,386
Likes
24,749
Location
Alfred, NY
Parasound Halo Series, Pass Labs, PS Audio, Nu Prime, ...

I only have first-hand experience with Halo. I would suggest that there's no sound shaping of any sort.
 

FeddyLost

Addicted to Fun and Learning
Joined
May 24, 2020
Messages
752
Likes
542
About Richard Clark's test.
I think that even with equalization (as FR can be significantly non-linear with real tough load) it will be possible to distinguish AHB or good Hypex-based amp from something like that
https://www.stereophile.com/content/borderpatrol-p21-exd-power-amplifier-measurements
Especially on some "bad loads" with low distortion like Magico floorstanders in well-treated room with low RT and ambient noise levels.
Maybe even with cheaper Revel floorstanders with sufficiently low distortion and IMD.
Actually, as a moderate metalhead, i really don't understand when reviewers write about good timbral rendering and microdynamics (!) with amps having -50 db IMD at 1w into resistor and allover performance like quoted earlier.
Maybe they don't listen anything like i.e. Behemoth - Conquer all, but if any of them will try to use test tracks like this (with dense spectre, compression and good rhytmic correlation) on tube amps, results might be ... somehow disappointing.
While i was writing this, i had an interesting question: how bad will look measurement "from input to listening point" in case of different systems and what is best result possible with moderate SPL (below 100 db peak)?
I suppose numbers might be terrific ...
 
Top Bottom