Can You Trust Your Ears? By Tom Nousaine

[Jakob1863]

So answer my question, where are the good controlled listening test results showing amplifiers sound different? Once again you offer nothing of substance.

Btw, you avoided carefully to touch the crucial points (i.e. what measurements reflect the "transparency" of an amplifer); i did not ask for a specific amplifier that you would consider to be transparent, although personally i´d consider the mentioned amplifiers as very good, means wrt measured performance and sound quality as well.

I don´t know much about the swedish tests oivavoi mentioned - is there some documentation available - but i´ve said before, that we started with doing controlled listening tests back in the 1980s, after having read some publications from Dan Shanefield and others, who brought the idea to the audio field, and those tests included capacitors, amplifiers, preamplifiers and cd-players as well. Funny that the 1982 article about the different sound of amplifiers (probably due to clipping) was among those articles, but i didn´t remember until amirm wrote about it.

I totally agree with you, that such results aren´t very surprising when using typical electrostatic speakers, given their low efficiency (full range often between 76 dB and 82 dB) and their complex impedance.
But even with normal passive loudspeakers (including two way mini monitors and three way monitors) it was always possible to confirm the sighted impression in controlled blind listening tests. Which doesn´t mean that all amplifiers were vastly different.

But the interesting point was, that beside the usual specs (given by manufacturers) more sophisticated with complex load simulation, additional IMD/TIM measurements did not reflect those perceptable differences.

PS. looking at well documented listening tests for all kind of "debateable effects" there are some to find, which offer evidence that the effects were detectable. Usually there is a correlation between training, experience and accomodation to test conditions and rejection of the null hypothesis. Otoh, as perfect tests are unlikely, there are nearly always some questionable points left.

 

[oivavoi]

I don´t know much about the swedish tests oivavoi mentioned - is there some documentation available -

There's not much information available online, unfortunately. Most of it is in Swedish, and only in print, in the magazine Musik & Ljudteknik. It's not peer reviewed either. But the guys doing the tests have been audio professionals.
http://www.lts.a.se/lts/tidskrift-musik-och-ljudteknik

but i´ve said before, that we started with doing controlled listening tests back in the 1980s, after having read some publications from Dan Shanefield and others, who brought the idea to the audio field, and those tests included capacitors, amplifiers, preamplifiers and cd-players as well. Funny that the 1982 article about the different sound of amplifiers (probably due to clipping) was among those articles, but i didn´t remember until amirm wrote about it.

I totally agree with you, that such results aren´t very surprising when using typical electrostatic speakers, given their low efficiency (full range often between 76 dB and 82 dB) and their complex impedance.
But even with normal passive loudspeakers (including two way mini monitors and three way monitors) it was always possible to confirm the sighted impression in controlled blind listening tests. Which doesn´t mean that all amplifiers were vastly different.

But the interesting point was, that beside the usual specs (given by manufacturers) more sophisticated with complex load simulation, additional IMD/TIM measurements did not reflect those perceptable differences.

PS. looking at well documented listening tests for all kind of "debateable effects" there are some to find, which offer evidence that the effects were detectable. Usually there is a correlation correlation between training, experience and accomodation to test conditions and rejection of the null hypothesis. Otoh, as perfect tests are unlikely, there are nearly always some questionable points left.

Have the results of any of these tests been published anywhere?

 

[Jakob1863]

There's not much information available online, unfortunately. Most of it is in Swedish, and only in print, in the magazine Musik & Ljudteknik. It's not peer reviewed either. But the guys doing the tests have been audio professionals.
http://www.lts.a.se/lts/tidskrift-musik-och-ljudteknik

http://www.lts.a.se/lts/tidskrift-musik-och-ljudteknik

Peer review is imo in our context not so important; i couldn´t find anything about it, although there is a weak memory of a discussion in a ?swedish? forum a couple of years ago, where some tests were mentioned.

Have the results of any of these tests been published anywhere?

No, these were not published.

There once was a (now defunct) swedish website describing their attempt in the evaluation of two cd-players, done by John Stalberg, Thomas Akerlund and Mikael Sundman:

http://web.archive.org/web/20040223234411/www.jrsaudio.se/dbtoncdplayers.htm

 

[oivavoi]

No, these were not published.

And there's the rub... I will take your word for it, as you seem to be very well-versed in both research design and the actual topics at hand. But as long as there's no actual documentation, it can't formally count as evidence. Then it's also difficult to know what the test conditions were, and what the external validity of the tests might be.

 

[Jakob1863]

Of course, although i could supply the additional information needed....

 

[oivavoi]

Of course, although i could supply the additional information needed....

If possible, please do!

 

[fas42]

I´d think it is more like "the recording could be totally sufficient" (provided something really horrible is avoided), but it needs a reproduction system that doesn´t have pronouced weak points and is capable of delivering full bandwidth at raised levels without compression.
Imo these systems are quite rare, but they allow to hear a record and, although recognizing everything that might have gone wrong, nevertheless appreciate the music.

Usually flaws of recordings meat weak points of the reproduction system and the result is unpleasantness; to avoid that we are either trying to process the recording or to find those recording the fits to the weak points....

It´s projected for the future that end points (audio gear) will have built in what is needed to process the input to match the situation, think for example compression needed for reproduction in cars.

Yes, the system has to be competent, and an easy measure of that competence is its ability to reproduce 'difficult' recordings so that one "appreciates the music". Where the audio industry has lost its way is that the latter hasn't been focused on; instead, meeting easy to measure performance metrics has been the goal of many of the companies - which is why the audio scene is such a mess!

A low resolution playback unit such as a tabletop radio can be very satisfying to listen to, because its own sins are not aggressively unpleasant, and it doesn't highlight weak aspects of the recording. Once one attempts to "improve" the quality of playback it's extremely easy to "go off the rails" - an analogy is to take a very lowly, pedestrian motor vehicle design and try to upgrade that basic package to become a very high performance machine; if the engine is too powerful then the unsophisticated suspension is shown to have major limitations, and the braking capability is woefully inadequate. Meaning, at all times the overall package has to be in balance, and if that is done with intelligence then one can always "appreciate the music".

That such systems are very rare, in the high performance sector, is sad - however, my own experiments have shown that it is possible at all price points, that the balance is always achievable. In fact, no processing of the input is required; merely, adequate engineering and implementation of critical areas.

 

[Blumlein 88]

Btw, you avoided carefully to touch the crucial points (i.e. what measurements reflect the "transparency" of an amplifer); i did not ask for a specific amplifier that you would consider to be transparent, although personally i´d consider the mentioned amplifiers as very good, means wrt measured performance and sound quality as well.

I don´t know much about the swedish tests oivavoi mentioned - is there some documentation available - but i´ve said before, that we started with doing controlled listening tests back in the 1980s, after having read some publications from Dan Shanefield and others, who brought the idea to the audio field, and those tests included capacitors, amplifiers, preamplifiers and cd-players as well. Funny that the 1982 article about the different sound of amplifiers (probably due to clipping) was among those articles, but i didn´t remember until amirm wrote about it.

I totally agree with you, that such results aren´t very surprising when using typical electrostatic speakers, given their low efficiency (full range often between 76 dB and 82 dB) and their complex impedance.
But even with normal passive loudspeakers (including two way mini monitors and three way monitors) it was always possible to confirm the sighted impression in controlled blind listening tests. Which doesn´t mean that all amplifiers were vastly different.

But the interesting point was, that beside the usual specs (given by manufacturers) more sophisticated with complex load simulation, additional IMD/TIM measurements did not reflect those perceptable differences.

PS. looking at well documented listening tests for all kind of "debateable effects" there are some to find, which offer evidence that the effects were detectable. Usually there is a correlation between training, experience and accomodation to test conditions and rejection of the null hypothesis. Otoh, as perfect tests are unlikely, there are nearly always some questionable points left.

Specifics of the Swedish AES testing. Their method was as follows.

They developed a load for the amplifier that was very close to the same impedance and reactance of the speakers they were using. I don't know or forget what it was. They then inserted the test amplifier between the pre-amp and their reference power amp, with the simulated speaker load being driven by the test amp. Tapped it for unity gain and fed it onto the reference power amp. The test: did the direct connection sound different compared to having the test amp in series with the actual power amp. Several members (I don't know how many) listened first sighted and made a judgement. Then blind to see if they could reliably detect a difference. Don't know the details of how many trials. Nor do I know a list of all the amps tested by them this way. They also did additional testing to determine what was different, but I can't find any details of that.

In sighted listening only two power amps were judged to not alter the sound. A large Audio Research solid state amp, and the largest Bryston. In blind testing the Audio Research was perceptibly different. The Bryston was barely detected, and their electronic testing with snooping determined why. I seem to recall it involved feedback handling and the input circuit. They suggested Bryston alter those aspects of the circuit slightly which they did. That newer revision amp was undetectable sighted or blind. Bryston uses this design circuit currently.

As for my opinion about what makes an amp transparent, I believe it is actually simple. If all your conventional measures are good under load of the loudspeaker at the loudspeaker terminals you'll not be able to detect it. The most common issue is variable impedance of the speaker reacting with output impedance of the amp to alter frequency response enough to be heard. I believe the second most common issue (though I am less certain of this portion) is amps with different current output capability being driven into regions of slightly increased distortion or different distortion profiles at different points where large reactance levels require much more current from the amplifier power supply. Along with the manner in which various designs recover from such events likely differing.

So get the amps to do 20-20khz within .1 db, distortion of THD and IMD below .1% with low enough noise levels when actually driving the loudspeaker and I don't see how they can or will sound different. When such performance is reached with pre-amps or digital source devices it becomes transparent. Though not in recent years, I have measured a couple dozen amps hooked to my speakers (which varied over the years) just for frequency response at moderate levels using a simple wideband voltmeter and frequency generator at the speaker terminals. That mostly explained sound differences that matched perceptions right there.

In the end if amps sound different it can only be because they present a different signal to the speaker terminals. So it isn't some mystery.

 

[tomelex]

In answer to the OP, you can trust your ears, but I can't trust your ears. I can trust my ears to reveal large enough variations, but when its not readily apparent, you begin to "decide" what you are hearing.

 

[Fitzcaraldo215]

In answer to the OP, you can trust your ears, but I can't trust your ears. I can trust my ears to reveal large enough variations, but when its not readily apparent, you begin to "decide" what you are hearing.

That's close to the way I see it, except I don't believe I can completely trust my own ears, either. Measurements are potentially better, but we just don't have enough of them of consistent quality. So, we wind up having to rely on the far less perfect mechanism of using our own ears. It is often all we have. (Back in my teens, we believed we could get satisfaction by just picking gear with the "best" specs in the Allied or Lafayette catalogs. Yeah, right.)

It is not a huge problem, because when you use your own ears, you have no one to blame but yourself if you are not satisfied with any resulting choices you make. In the end, most differences are not huge these days, and most stuff plays music just fine, anyway, even if it is not quite the perfect choice. You will get used to the imperfections, and you might even come to prefer them over time. And, there is always something else you have not heard or something new coming out. You have to draw the line somewhere.

But, some choices may be a bit more satisfying than others, especially with speakers, so it is worth taking some time and care in listening to see if there is any advantage of one piece vs. another. You need to challenge what you think you are hearing though, because even your own hearing is not that reliable. Home audition of new gear in my own system has taught me a lot, except that is tough to do with bulky speakers, especially for me with 7.1 channels.

But, as I have said before, I am happier than ever with gear mostly 5-15 years old, in spite of the imperfections in the selection process. I do think it has gotten harder to go really wrong with audio products today as opposed to 30,40, 50 years ago.

 

[Jakob1863]

Specifics of the Swedish AES testing. Their method was as follows.<snip>

Thanks for your description; now i remember having something read about it a couple of years ago on a swedish website:

http://www.sonicdesign.se/amptest.htm

and there is even a translation of the Bryston amp review:

http://www.mjaudiolab.pl/images/stories/Bryston/swedish14bsstreview.pdf
(i think the author of both is the same Ing. Ingvar Öhman)

So get the amps to do 20-20khz within .1 db, distortion of THD and IMD below .1% with low enough noise levels when actually driving the loudspeaker and I don't see how they can or will sound different. When such performance is reached with pre-amps or digital source devices it becomes transparent. Though not in recent years, I have measured a couple dozen amps hooked to my speakers (which varied over the years) just for frequency response at moderate levels using a simple wideband voltmeter and frequency generator at the speaker terminals. That mostly explained sound differences that matched perceptions right there.

In the end if amps sound different it can only be because they present a different signal to the speaker terminals. So it isn't some mystery.

From our tests i have to conclude that the list of specs above most likely isn´t yet sufficient to eliminate any sonic differences.

But in any case, i think we can agree, that manufacturer specs quite often were/are not comprehensive and that somebody else has to do additional measurements and in case that wasn´t, a consumer had to do some listening to find the best match.

 

[Jakob1863]

That's close to the way I see it, except I don't believe I can completely trust my own ears, either. Measurements are potentially better, but we just don't have enough of them of consistent quality. So, we wind up having to rely on the far less perfect mechanism of using our own ears. It is often all we have. (Back in my teens, we believed we could get satisfaction by just picking gear with the "best" specs in the Allied or Lafayette catalogs. Yeah, right.)

As you said, measurements are more reliable wrt to known accuracy, replication/repetition and therefore consistency is much higher, but first of all we have find the cause and effect relationship to know what matters and why. Imo we haven´t solved that problem yet for everything.

Trust in our ears is another important points, we can and have to learn about listening for evaluation purposes and i´m sure there kicks some sort of talent in. And imo we are able to learn to handle our not so perfect accuracy, see for example the recording engineers tale that Blumlein88 mentioned some time before. I´m sure we all have heard the story of the guy that optimized the EQ setting for a couple of time only to realize afterwards that the bypass switch was on. Works on other things too, as we probably all know.....

But, nevertheless, recording engineers know about it, but still rely mainly on their ears, as they know humans aren´t trustworthy to 100% but most of time. They are used to work by ears for hours everyday.

It is imo quite often a matter of combination, and vintage gear does not vanish, so we still have to work out what happens.

 

[Cosmik]

As for my opinion about what makes an amp transparent, I believe it is actually simple. If all your conventional measures are good under load of the loudspeaker at the loudspeaker terminals you'll not be able to detect it. The most common issue is variable impedance of the speaker reacting with output impedance of the amp to alter frequency response enough to be heard. I believe the second most common issue (though I am less certain of this portion) is amps with different current output capability being driven into regions of slightly increased distortion or different distortion profiles at different points where large reactance levels require much more current from the amplifier power supply. Along with the manner in which various designs recover from such events likely differing.

So get the amps to do 20-20khz within .1 db, distortion of THD and IMD below .1% with low enough noise levels when actually driving the loudspeaker and I don't see how they can or will sound different.

So the question is: what amplifier configuration or topology achieves what is required? These discussions always make it sound as though designs are ephemeral and stop working after a while, hence the need to be constantly listening, testing, revising, starting new amplifier manufacturing companies. Surely this is such a dull, mundane function, that somebody could just come up with a design that works satisfactorily et voila, problem solved forever.

Did the Quad 405 achieve what is required, in 1975? If so, why are people still re-inventing the wheel? Does a TI Class AB chip for $5 also solve the problem - possibly even better? Does a TI Class D chip do it even better?

Why is this still being discussed?

 

[Jakob1863]

So the question is: what amplifier configuration or topology achieves what is required? These discussions always make it sound as though designs are ephemeral and stop working after a while, hence the need to be constantly listening, testing, revising, starting new amplifier manufacturing companies. Surely this is such a dull, mundane function, that somebody could just come up with a design that works satisfactorily et voila, problem solved forever.

Did the Quad 405 achieve what is required, in 1975? If so, why are people still re-inventing the wheel? Does a TI Class AB chip for $5 also solve the problem - possibly even better? Does a TI Class D chip do it even better?

Why is this still being discussed?

Blumlein88´s list follows that classical black box approach- it doesn´t matter what´s inside if the measured specs are below the list´s numbers- but as described, we have to know if the factual measured numbers are really below in combination with the specific loudspeaker and listening conditions.

Wrt to the Quad 405 i was quite surprised to notice in the 1980s that it wasn´t the best match to the Quad ESL (57) (sighted listening) although expecting it were.....

Amplifier topology is only one area of interest, the transformation into the end audio gear is another and along the line one can do a lot of not so perfect work.

 

[NorthSky]

If we can't trust our ears, which ears are we going to trust?

 

[Cosmik]

... it doesn´t matter what´s inside if the measured specs are below the list´s numbers- but as described, we have to know if the factual measured numbers are really below in combination with the specific loudspeaker and listening conditions.

There is also another angle: Devise a system that reduces the difficulty of the amplifier's job.

 

[Jakob1863]

@NorthSky,

If we can't trust our ears, which ears are we going to trust?

From my point of view, in the long run you can trust your ears, but nevertheless you can be fooled anytime.
It needs practice (as stated earlier, everybody has to learn to listen for evaluation purposes), it needs experience, noone of us knows what a recording should sound like from the beginning and noone knows which level of sound quality is possible overall. And it needs some selfreflection, but again, at lot of people have to rely on their "ears" - means hearing sense - for doing/improving their work and obviously they don´t do wrong most of the time......

@Cosmik,

There is also another angle: Devise a system that reduces the difficulty of the amplifier's job.

Sure...., but to see if it works already sufficiently, evaluation is still needed.

 

[Blumlein 88]

Thanks for your description; now i remember having something read about it a couple of years ago on a swedish website:

http://www.sonicdesign.se/amptest.htm

and there is even a translation of the Bryston amp review:

http://www.mjaudiolab.pl/images/stories/Bryston/swedish14bsstreview.pdf
(i think the author of both is the same Ing. Ingvar Öhman)



From our tests i have to conclude that the list of specs above most likely isn´t yet sufficient to eliminate any sonic differences.

But in any case, i think we can agree, that manufacturer specs quite often were/are not comprehensive and that somebody else has to do additional measurements and in case that wasn´t, a consumer had to do some listening to find the best match.

So which society (secret) do I have to join? Are there member's dues? Will I be shown the secret handshake? So far you have offered a long winded version of , "I don't agree" without offering any details of your methods, how tests were done, why you my list of specs will not eliminate sonic differences. To have good results you would need to have tested amps with speaker loads that meet my described performance envelope, and yet were able to show in a reasonable manner they sound different.

We do agree manufacturer specs are often not comprehensive or done in standard ways. That isn't the same as saying the properly done tests aren't useful.

 

[Blumlein 88]

So the question is: what amplifier configuration or topology achieves what is required? These discussions always make it sound as though designs are ephemeral and stop working after a while, hence the need to be constantly listening, testing, revising, starting new amplifier manufacturing companies. Surely this is such a dull, mundane function, that somebody could just come up with a design that works satisfactorily et voila, problem solved forever.

Did the Quad 405 achieve what is required, in 1975? If so, why are people still re-inventing the wheel? Does a TI Class AB chip for $5 also solve the problem - possibly even better? Does a TI Class D chip do it even better?

Why is this still being discussed?

For the same reasons we don't have one car that solves the transportation problem, or one toothbrush design.

One example if I prefer K-horns a fine 20 watt amp might be all I need. If you have Soundlabs, then you need about 500 watts to have a chance of having all you need. With a K-horn a xmfr coupled amp might be made that doesn't interact with the load. With a Soundlab it will be nearly impossible not to get some resonances between a xfmr and the capacitance of the panel.

Then you have people with funny beliefs. Some believe an SET performs some magic SS amps are incapable of. The truth is the SET colors sound the way they like. Color it that way in the SS amp and you have the same result. I have suggested this to those who like SET sound and constantly struggle to get enough power they feed one into a big quality SS amp and problem solved. I have demonstrated the effect to some people I know. Even after hearing it, experiencing it themselves, they find it hard to just let go and do it that way. The problem seems ego related in males more than females. While females in the audiophile world are rare, they often will experience a demo of something like this and go , "oh, okay that works". They give it no more thought.

Why do we have so many DACs? I have good reason to believe most are fully transparent. I posted an 8th gen copy of some music vs the digital originals. Other than one where I made a mistake cutting the ends off it didn't appear people could hear a difference. I can't. If you can't hear the degradation times 8 you can't hear it times one. The ADC and DAC used didn't cost $1k combined. So why don't we have the one DAC?

In fact history indicates to me that when we approach or reach transparency in a given type of gear the design variations suddenly proliferate in near exponential fashion. When real differences are gone the differences are all in people's minds. At which point the limits are only those of imagination. Tell a good enough story and you can sell a near infinite number of design variations.

There are many examples, I have one particular one in mind. A $500 DAC uses the same DAC chip as a $6k DAC. While there is more than just the DAC chip, the chip used will place limits on the ultimate level of performance possible. It appears both makers are right at those limits. Yet one is a darling held in extremely high esteem and the other is not. It isn't the cheap one that you think would make people happy to have with very nice performance. It is the expensive one that offers just about the same performance.

 

[Blumlein 88]

There is also another angle: Devise a system that reduces the difficulty of the amplifier's job.

Yes, and more effort could have gone into that. One speaker that does this in a way are the Magnepans. They offer a nearly pure resistive load of about 4 ohms. They are inefficient and will require some decent current and power, but no quirky reactance. In the days before they used ribbon tweeters they were even better in that regard.

One way of sidestepping that problem is to use line level crossovers. The amp then feeds one speaker and no high level crossover components. This helps quite a bit. The problem is more marketing that solution. The solution is actually rather obvious in addressing the amp/speaker problem.