Master Thread: Are Measurements Everything or Nothing?

[krabapple]

Here is the evidence:

At Benchmark we ran an ABX test using a 1 kHz low-level tone. We compared a class AB amplifier to the Benchmark AHB2. The test level was 0.01 watt, producing an SPL of 67 dB at the listening position. The measured distortion produced by the class AB amplifier was reproduced at a level of 14 dB SPL. In contrast, the measured distortion produced by the AHB2 was well below 0 dBSPL (actual level was calculated at -21 dBSPL). Input voltages to the speaker terminals were set to 0.2828 Vrms using a precision volt meter.

Given these well-controlled conditions it was very easy to hear the difference between the two amplifiers in a fully-blind random ABX test. I scored 100% correct on every ABX test sequence I ran. Other listeners achieved similar results.

You can read the entire writeup here: "Power Amplifiers - A "First Watt" ABX Test"

Given the results of this test, there is no question that power amplifiers can sound different even when they have respectable specifications at high output levels. This specific test shows that we should pay more attention to the 0.01 watt THD.

The question was whether we can predict difference from measurements. You found a distinct measurable difference in distortion at very low wattage, and you heard it using a test tone in a blind test. Your result suggests we need to measure more, specifically low wattage THD.

(It does not, it must be emphasized, suggest that 'we can't measure everything we hear' from amps...and I know you have not suggested that, though others have)

Maybe we need to measure more... if that difference proves relevant to normal listening.
Did you compare the two amps blind using music, spoken word or something else more typical of the consumer experience? Did hearing the test-tone difference prime you to perform at p<0.05 with more typical content?

 

[SIY]

Your result suggests we need to measure more, specifically low wattage THD.

I was reminded of the cheat to show that 24 bit can be distinguished from 16 bit encoding by the expedient of recording a tiny signal, then turning the volume way up to hear the noise floor.

This, of course, is worse since it uses a broken unit (in this case, clearly misadjusted bias).

 

[pma]

I measured just one amplifier with terrible crossover distortion, which had specific low power issues. It was based on OPA549

https://www.ti.com/lit/ds/symlink/opa549.pdf

and the results were like this:

I believe it would be a candidate for a positive ABX test result.

 

[John_Siau]

One single number says nothing, regardless it is expressed in % or dB (any technically competent person is able to transfer % to dB and vice versa).

I cannot imagine a competent class AB amplifier with high crossover distortion products at 10mW power. So I would respectfully request for @John_Siau to post a proof in a form of a distortion spectrum graph of the ill behaving amplifier. It is the only way to lead a scientific discussion based on facts and not on the stories based on one’s authority, especially if there is a business interest.

Here are the plots for the offending amplifier (shown in green on all graphs). The second amplifier used in the ABX test was the AHB2 (shown in red on all graphs). These tests plots were made in 2016. The offending amplifier was an RA-500 "studio reference" amplifier. We had two samples that were both purchased new at the time of the test. Both behaved identically. This amplifier was passively cooled and was designed to be rack mounted. I suspect the amplifiers may have been under biased because of thermal issues related to the passive cooling and the need to run in a studio equipment rack. It is also possible that there was some problem at the factory and these units got shipped with incorrect bias settings. I searched our facility and we no longer have these two samples. In any even, the RA-500 is no longer available.

The first plot is an FFT of the 1 kHz, 1Watt output spectrum of both amplifiers immediately after a cold startup.

The vertical axis (dBrA) is dB relative to 1 W.

From this FFT, you can see that the offending amplifier was working properly when stone cold. It did display a fair amount of power supply noise, but probably low enough to be inaudible with the 87 dB speakers used in the test. The 120 Hz and 180 Hz power supply noise may have been slightly above the threshold of hearing, but I specifically do not recall hearing AC line noise.

Also please note that H3 and H5 are too low to reach 0 dB SPL with 87 dB speakers.

Additional notes:

The 1kHz fundamental was removed with the AP2722 notch filter to improve the resolution of the FFT. This was a technique that we had to use with the older Audio Precision test stations. As many of you may know, the APx555 uses this technique internally, but then splices the 1 kHz fundamental back in.

I am trying to locate the FFT that shows the crossover distortion after the amplifier was warm but not overheated. These measurements were made in 2016 and we have since upgraded to APx555b test stations, so these plots may be lost. I will post them if we locate them.

I do recall that the perfect scores on the ABX tests were achieved by listening for harmonics of the 1 kHz tone.

The THD+N vs. Power curves always displayed the effects of crossover distortion. The minimum THD+N of the offending amplifier (red curve) is -74 dBW (at 0.01W), and this is what was measured and documented when the ABX test was run.

Please note that this "malfunctioning" amplifier still met the manufacturer's specifications for THD+N.

Here is the THD vs. Output Power sweep. The THD+N level is -72 dBW which is just 2 dB lower than the THD+N test above. This confirms that the THD+N reading was mostly distortion. Note that the -72 dBW THD would have produced about 15 dB SPL at the listening position. This agrees with our written notes regarding the ABX test. The harmonics caused by the biasing problem were high enough to be audible.

So what we have here is a very specific case of one brand and model of class AB amplifier that easily drifted into an under biased condition. Once it warmed up and entered this under biased condition, THD could be heard on a 1 kHz tone at very low power levels (0.01 Watt) and a listening level of 67 dB SPL (amplitude of 1 kHz tone). The sum total of the harmonics would have reached about 15 dB SPL at the listening location.

Conclusions:

Harmonic distortion at 15 dB SPL was heard when playing a 67 dB SPL 1 kHz tone. This was confirmed with perfect ABX scores from multiple listeners. The high scores indicate that the distortion was easy to hear when using a pure 1 kHz test tone.

The distortion was caused by crossover distortion in the offending amplifier (see green curves). We had two RA-500 amplifier samples and both had the same problem on they each had it on both channels. They were purchased at the same time, so it is hard to say if it was a production problem or a design problem.

The audibility of the distortion was only confirmed on a 1 kHz test tone. This may imply that it could be heard on some music samples, but the answer to this question is well outside of the scope of this simple test. A much more involved test would be required to determine audibility when playing music.

This crossover distortion defect was present in both channels of two band new RA-500 power amplifiers. This problem could have been solved with thermally stable bias circuits, or by proper factory adjustments, or a combination of both. Clearly these two test samples were not thermally stable. For those of us who design power amplifiers, it seems incredible that these amplifiers shipped with this biasing problem. It is hard to know if this is unusual, especially in lower cost class AB power amplifiers. The RA-500 was a low-cost "studio reference" class AB power amplifier.

This is why we need people like Amir and John Atkinson, who have provided us with a large database of tests.

Thanks to everyone who has made comments regarding this ABX test. I appreciate the need to keep the conclusions specific to the test that was conducted.

 

[JP]

Harmonic distortion at 15 dB SPL was heard when playing a 67 dB SPL 1 kHz tone. This was confirmed with perfect ABX scores from multiple listeners. The high scores indicate that the distortion was easy to hear when using a pure 1 kHz test tone

The audio was a pure tone for all the listening tests? Was this disclosed in the marketing blog? Any attempts made with music?

 

[pma]

So this is OPA549 at 10mW/4ohm, but it is an example of the ill-designed chip amplifier unsuitable for audio. The 10mW specific test is not needed, crossover distortion is seen at any signal level. It is a generator of crossover distortion that is visible even with the oscilloscope.

(Edit: THD crossover distortion hump shown in higher X-axis resolution, at 1kHz test frequency)

In case that any commercial audio amplifier shows such crossover distortion, the designer should be punished .

 

[pma]

Here are the plots for the offending amplifier (shown in green on all graphs). The second amplifier used in the ABX test was the AHB2 (shown in red on all graphs). These tests plots were made in 2016.

Thank you for the data provided, I appreciate it.

 

[MAB]

The audio was a pure tone for all the listening tests? Was this disclosed in the marketing blog? Any attempts made with music?

Pure tone, 1kHz only is mentioned:

With an audio analyzer we can extract the error waveform (lower trace) and amplify it so that we can see what our ears are hearing. In this case, the error signal has been amplified by 1024 (about 60 dB). At a 1 watt output, this class-AB amplifier produces a distortion waveform that measures 70 dB below the output level of the 1 watt test tone (see graph at the bottom of this application note). This means that the power produced by the THD+N is 70 dB below 1 watt. If we drop the level of the test tone by 20 dB, the output power is 0.01 W. At this output level the amplifier was still producing distortion at a level of 73 dB below 1 watt. At 0.01 watt the distortion waveforms look virtually identical to the 1 watt waveforms. This distortion was clearly audible when we used this amplifier to drive a speaker at 0.01 watt. For our tests we used a stereo pair of Benchmark SMS1 speakers with a sensitivity of 87 dB 1 watt, 1 meter. The tone was reproduced at a sound pressure level of about 67 dB (measured 67 to 68 dB SPL) at the listening position while the amplifier distortion was reproduced at a calculated sound pressure level of about 14 dB (87 dB - 73 dB = 14 dB). With a 0.01 watt 1 kHz test tone, the amplifier distortion was clearly audible through the loudspeaker.

Power Amplifiers - A "First Watt" ABX Test

In Benchmark's listening room we recently demonstrated the importance of the first watt using two 100 watt stereo power amplifiers. One amplifier was a traditional class-AB amplifier, the other was Benchmark's AHB2 power amplifier with feed-forward error correction. Using a double-blind ABX...

benchmarkmedia.com

It's a legit test result, even if the stats aren't presented. I now understand. And am slightly intrigued as to if I could identify, likely many of us would be able to unless new revelations occur.

So what we have here is a very specific case of one brand and model of class AB amplifier that easily drifted into an under biased condition. Once it warmed up and entered this under biased condition, THD could be heard on a 1 kHz tone at very low power levels (0.01 Watt) and a listening level of 67 dB SPL (amplitude of 1 kHz tone). The sum total of the harmonics would have reached about 15 dB SPL at the listening location.

This was an important revelation that gives context, and I now understand. I certainly didn't before.

FWIW, I posted some time ago a defense of good performance at low levels. I took microphone measurements of several amps from a vintage micro integrated, to a Hypex and a PuriFi, all hooked up to a high sensitivity compression driver. I ranked the amps by measured noise, after carefully controlling the measurement process, which I documented, including the condition and age of the amps. One was a 20+ year old Bryston 3B ST that I use daily. The point was to demonstrate amps do sound different, but I was also careful to point out the corner case aspect of my study, where it is applicable, and where it is not. I did note the old Bryston was fairly quiet but sounded harsh compared to the a couple less quiet amps. I didn't quantify that though. For sure you have made me wonder about the bias, and perhaps I need to do a thread about proper bias, and what happens when you have an amp that has been allowed to drift too low in bias. But who knows what I will really find in the old Bryston, it was directly hooked to a compression driver close mic'ed...

Thanks for the info, it does help my understanding of your test result, and the state of amplifier you used.

 

[Nuni]

I don't know what kind of hack you are, it's true, but also irrelevant and I don't care.

My question "So you are saying you're clueless about science?" was prompted by this gem of yours:


You think that's a scientific approach?

Hack? At least I can post in meaningful English which appears to be more than you can do.

 

[BDWoody]

Hack? At least I can post in meaningful English which appears to be more than you can do.

Ok, we've prolonged the agony enough...

You can move along now.

 

[Jim Taylor]

Hack? At least I can post in meaningful English which appears to be more than you can do.

A few pieces of advice:

1) Focus on the problem ... and the problem only. Discussions that get sidetracked tend to dilute the quality of information.

2) If someone posts something that irritates you, simply ignore it. Remember; " .... just because a dog barks at you doesn't mean that you have to bark back."

3) Emotion allows people to manipulate you. It is, therefore, a weakness. The more emotional your responses, the more obvious your weaknesses.

Jim

 

[AdamG]

Emotion allows people to manipulate you. It is, therefore, a weakness. The more emotional your responses, the more obvious your weaknesses.

I’m getting all emotional about the wisdom bomb you just dropped. Now my eyes might be leaking because this brilliant post will not get a response. A few clicks too late for the visiting Troll. But the rest of us can benefit.

On a side note. We just cracked 57,000 Members!

 

[SIY]

We just cracked 57,000 Members!

Do you need some Krazy Glue?

 

[Jim Taylor]

... the wisdom bomb you just dropped.

Ain't no wisdom bomb. Everything I learned, I learned the hard way, not from reading Plato. If I hit my head on the beam enough times, I learned to duck.

Jim

 

[Blumlein 88]

Ain't no wisdom bomb. Everything I learned, I learned the hard way, not from reading Plato. If I hit my head on the beam enough times, I learned to duck.

Jim

Sadly I see way too many who flunk out of the school of hard Knox these days.

 

[DonH56]

On a side note. We just cracked 57,000 Members!

I'm pretty sure a lot of us were already cracked...

 

[Newman]

2) If someone posts something that irritates you, simply ignore it. Remember; " .... just because a dog barks at you doesn't mean that you have to bark back."

Clearly, Jim, you are not a dog living near to me!

 

[pma]

Here are the plots for the offending amplifier (shown in green on all graphs). The second amplifier used in the ABX test was the AHB2 (shown in red on all graphs). These tests plots were made in 2016. The offending amplifier was an RA-500 "studio reference" amplifier. We had two samples that were both purchased new at the time of the test. Both behaved identically. This amplifier was passively cooled and was designed to be rack mounted. I suspect the amplifiers may have been under biased because of thermal issues related to the passive cooling and the need to run in a studio equipment rack. It is also possible that there was some problem at the factory and these units got shipped with incorrect bias settings. I searched our facility and we no longer have these two samples. In any even, the RA-500 is no longer available.

I have tried to replicate your test, with 2 class AB amplifiers, A250W and OPA549. The later has quite horrible crossover distortion, the former is free of low power crossover distortion. The THD vs. power plots, scaled in dBW as you did, look like this:

For those less familiar with dBW scale I am also posting the more usual dBr scale related to base frequency amplitude:


10mW spectra (2kHz tone) look like this:


Now, for those interested in possible sound difference with the 1kHz sine signal recorded from both amplifiers at 10mW/4ohm, below you will find the zip file with 2 flac files. Please feel free to make a foobar ABX test and tell me if you can hear the difference, and possibly post a test report.

Note: test files (1kHz) are reduced to 16bits/44.1kHz, thus lower "ENOB"


Edit #2: generated 1kHz sine tone added for possible ABX comparison

 

[restorer-john]

Alesis RA500.

Schematics attached...

Built by Homni : https://www.homni-elec.com/facility/16.htm

 

[John_Siau]

The audio was a pure tone for all the listening tests? Was this disclosed in the marketing blog? Any attempts made with music?

Yes, here is the description from our blog:

At a 1 watt output, this class-AB amplifier produces a distortion waveform that measures 70 dB below the output level of the 1 watt test tone (see graph at the bottom of this application note). This means that the power produced by the THD+N is 70 dB below 1 watt. If we drop the level of the test tone by 20 dB, the output power is 0.01 W. At this output level the amplifier was still producing distortion at a level of 73 dB below 1 watt. At 0.01 watt the distortion waveforms look virtually identical to the 1 watt waveforms. This distortion was clearly audible when we used this amplifier to drive a speaker at 0.01 watt. For our tests we used a stereo pair of Benchmark SMS1 speakers with a sensitivity of 87 dB 1 watt, 1 meter. The tone was reproduced at a sound pressure level of about 67 dB (measured 67 to 68 dB SPL) at the listening position while the amplifier distortion was reproduced at a calculated sound pressure level of about 14 dB (87 dB - 73 dB = 14 dB). With a 0.01 watt 1 kHz test tone, the amplifier distortion was clearly audible through the loudspeaker.