• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required as is 20 years of participation in forums (not all true). Come here to have fun, be ready to be teased and not take online life too seriously. We now measure and review equipment for free! Click here for details.

Correlating measurements to audibility

Joined
Apr 29, 2017
Messages
62
Likes
30
#1
Hi guys,

I've read from some experts that amps that measure good can sound bad, and amps that measure bad can sound good. Now I don't subscribe to that belief, but based on articles from Stereophile and other online publications the consensus seems to be that measurements aren't telling the whole story.

For more information on this :

https://www.stereophile.com/content/if-either-these-amplifiers-right#IwQJSUKJeSCPLXjZ.99

What are your thoughts on this? My understanding as a laymen, is that measurements can be fairly poor and still be audibly benign if the measurable flaws are below detection thresholds , and they can be near perfect and potentially have audible (or perceptual rather) consequences if the listening methodology is uncontrolled.

Do you agree with Stereophile?
 
Joined
Apr 29, 2017
Messages
62
Likes
30
#2
Open quote : “…Distortion is another controversial one : second harmonic distortion ( a signal at twice the frequency of the stimulus) is also subjectively innocuous. Third harmonic adds sharpness, but at low levels gives the sound incision and a hard edged quality that some like seem to like as it seems to ‘sharpen things up’. Much above this however and the ear starts to hear distortion not as a change in quality of the music, but as a separate interfering signal. Then there’s the modulation pattern of the distortion, or whether it is correlated with the music.

Cartridge mistracking is an interesting as an illustrative example : just before mistracking a pickup cartridge produces lots of distortion, but it isn’t heard as such. As soon as the stylus let’s go, the signal starts to take on the modulation pattern as higher harmonics join lower ones ( that characteristic zzz…zzz) and this the ear hears immediately.

Digital distortion is often uncorrelated with the music and possesses components that may be of fifteen times higher frequency than the stimulus; the ear detects this as something totally removed from the music, as greyness, harsh, hardness and what have you. So there’s distortion – and distortion! So products can measure badly, but sound quite good all the same.

The converse is more difficult to explain : products that measure well and sound bad. That is where argument breaks out! My oft quoted example was one of the first MOSFET amplifiers, launched in the late 1970’s, the Hitachi HMA-7500. This was distortion-free, even at high frequencies, and measured perfectly in every other respect, yet there was no sound quality benefit. The reverse in fact had the a peculiarly unengaging sound, characterized by a lack of stage depth and a slightly opague quality. It was lifeless and unengaging to listen to, one reviewer said amplifiers like this sound “boring” and, in a nutshell, he was right. Yet, it measured perfectly, so here was a total contradiction. Why?

Many people have put forward reasons, but to this day I am unaware of any measurable proof. Some amplifiers today, often based around MOSFET’s, still measure perfectly and yet seem to benefit little, if at all, in sound quality. The general suspicion is that excessive feedback is the culprit, and, in case of the HMA- 7500, poor 1970’s component quality.

These days component quality has improved and so has sound quality, but still it is difficult to measure any difference between then and now, so what is the ear hearing, heaven knows! Thankfully, this situation, where a product measures perfectly but doesn’t necessarily sound so good, mainly affects amplifiers and we just have to live with it. It doesn’t negate the value of measurement, but it does suggest that with amplifiers the measurements we use are best at determining general operability rather than ultimate sound quality.

Unfortunately, whilst this contradiction between measurement and sound quality exists with amplifiers there will always be room for argument, the most common being that our subjective assessment is flawed, not the amplifier. If this was the case then an awful lot of people around the world are deluded. All the same, at Hi-Fi World we endeavour to have more than one person listen to a product, often under different circumstances and always without prior discussion, to ensure there is consistency of view.

Generalising then, if a product measures well then there is a good chance it will at least sound respectable. If it measures badly then the reason why is the issue when it comes to influence upon sound quality. In this case measurement usually warns that sound quality traits are due to measured imperfection, avoiding the unfortunate situation, where a reviewer hears the effect, likes it and declares the product a winner. Doh!
……” End quote. In the last paragraph he opines on the use of room treatments.

Hi-Fi WORLD, May 2007, page 105. Noel Keywood, Publisher 9 and engineer whom measures all the equipment reviewed).
Just another article from an audio journalist (Noel Keywood) who apparently has over 30 years experience measuring gear. I have no idea if his subjective testing was conducted blind or sighted.
 

solderdude

Major Contributor
Joined
Jul 21, 2018
Messages
1,204
Likes
1,862
Location
The Neverlands
#3
Hi guys,

I've read from some experts that amps.....
The word experts perhaps should be written like this ... 'Experts'

Some devices don't measure well but many feel they sound excellent.
Some devices measure really well but some feel they sound like crap.
Some measurements are done poorly.
Measurements could be interpreted incorrectly.
Some specific measurements don't show otherwise obvious flaws that are shown with other measurements.

With measurements a few things need to be in place.
A: They need to be done properly.
B: The proper parameters must be measured.
C: The proper test equipment must be used.
D: The tests need to be done under the right conditions.
E: The interpretation needs to be done correctly.

Fail one of them and drawing the wrong conclusions is easy to do.

In the end what matters is that the equipment that one listens to sounds right to the owner(s)

Has been discussed before here
 
Joined
Apr 29, 2017
Messages
62
Likes
30
#4
Apologies, I didn't know there was an active thread about measurements, I just started a new topic but I never scrolled down to see what other topics were being discussed.
 

March Audio

Major Contributor
Manufacturer
Patreon Donor
Joined
Mar 1, 2016
Messages
3,376
Likes
2,720
Location
Perth Western Australia
#5
First question you need to ask is:

How many Stereophile subjective listening tests are performed blind under controlled conditions?
 
Joined
Jan 17, 2019
Messages
47
Likes
23
#6
Hi guys,

I've read from some experts that amps that measure good can sound bad, and amps that measure bad can sound good. Now I don't subscribe to that belief, but based on articles from Stereophile and other online publications the consensus seems to be that measurements aren't telling the whole story.

For more information on this :

https://www.stereophile.com/content/if-either-these-amplifiers-right#IwQJSUKJeSCPLXjZ.99

What are your thoughts on this? My understanding as a laymen, is that measurements can be fairly poor and still be audibly benign if the measurable flaws are below detection thresholds , and they can be near perfect and potentially have audible (or perceptual rather) consequences if the listening methodology is uncontrolled.

Do you agree with Stereophile?
The results stating that the DAC was so audibly pleasurable versus how bad it performs is just weird. To charge over $1800 for a device with such a cheap DAC chip is also problematic.
 

flipflop

Active Member
Joined
Feb 22, 2018
Messages
293
Likes
229
#7
How many Stereophile subjective listening tests are performed blind under controlled conditions?
Their sister site, innerfidelity, did run a few blind tests during the so-called Big Sound 2015 event. Tyll made a big deal out of it before actually conducting the tests, but when the participants all failed horribly, he decided not to publish the data. Some of the articles do briefly mention the subjects' performance, though. Not surprisingly, they were only able to discern between amps with the HD 800 due to its impedance swing. They failed the amp tests with the planar HE-1000 and couldn't tell the DACs apart with either headphone.
https://www.innerfidelity.com/content/big-sound-2015-participant-report-brian-na-blur
https://www.innerfidelity.com/content/big-sound-2015-participant-report-hands-tyler
 

sergeauckland

Addicted to Fun and Learning
Patreon Donor
Joined
Mar 16, 2016
Messages
776
Likes
1,359
Location
Suffolk UK
#8
I have never, ever, heard a product that measure well but sounded bad. It does require, however, for the measurements to be comprehensive, but if distortion is low, at all frequencies and permitted loads, frequency response flat and noise low, it can't sound bad.

That doesn't of course mean that somebody might prefer the sound of something that measures worse.

S.
 

SIY

Major Contributor
Technical Expert
Joined
Apr 6, 2018
Messages
1,906
Likes
2,876
Location
Phoenix, AZ
#9
Three things that don't exist:

1. Specific examples of devices that "measure well, sound bad."
2. Specific examples of demonstrated sonic differences that don't correlate with measurements.
3. Groupies for guys with elaborate high end hifi systems (caveat being "because of that system," not "because they have lots of money").
 
Last edited:
Joined
Apr 29, 2017
Messages
62
Likes
30
#10
Three things that don't exist:

1. Specific examples of devices that "measure well, sound bad."
2. Specific examples of demonstrated sonic differences that don't correlate with measurements.
3. Groupies for guys with elaborate high end hifi systems (caveat being "because of that system," not "because they have lots of money").
Stereophile is probably the most well known high-end audiophile website that has extensive experience measuring audio equipment. You would think John Atkinson should know that the listening should be done blind to reliably correlate measurements.
 

M00ndancer

Active Member
Patreon Donor
Joined
Feb 4, 2019
Messages
147
Likes
96
Location
Sweden
#12
Stereophile is probably the most well known high-end audiophile website that has extensive experience measuring audio equipment. You would think John Atkinson should know that the listening should be done blind to reliably correlate measurements.
You have to keep in mind that Stereophile (and other publications) rely on the revenue from advertisements and sponsors. If said sponsored and advertised product aren't better than others then they have no business.
 

SIY

Major Contributor
Technical Expert
Joined
Apr 6, 2018
Messages
1,906
Likes
2,876
Location
Phoenix, AZ
#13
You have to keep in mind that Stereophile (and other publications) rely on the revenue from advertisements and sponsors. If said sponsored and advertised product aren't better than others then they have no business.
By contrast, two of my reviews at AX have caused manufacturers to pull ads. Kudos to my editor, who completely stood by me. I guess that's why Atkinson is rich and I'm not. :cool:
 
Joined
Nov 2, 2018
Messages
80
Likes
40
#14
The word experts perhaps should be written like this ... 'Experts'

Some devices don't measure well but many feel they sound excellent.
Some devices measure really well but some feel they sound like crap.
Some measurements are done poorly.
Measurements could be interpreted incorrectly.
Some specific measurements don't show otherwise obvious flaws that are shown with other measurements.

With measurements a few things need to be in place.
A: They need to be done properly.
B: The proper parameters must be measured.
C: The proper test equipment must be used.
D: The tests need to be done under the right conditions.
E: The interpretation needs to be done correctly.

Fail one of them and drawing the wrong conclusions is easy to do.

In the end what matters is that the equipment that one listens to sounds right to the owner(s)

Has been discussed before here
As my dear old daddy used to say, "an 'ex' is a has-been and a 'spurt' is a drip under pressure."
 
Joined
Apr 29, 2017
Messages
62
Likes
30
#16
I was doing some research last night and I came across this interesting article on Stereophile which seems to be at odds with their position as referenced in the article I posted at the start of the thread :

The Carver Challenge.

Bob made his inexpensive solid-state amp sound "as good" as Stereophile's reference tube amp costing several times the price. By simply tweaking his amp using measurements. John Atkinson couldn't hear the difference!

Does anyone know if there are any arguments against the Carver Challenge or is this basically a slam dunk that proves that "But as we approach the 21st century's third decade, the arguments over what sounds musically magic and what measures well continues unabated.—John Atkinson" is utter nonsense?
 

Shadrach

Active Member
Joined
Feb 24, 2019
Messages
216
Likes
273
#17
I was doing some research last night and I came across this interesting article on Stereophile which seems to be at odds with their position as referenced in the article I posted at the start of the thread :

The Carver Challenge.

Bob made his inexpensive solid-state amp sound "as good" as Stereophile's reference tube amp costing several times the price. By simply tweaking his amp using measurements. John Atkinson couldn't hear the difference!

Does anyone know if there are any arguments against the Carver Challenge or is this basically a slam dunk that proves that "But as we approach the 21st century's third decade, the arguments over what sounds musically magic and what measures well continues unabated.—John Atkinson" is utter nonsense?
It's even worse than the Carver Challenge portrays.
There is an interesting challenge by Richard Clark on the net somewhere.
What becomes apparent but not mentioned as much is from this challenge it becomes obvious that some amplifier manufacturers make amplifiers that are deliberately 'tuned' to sound different. This of course makes complete nonsense of the idea of transparency; to be transparent an amplifier must not alter the signal provided by the preceding component in any way. This was obviously not the case in the above challenge nor is it the case in many other amplifiers.
Despite the deliberate 'tuning' of amplifiers the Richard Clark challenge concerning power amplifiers has never been one. There used to be a substantial cash prize involved for anyone who could tell power amplifiers apart in an ABX test. Thousands have take the challenge.
 

MattHooper

Active Member
Joined
Jan 27, 2019
Messages
173
Likes
142
#18
Hi guys,

I've read from some experts that amps that measure good can sound bad, and amps that measure bad can sound good. Now I don't subscribe to that belief, but based on articles from Stereophile and other online publications the consensus seems to be that measurements aren't telling the whole story.

For more information on this :

https://www.stereophile.com/content/if-either-these-amplifiers-right#IwQJSUKJeSCPLXjZ.99

What are your thoughts on this? My understanding as a laymen, is that measurements can be fairly poor and still be audibly benign if the measurable flaws are below detection thresholds , and they can be near perfect and potentially have audible (or perceptual rather) consequences if the listening methodology is uncontrolled.

Do you agree with Stereophile?
I'm quite dubious about the claim of amps that measure well but sound "bad."

But I'm less dubious about the claim amps that measure "poorly" can sound "good."

I have an Eico HF-81 tube amp. If you look at the stereophile review, it sure doesn't measure well. But I sure do find the sound coming through my speakers using that amp sounds "good" to me. I guess I enjoy the possible colorations it adds.
 

tomelex

Addicted to Fun and Learning
Patreon Donor
Joined
Feb 29, 2016
Messages
757
Likes
263
Location
So called Midwest, USA
#19
This is the way I look at this situation. Essentially measurements can only be compared to measurements. Preferences can not be compared to anything. However, those who claim to hear differences between similar designed gear (this is important, similar means enough measurments to characterize the devices under the same load conditions) I would like them to submit to an extensive hearing test. The heating test results will not be good let me tell you.
I remember in my young days listening to systems at the audio saloon and hearing huge screechy highs and the middle aged gentlemen selling the gear just smiling away at how well it sounded, well, how well it sounded to his depressed high frequency hearing. Typically I had to turn the treble down almost as far as it could go before I started smiling.
 

sergeauckland

Addicted to Fun and Learning
Patreon Donor
Joined
Mar 16, 2016
Messages
776
Likes
1,359
Location
Suffolk UK
#20
One transparent amplifier will sound the same as another. If it doesn't, the one or the other can't be transparent. The proof of transparency can be found in a straight-wire bypass test. If an amplifier can be bypassed by a piece of wire, then ipso-facto, it's transparent.

Much more difficult is to make one non-transparent amplifier sound identical to another. That's the clever bit in Carver's work.

If two non transparent amplifers (or even one transparent, one non transparent)are evaluated blind, then which is preferred, is entirely a matter of personal preference, akin to burgundy or bordeaux and tea or coffee.

Transparency can be expected from measurements, but if confirmation is need, then the somewhat tedious straight wire bypass test can be used.

S
 
Top Bottom