Master Thread: Are Measurements Everything or Nothing?

[NicC]

Woohoo, I found my answer on page 1!

Amir writes: I have a scale for how much measurements matter for each category of products:

DACs: 100%
Amplifiers (headphone and speaker): 80 to 90% due to variability of available power. Hard to internalize how much power is available/enough without listening tests.
Speakers: 70 to 80%
Headphones: 50 to 80% (measurements too variable)

 

[westsail42]

Measurements are immensely useful, depending on the importance you place upon them . Measurements are a "data point", reviews are a "data point", your own personal audition is a "data point." But measurements are as close to "objective" (vs subjective) as you can get.

Personally, I use measurements to weed out likely vs unlikely candidates for whatever I am looking for (and depends on type of component). But ultimately it is the audition, whether at dealer or (ideally) in my own system (if it can be returned).

YMMV

 

[MaxwellsEq]

Hi everyone,
As I'm relatively new here, I hope you’ll forgive me if some of my questions come across as basic—or occasionally even a little ‘inflammatory.’ That’s definitely not my intention.

After reading a Stereophile article on cables (discussed in another thread), and some of the follow-up comments—particularly one criticising the so-called 'measurement people'—I started wondering: in which types of equipment are measurements most meaningful?

Has any gear ever measured one way but sounded completely different in practice?

I ask because if measurements have, say, a 99.9% success rate in predicting real-world performance (I made this number up), why would some people still dismiss the science? Is it perhaps because they’ve owned something that measured poorly but sounded fantastic to them? I vaguely recall a test on an expensive pair of HiFiMAN headphones being a bit controversial—something about distortion at high volume levels, if I remember right.

Does the non-measurement ‘mantra’ mean that the subjective listening experience is all that matters? Or are there measurement values that simply aren’t audible?

I'm just trying to expand my understanding—not looking to stir the pot or critique any particular gear. I'm genuinely curious about the relationship between measurements, gear, human hearing, and why some people spurn the science. Is it as simple as some people like pineapple on their pizza and others don't?

Measurements have an extra advantage - they can expose the knowledge and competence of the designer and manufacturer.

You will encounter debates about whether a device with distortion at -70dB will sound worse than one at -90dB. The difference is probably inaudible, but the second device (at -90dB) probably demonstrates better design or manufacturing competence (assuming other measurements are good).

 

[Heinrich]

Hi everyone,
As I'm relatively new here, I hope you’ll forgive me if some of my questions come across as basic—or occasionally even a little ‘inflammatory.’

Yep, all forgiven. And that could be it.

Two examples, where the "audiophiles" are nuts, but still have a point. The "measurement people" seemingly have an easy time to dismiss uninformed criticism. Not that nice, but I understand it may become bothersome to discuss same topics with laymen every other day.

a) Harmonic distortion: irrelevant in today's electronics, and with that notion differences between amps, DACs etc are deemed to be irrelevant too. But the logic is not sound. HD is a proxy only, and not as such the thing that makes the sound bad. A deeper analysis on how HD correlates with perceived sound quality is missing; some say, that HD is not correlated so far, scientists, doctors (e/g Dr Earl Geddes) do that.

b) Spinorama: if I as a consumer measure the direct sound at my (various) listening position(s), and I measure the diffuse field, what do I need the spinorama for? Especially the 'listening window' of the spinorama doesn't make no sense no more. And all the details, with ripples and wiggles in the non-axial response of the speaker, who cares? I've got the grand total of all that already by measurement in my particular situation at home. That is what counts, eventually, not the basically unknowing, only estimating out of statistics, predictions that the spinorama is made for. Still people would say, it sounds bad, bad design etc because the spin is.

I would say, reiterated, all those "audiophiles" are drawn, by the evil advertising, into technical debates, that they don't understand. They ask technicians for advice sometimes, Let's be nice(r).

 

[somebodyelse]

Headphones: 50 to 80% (measurements too variable)

It's not just that the measurements are variable for headphones - their frequency response is modified by the shape of the head and ear, by the fit on or in that ear, and by how repeatably you can get them in the same place. This is why some sites show how much variation there is with multiple rounds of removing and replacing headphones, or variation with something like hair or glasses arms breaking the seal. The same applies to the different shapes of test fixtures, which is why the target curves are specified for a specific test fixture. If you are close to the average then the closeness of measurement to target curve will be quite useful. The further you are from the average, the less useful it will be. Making a system that can easily match the response to the individual will be a significant step forward, especially for those who are far from the average.

 

[MatrixS2000]

I ask because if measurements have, say, a 99.9% success rate in predicting real-world performance (I made this number up), why would some people still dismiss the science?

Usually because they don’t understand it or have not thought through the entire process. One of the best examples of this are measurement deniers who claim that measurements are incapable of capturing everything we hear. However, this perspective overlooks a fundamental truth: if we are able to hear something in a recorded piece, it must have undergone a process of measurement at some point. Sound waves are recorded and stored through precise measurements. When we play back a recording, we are simply experiencing those original measurements transformed back into sound.

 

[voodooless]

why would some people still dismiss the science

Because in terms of audio science, most people live in the dark ages. If they have an ailment, they go for blood letting, if you have a mental problem, you drill a hole in your skull. Obviously, you can also look at the stars and find out what's wrong with you. You could also wear an amulet of dried animal parts for good measure.. Audio is practiced in a similar manner in the 21st century.. It's really idiotic!

 

[Jim Creek]

Measurements define behavior. The ideal behavior is a straight line FR from 20hz to 20 khz with 0% THD and 0db noise. Variation from the ideal is inevitable. Measurements will show us deviations from the ideal and predict behavior.

 

[MattHooper]

Also, while reading reviews and forums I found a lack of consistency, one person described a certain component as thin and bright, somebody else called it warm. If there was some real and repeatable effect, shouldn't listeners impressions be somewhat similar?

Agreed that you can find inconsistency in the reports of audiophiles.

Though, for speakers, if I’m looking for subjective impressions and I find cases where there is a fair amount of consistency between what reviewers are describing and other audiophiles, then I’ve found that helpful,

For me, some qualities aren’t easy to divine from the measurements, and since my interest in loudspeakers tends to be wider than “ASR approved / measured” models, seeing what other audiophiles have to say about speakers I’m interested can be worthwhile.

 

[NicC]

I'm going to make a confession here - I have to copy/paste many responses into ChatGPT to grasp what is being explained But wow, the more interesting audio becomes. When I searched up '0db noise', I was informed that:
"0 dB noise: means no hiss, hum, or static added by the gear. In reality, you can’t get exactly 0 — but closer to 0 is better."
I had no idea systems add noise. I mean, I used to play bass, and many times I could hear a hum in the amp due to a bad lead or such. I'm beginning to think maybe I don't actually hear a lot of the 'points' that get argued sometimes about measurements. Would that be an accurate assessment?
I'd like to add, that learning and digesting on this forum for the last few months led to a purchase (speakers) which utterly transformed my listening experience! Wish I'd done it earlier.

 

[MaxwellsEq]

I'm going to make a confession here - I have to copy/paste many responses into ChatGPT to grasp what is being explained But wow, the more interesting audio becomes. When I searched up '0db noise', I was informed that:
"0 dB noise: means no hiss, hum, or static added by the gear. In reality, you can’t get exactly 0 — but closer to 0 is better."
I had no idea systems add noise. I mean, I used to play bass, and many times I could hear a hum in the amp due to a bad lead or such. I'm beginning to think maybe I don't actually hear a lot of the 'points' that get argued sometimes about measurements. Would that be an accurate assessment?
I'd like to add, that learning and digesting on this forum for the last few months led to a purchase (speakers) which utterly transformed my listening experience! Wish I'd done it earlier.

Google "thermal noise".

It's totally unavoidable. It can be minimized by chilling with liquid nitrogen etc.

 

[rdenney]

Hi everyone,
As I'm relatively new here, I hope you’ll forgive me if some of my questions come across as basic—or occasionally even a little ‘inflammatory.’ That’s definitely not my intention.

After reading a Stereophile article on cables (discussed in another thread), and some of the follow-up comments—particularly one criticising the so-called 'measurement people'—I started wondering: in which types of equipment are measurements most meaningful?

Has any gear ever measured one way but sounded completely different in practice?

I ask because if measurements have, say, a 99.9% success rate in predicting real-world performance (I made this number up), why would some people still dismiss the science? Is it perhaps because they’ve owned something that measured poorly but sounded fantastic to them? I vaguely recall a test on an expensive pair of HiFiMAN headphones being a bit controversial—something about distortion at high volume levels, if I remember right.

Does the non-measurement ‘mantra’ mean that the subjective listening experience is all that matters? Or are there measurement values that simply aren’t audible?

I'm just trying to expand my understanding—not looking to stir the pot or critique any particular gear. I'm genuinely curious about the relationship between measurements, gear, human hearing, and why some people spurn the science. Is it as simple as some people like pineapple on their pizza and others don't?

Humans have an envelope of hearing ability, let's say. Let's also say that envelope is defined by two things: the physical performance of our ears and the interpretive accuracy of the brain that is receiving what the ears detect. It turns out this amorphous blob of detection space is pretty big, but in some dimensions not as big as people think. For example, people argue about differences in distortion that are at 0.001% (-100 dB) or even 0.0001% (-120 dB) of the level of the music playing, because we can measure that. Most people, however, can't hear harmonic distortion at all until it reaches something more like 1% (-40 dB). Even then, most speakers can't delivery 1% distortion at lower frequencies when played loudly enough that the 1% signal would even be audible above ambient noise. Those headphones played at high volumes probably distort to levels several orders of magnitude higher than any of the electronics.

But people delude themselves. They trade in a DAC that produces distortion and noise at a level of -100 dB for a DAC that does so at -120 dB, and then persuade themselves that they can hear the difference.

This is a form of sighted bias, obviously, just as surely as the relative prices, the brand logo on the front of the box, or the length of the hair of the equipment's rock-star designer.

Most people's eyes glaze over when they see SINAD figures and power-vs-distortion charts. And a speaker spinorama, or the frequency response of headphones relative to the Harman curve? Forget it. They are happy to let others do that understanding for them and take their word for it. Or, they frankly hate the very idea of engineering and believe that all experience is aesthetic subjectivity driven by artists rather than engineers. That's just the way people are. But the latter people are much more likely to be taken in by charlatans who know how to manipulate their aesthetic subjective perceptions.

But I'll bet the people here are getting a lot more sound system performance for their money than are the folks pursuing the myth and lore of the usual audio enthusiast groups.

Rick "high-tech versus high-touch" Denney

 

[Romario]

I don't know how to approach this but here I go. I'm not a native English speaking dude so I might come across as blunt as my country is known for.
In posts about amplification an user explains the differences in his experience from one to another amplifier and gets immediately corrected that it could not be the case because FR of all amps are the same. It's all biased and fantasy and whatnot. However, anyone with a few amps at his disposal notices differences between them, big or small, for better or worse. They all measure differently except for FR so why shouldn't they sound differently?
Like the BT codec SBC, it has the same FR as LDAC but the latter has the better resolution. As long the user notices differences and doesn't claim that all other members should do the same why can't people leave that be?

 

[Purité Audio]

It’s the ‘I noticed a difference’ that is nearly always the issue.
Keith

 

[Mart68]

However, anyone with a few amps at his disposal notices differences between them, big or small, for better or worse. They all measure differently except for FR so why shouldn't they sound differently?

Because in most cases the differences in measurements are insufficient to be audible, by a large margin.

 

[Mort]

As long the user notices differences and doesn't claim that all other members should do the same why can't people leave that be?

We do. But there are a multitude of places to do that in the Internet. Go there and have fun. This isn't the place for that.

 

[solderdude]

They all measure differently except for FR so why shouldn't they sound differently?

Because of hearing limits.
We can easily measure 0.001% distortion but cannot hear it.
We can easily measure -120dB noise floor but we cannot hear it.
We can easily measure far below and above the audible frequency range... but we cannot hear it.
We can easily measure small and gentle phase shifts but we cannot hear it.

That's why all devices measure differently but may not be audibly different (when used within their limits).

Like the BT codec SBC, it has the same FR as LDAC but the latter has the better resolution. As long the user notices differences and doesn't claim that all other members should do the same why can't people leave that be?

Codecs like this are lossy and data compressing so there are substantial differences that under the right circumstances can be audible and are very measurable.

 

[Romario]

Because of hearing limits.
We can easily measure 0.001% distortion but cannot hear it.
We can easily measure -120dB noise floor but we cannot hear it.
We can easily measure far below and above the audible frequency range... but we cannot hear it.
We can easily measure small and gentle phase shifts but we cannot hear it.

That's why all devices measure differently but may not be audibly different (when used within their limits).


Codecs like this are lossy and data compressing so there are substantial differences that under the right circumstances can be audible and are very measurable.

But most amps don't measure 120 db, right? Its between 80 and 100 mostly. And crosstalk is also very variable.

 

[Romario]

We do. But there are a multitude of places to do that in the Internet. Go there and have fun. This isn't the place for that.

Thanks

 

[Romario]

So when one exchanges an amp you don't remeasure dsp settings and keep it like it was?