Master Thread: Are Measurements Everything or Nothing?

[MattHooper]

Clean measurements is what assists me to "like" a device... then I know my device chain is as transparent as possible. I am then at rest to enjoy the system and the music that ensues.


JSmith

That’s the type of approach I was referencing that makes a lot of sense and it really works for some people, especially for ASR members.

I have long given up trying to identify differences between competently designed electronics after they deliver on some basic performance parameters. I spent weeks agonizing whether to keep a Benchmark AHB2 or a NAD M22 amp. It was a silly and unrewarding waste of time. Never again.

I don’t really give much thought to solid state amplification for a similar reason. My Benchmark LA4 is simply transparent - objectively so via measurements - which is what I wanted, and that’s that.

Intellectually, I know I could’ve found plenty of similarly, transparent solid state preamplifiers that were cheaper. But I really appreciated the level of engineering, as well as the really amazingly low distortion measurements (it had certain features I wanted as well in the preamp), and there was a certain satisfaction and peace of mind in buying the benchmark gear (which is also why I use their DACs). If I want a bit of coloration, I insert my CJ tube preamp into the chain

 

[Mart68]

I am the exact same Pablolie. I seriously attempted differentiating gear that is not different to the human ear. Now I just purchase based on warranty terms, availability, features and cost. I enjoy the gear now and do not obsess over miniscule sound variations.

Likewise, if ever there was a way to kill the enjoyment, it's that. I just bought stuff I liked the look of and guess what it's fine.

My speaker choice though was 50 percent head / 50 percent heart.

 

[MattHooper]

. I enjoy the gear now and do not obsess over miniscule sound variations.

Likewise, if ever there was a way to kill the enjoyment, it's that.

I totally get that.

But whether you enjoy obsessing over the (in the big picture) tiny differences he’s going to vary between folks.

To a certain degree, it’s the job of an audiophile - or any hobby enthusiast - to care about “the little things” (that’s why for instance you have debates here over even small frequency deviations in measurements, or other aspects of performance, and you’ve got people who have gone to significant length trying to iron out the last wiggles in there room measurements that would leave most normal listeners scratching their heads).

I don’t mind obsessing over the small details because I can enjoy it when I do, and I often appreciate the result of that attention.

In the past couple years I did some
“ obsessing” over certain subtle sonic differences I wanted to achieve, and the end result is that I continue to enjoy the end results.

 

[Heinrich]

To be clear, this applies to speakers - where there assuredly are measurable and more importantly audible tradeoffs that must be made - and not to electronics, where unless you specifically want an effect cleaner is better.

IM on speakers is not measured enough, and the only person doing it with any sort of meaningful rigor that I know of is Erin. It's a complex metric to measure because it (shocker) is affected by frequency response.

Of course the amp/ isn‘t a problem anymore today, same with DACs. Erin shows IM measurements which underline their relevance. Still the industry, objectivist forums included, seems to ignore it. This is a case where „the data“ is incomplete, giving some wiggle room for a subjective evaluation. A case where standard measurements don‘t tell it all.

Another example is Wilson Audio‘s tunetott, showing data that actually hurt, design flaws galore. In glaring contrast to its appearance on paper it got the verdict „ enjoyable“ on this board, based on subjective evaluation.

Coming back to electronics, well, a preference driven evaluation, what could it show? The race for SINAD maybe fun, but doesn’t help with rational decision making. And we had the case, that the quite relevant phono preamps‘ input noise current(!) isn‘t covered well in standard measurements. I imagine a conversation like „There‘s some hiss!“ – „Nope, has SINAD xyz, what you hear is fantasy.“ It happened, actually.

In short, to say that now we have all we need isn‘t wise, and neither it is nice to condemn an honest(!) subjectivist approach.

 

[solderdude]

In short, to say that now we have all we need isn‘t wise,

For electronics all we need to know can be measured.
But not everything that needs to be measured is always measured and it is not wise to only look at just some basic parameters.
On top of that the really important factor is to correctly interpret the measured data.
For transducers ... sure... 'measurements' are indicative at best but even indicators can be of help to some people.

and neither it is nice to condemn an honest(!) subjectivist approach.

Subjective evaluations are fine if the goal is to buy something one is pleased with.
Subjective evaluations are not fine if one wants to make a case about 'better sound' ... because that is ONLY in the eyes of the beholder and has little to no validity for the 'science part' of the subjective evaluation.

 

[TTT15]

Science and technology moves on. Newton gets replaced by Einstein. Materials improve, storage improves, processing improves , knowledge improves etc There is invariably few absolutes especially when it comes to human interaction. None of that is to diminish the research and measurements that we do have at all. That is just the way it goes. The ASR survey shows that hard core measurement absolutists are in the minority tail of the normal like curve. Unfortunately in this world on many topics it is the people with the most extreme views who tend to be the loudest and most strident but that isn’t typically representative of the majority who tend to be more nuanced in their views.

 

[olds1959special]

Measurements serve as a guide but don't tell you if something will sound good or not. Only trial and error with listening can determine that.

 

[TTT15]

For electronics all we need to know can be measured.
But not everything that needs to be measured is always measured and it is not wise to only look at just some basic parameters.
On top of that the really important factor is to correctly interpret the measured data.
For transducers ... sure... 'measurements' are indicative at best but even indicators can be of help to some people.


Subjective evaluations are fine if the goal is to buy something one is pleased with.
Subjective evaluations are not fine if one wants to make a case about 'better sound' ... because that is ONLY in the eyes of the beholder and has little to no validity for the 'science part' of the subjective evaluation.

I generally agree with all your perspective but just theoretically can we measure how accurately electronics interpolates the original recorded soundwave sample data. And how do you weight the different measurements or is that subjective so if you are using measurements you are still being subjective in how you use them.

 

[solderdude]

just theoretically can we measure how accurately electronics interpolates the original recorded soundwave sample data.

Yes, it is shown in the distortion measurements.
When interpolation is not 'perfect' that would express itself as unwanted HF signals.
Just look at the plots of filters that have information well over that of the audible range.

Below a good and cheap DAC

When looking for the 'best possible filter inside a DAC':

When afraid for inter-sample overs simply lower the volume digitally by about 3dB before it goes to the DAC.

When distortion is low and there are no mirror images the filter is as perfect as can be.

Here you can clearly see how crappy all filterless DACs are and how poor performing the 'slow' filters are.

Good thing is that IF that is your thing you can upsample yourself with excellent filters and use a DAC at a higher sample rate.

In practice... when using a 'fast linear phase' filter it is as good as reconstruction can get and much, much better than any ear/brain can ever hope to achieve.

If signal fidelity is important stay away from R2R and stay away from any filter that is not fast + linear phase.

how do you weight the different measurements or is that subjective so if you are using measurements you are still being subjective in how you use them

Below audible thresholds using music means it is inaudible.
That's all that matters, not if test results show absolute perfection.
You just have to know what the thresholds are.... which is the knowledge part and an interpretation.
You could always moan about the thresholds being a bit higher or lower and audibility of it with certain recordings but audibility thresholds that are known and even when you would be as strict as humanly possible is still magnitudes 'worse' than what measurements can show.

For music reproduction perfection is NOT needed. As good as possible is already severe overkill but rather its the hunt for best measurements.

Of course... this has nothing to do with preference or even 'liking' anything.

 

[danadam]

Newton gets replaced by Einstein.

Replaced? Did it stop working for your everyday needs?

 

[voodooless]

Yes, it is shown in the distortion measurements.
When interpolation is not 'perfect' that would express itself as unwanted HF signals.
Just look at the plots of filters that have information well over that of the audible range.

View attachment 486273

Below a good and cheap DAC
View attachment 486274

When looking for the 'best possible filter inside a DAC':
View attachment 486275

When afraid for inter-sample overs simply lower the volume digitally by about 3dB before it goes to the DAC.

When distortion is low and there are no mirror images the filter is as perfect as can be.

Here you can clearly see how crappy all filterless DACs are and how poor performing the 'slow' filters are.

Good thing is that IF that is your thing you can upsample yourself with excellent filters and use a DAC at a higher sample rate.

In practice... when using a 'fast linear phase' filter it is as good as reconstruction can get and much, much better than any ear/brain can ever hope to achieve.

If signal fidelity is important stay away from R2R and stay away from any filter that is not fast + linear phase.

We already did this whole exercise several times before, for instance, here:

Master Thread: Are Measurements Everything or Nothing?

The word better is a subjective term Not necessarily.

www.audiosciencereview.com

At this point, why even bother...

 

[Purité Audio]

Measurements serve as a guide but don't tell you if something will sound good or not. Only trial and error with listening can determine that.

No not really, measurements will show if a component is audibly transparent ,however you may not enjoy transparency and enjoy added distortion.
Keith

 

[Mnyb]

All is not what ot looks like . Fact electronic devices has been transparent to humans for decades. ( if well executed . No one thinks ”all DAC sounds the same ” it’s a shorthand, in fact ”almost all DAC’s sound the same , but there are still outliers often extremely cheap devices or extremely expensive ones with weird cargo cult tech ” )

But mainstream audiophile press had another narrative since the 80’s and I and many others got sucked into to this world of listening to cables and other nonsense .

This clashes hard when you weared of course for 20 years and someone gives you a reality check .

But there is so much more to products than pure audio performance . Quality look and feel and the whole support chain local representation and brand vibe .

If I had an endless money supply i would buy accuphase in a heartbeat anyway .

Speakers and headphones are still a bit of the wild west, more work needs to be done .

And 3rd party testing electronics even such a boring thing as a DAC is important , for accountability.
All to often there are bad and plain weird results ? One think the designer should have read the standard textbooks of his craft and earned the university exam ? But there honest mistakes, overestimating one’s competence, QC errors and sadly hairbrained designers that believes alternative facts doing expensive unreliable contraptions that they call high-end .

And a spectrum inbetween. Examples brands that actually know better , bringing discrete or R2R solutions to market, because they sell.

A more informed cadre of consumers would bring better products to market, thats the most important thing ASR could bring.

The confusion so called subjective audiophiles brought for decades makes a market with very overpriced weird products that don’t actually achieve any fidelity goals.

Rambling. So called objective audiophiles use very subjective methods like listening testing too . The difference is basic controls to remove bias , without such controls your test is not information its just a description of your experience at that moment , no one can tell if any soundwave where different or not .

 

[solderdude]

We already did this whole exercise several times before

Yep we did .... many, many, many times before.

And indeed you can lead a person to measurements but can't make them understand them when they are convinced measurements don't show what they want them to show.
The key here is ... audibility thresholds of the human hearing.

 

[voodooless]

Science and technology moves on. Newton gets replaced by Einstein. Materials improve, storage improves, processing improves , knowledge improves etc There is invariably few absolutes especially when it comes to human interaction. None of that is to diminish the research and measurements that we do have at all.

Yes, you do this specifically to diminish the research. Saying the opposite doesn't make it true.

The key here is ... audibility thresholds of the human hearing.

Exactly! Lots of things improve, but we humans do not (or rather, we very, very slowly change due to evolution). The limits of the human sensory apparatus is very well established at this point.

 

[Geert]

Science and technology moves on

And established laws of nature remain the same, and the evolution of the human brain has been a process over millions of years. Just to say there also fundamental principles that are well understood and provide a framework to help us understand how things work.

Unfortunately in this world on many topics it is the people with the most extreme views who tend to be the loudest and most strident but that isn’t typically representative of the majority who tend to be more nuanced in their views.

Looks like you had a really traumatizing experience on this forum. Why else keep repeating this critique if it's only concerns a minority?

 

[Heinrich]

Science and technology moves on. Newton gets replaced by Einstein. Materials improve, storage improves, processing improves , knowledge improves etc There is invariably few absolutes especially when it comes to human interaction. None of that is to diminish the research and measurements that we do have at all. That is just the way it goes. The ASR survey shows that hard core measurement absolutists are in the minority tail of the normal like curve. Unfortunately in this world on many topics it is the people with the most extreme views who tend to be the loudest and most strident but that isn’t typically representative of the majority who tend to be more nuanced in their views.

How come that I as a trained objectivist can‘t agree more?
I think for amps and DACs and such the case is closed. I would be hard pressed to tell the difference between 14 or 20 bits of SINAD. To long for more than real 16 is a hobby horse. As a tech/ minded individual I get the fun of it perfectly, but it is not for everyone—a bit elitist?

I still question how, with today‘s standardized measurements, the distortion of lossy codecs would be measured w/o a reference to subjective evaluation. If there‘s detail missing, express it in percentage?

Speakers, is a KEF R3 equivalent to a 3way coaxial Genelec? For sure! Need more than an additional sub/ or bass management, for the hobby?

Headphones, that‘s the most prominent field where people show the deepest misunderstanding. It was so horrible that I gave up on any discussion—it has been my motivation to sign in here. Just and only irritation. @solderdude

 

[Newman]

Science and technology moves on. Newton gets replaced by Einstein.

Oh dear, you are still at it. Still reciting from the Science Denier's Playbook.

• "Science can't prove anything." Translation: I can deny any evidence-based conclusion because it's all just theories and hypotheses.
• "You haven't measured the right things, enough things, or accurately enough." Translation: if I don't like what the evidence suggests, I can tap-dance around it.
• "Science hasn't found the right answers yet." Translation: science-based knowledge is constantly evolving so I can confidently assert that (whatever I choose to deny today) is quite likely wrong from tomorrow's perspective. Nyah nyah-nyah-nyah nyah!
• "Scepticism is in the nature of the scientific method, so my doubting the findings of science makes me more 'science-y' than people who accept them." Translation: even though I am an ignoramus on the science itself compared to actual researchers, my constant questioning of whatever I feel like denying today makes me look like a genius! Especially compared you plebs who are foolish enough to believe it.
• "Remember Einstein." Translation: someone might come along tomorrow and turn everything we thought we knew on its head. So I can deny absolutely anything I like and nobody can say I am definitely wrong.

I think you have tried at least three of the above. So far. Honestly, it doesn't wash. We can see right through it.

It usually turns out that people who come here carrying on in accordance with the above playbook are actually seeking confirmation of their sighted listening experiences, and looking for excuses to dispute and deny (and 'question') any audio science that seems to be invalidating their sighted listening experiences. "It's just so obvious to me, I can't possibly be imagining it, so the science is wrong. Let me count the ways (to deny it)." These people especially invest themselves in disputing the science that indicates that sighted listening doesn't work as a means to assess the attributes of the sound waves themselves. If they can deny that one, then they are freee.

You really need to read Toole. And to do so with a mindset of learning about the subject, and not just learning what 'questions' to ask so you can deny the bits you don't 'like'.

And you know what? You won't even find anything about high res vs non-high res sampling rates. Because that is considered to not even be a topic worthy of discussion among audio science researchers. And no, before you crank up your excuses, no it is not because they all work for "one company" that doesn't sell high-res audio gear. Do drop the conspiracy angle (hey, I should have added that to the above list, and yes you have already tried it on).

 

[solderdude]

I still question how, with today‘s standardized measurements, the distortion of lossy codecs would be measured w/o a reference to subjective evaluation. If there‘s detail missing, express it in percentage?

There is little correlation between THD and perceived sound quality.
Witg transducers it is even worse as they differ per frequency and amplitude. Then there is masking, listening levels, room and fit issues changing things.
Measurements are only indicative here.
This also means there cannot be a percentage.

Headphones, that‘s the most prominent field where people show the deepest misunderstanding. It was so horrible that I gave up on any discussion—it has been my motivation to sign in here. Just and only irritation

That's because the correlation between 'measurements' and perception is the lowest. There are too many variables.
Fortunately there is still some correlation so far from pointless.

 

[Heinrich]

Oh dear, you are still at it. Still reciting from the Science Denier's Playbook.

You‘ve got it wrong. I‘m on your ignore list, maybe. We know that measurements as taken today are not all descriptive in regard to sound quality. HD was shown, for example by Geddes, to have little to no correlation to sound quality. This scientific result is eagerly ignored by the industry, except that HD is no ‚frontier‘ in the competition anymore? Still there is no cut-off in the published figures, like with Roll-Royce luxury cars on motorization famously saying just and only: „Sufficient.“.

That raises the general question, what a „good enough“ would be with certain parameters that are measured. If there‘s one, or in case even not, how to determine it, or in the negative case what the subjective(!) consequences of a „not totally perfect“ would be.

TTT15 is not a science denier, in my book at least. Some defense of today‘s measurement portfolio and evaluation defies simple logic. I clearly understand that people are fed up w/ semi criminal marketing in audio. We shouldn‘t allow that to blind us, and close down our minds.

There is little correlation between THD and perceived sound quality.

Headphones, …

That's because the correlation between 'measurements' and perception is the lowest. There are too many variables.
Fortunately there is still some correlation so far from pointless.

And ‚we‘ as objectivists (me included) should tell people to equalize to their individual (subjective) liking, and that shall be it. That‘s hard to grasp especially for audio enthusiasts. Just liking it isn‘t true, right?