Master Thread: Are Measurements Everything or Nothing?

[Newman]

Good point but one is objective and the other subjective. I guess the ultimate goal is to re produce what a live unamplified musical performance sounds like if you were sitting in the best seat in the house. But you would have to remember what you heard and then compare it to a recording of that same performance. As far as preferring more or less bass etc., if you are sitting in the best seat, are you wishing/wanting more or less of any thing? That performance at that time and in that place is what it is.

Completely agree that difficult for many people to be open to other points of view when they are emotionally and financially invested. But then if HiFi is completely defined by their measurements can we see by extension which is the best for each component? . How should we weight the different measurements and are they any other factors we should consider or are most of the world fools or are all the people who are not just not interested in HiFi or measurements also fools. Given how few women I see in the HiFi world we seem to be being judgemental about a lot of people

Given the various philosophical musings above, this might be a good time to recommend you each read Sound Reproduction by Dr Floyd E Toole. Some of the things that you think are unanswerable have been 'put to the test', as they say, and interesting and unexpected findings emerged.

It takes some time reading through the development of the topics to see how the evidence and findings fold together, and for me to jump in here with simple-seeming statements will only invite incredulity and an endless debate, because you really need to read the book to 'get it'.

Please do give it a go.

cheers

 

[TTT15]

I work hard for a living so I don’t have that sort of time but I’ve seen the vids and I’m aware of the conclusions and I’m completely willing to try products based on their findings.

However, the measurements that I see tend to cover products from specific manufacturers. No doubt other manufacturers also do their own research. I understand that he recommended that you should also listen for yourself. And that it isn’t possible to perfectly reproduce sound waves I’m still not aware of any published peer reviewed scientific research linking all measurements. I’m also not aware that there is only one measurement for any product but a number of measurements so how do you weight them.

To be honest I’ve tried a number of products with good measurements some of which I’ve enjoyed and some which I thought were ok but didn’t enjoy. Is there an individual measurement that measures how much an individual enjoys a particular product. Should people buys products they don’t enjoy just based on measurements . Should people purchase without listening. Or should people consider the various measurements as potentially useful info to consider together with all the other factors that people typically consider when making a purchase decisions in addiction to specs such as availability, customer service, build quality, reputation, reliability, features, aesthetics, price etc and is there a measure that weighs all of these factors. I think the market says that HiFi purchases just based on specs is relatively niche and if we are going to make decisions based on preference curves the overall preference consumer preference curve is to make a purchase decisions based on a number of factors.

 

[solderdude]

it isn’t possible to perfectly reproduce sound waves

Indeed this is not possible for sound waves. We would need 'holographic' transducers at the very least.
For amplifiers, DACs etc. reproduction can be awfully close to recorded wave forms (near perfection given laws of nature limitations).

I’m still not aware of any published peer reviewed scientific research linking all measurements.

When you mean perception and measurements... sure there will never be a 100% correlation because of perception.

I’m also not aware that there is only one measurement for any product but a number of measurements so how do you weight them.

There isn't.

Is there an individual measurement that measures how much an individual enjoys a particular product.

No, nor will there ever be.

Should people buys products they don’t enjoy just based on measurements.

No, but good measurements usually means good signal fidelity.

Should people purchase without listening.

Speakers ... preferably not. Best to audition at home, not in a shop.
Headphones .. preferable not (not only for sound but also comfort).
Electronics .. should be no problem.

Sometimes auditioning is not an option. Reviews and measurements can help narrow down choices.
Being able to return gear or sell onward is an option and could consider that tuition fees.

Or should people consider the various measurements as potentially useful info to consider together with all the other factors that people typically consider when making a purchase decisions in addiction to specs such as availability, customer service, build quality, reputation, reliability, features, aesthetics, price etc.

Yes, preferably.

and is there a measure that weighs all of these factors.

No of course not nor can/will there ever be.

I think the market says that HiFi purchases just based on specs is relatively niche and if we are going to make decisions based on preference curves the overall preference consumer preference curve is to make a purchase decisions based on a number of factors.

Yes, I figure 99% of all audio purchases worldwide are made that way.

Fortunately this does not invalidate the importance of measurements.
Interpreting correctly and understanding measurements is paramount though.

 

[RexrothPigeon]

I think you may well be able to say better with respect to one of many specific measures which is just one factor that some consumers may consider in their overall consumer purchasing decision.

Hifi equipment has a job: to reproduce the recordings with as high fidelity (faithfulness, accuracy) as possible.

So in the case of hifi equipment, yes, better is objective and measurable.

Some people may not prefer better hifi equipment, of course, and there is nothing wrong with that at all, but the equipment they prefer is objectively worse at doing its job (reproducing recordings).

 

[TTT15]

And that is fine in theory but we don’t have published peer reviewed scientific research specifically combining snd linking specific measurements to how much people enjoy products in practice.

We have lots of different measurements for a few brands and we have a lots of other factors that play into the consumer choice. So what actually matters is what is most enjoyable for a particular individual who are all different with different physiology and different preferences. Music is about enjoyment.

The only proven we have is nyquist Shannon which says possible in theory and Shannon Whittaker which says we need an infinite sinc function to perfectly reproduce sound perfectly which isn’t possible in practice. Monty videos also state filters are difficult and we don’t appear to have measurements which capture how accurate the sinc function is applied.

We don’t have a weighting of all the different measurement factors. We don’t have measurements for all products. We don’t have scientific studies which link measurements to enjoyment. Only a few limited tests for certain aspects on average for certain brands.

There isn’t an e = mc2 or field equations for HiFi products. Humans are different and subjective. So without definitive proof and without definitive equations which combine and link all the different measurements and all the other factors in practice consumers are left with a subjective choice. A particular cumsumer choice for some maybe to put more weight on certain measurements which haven’t been scientifically linked to human enjoyment but which is also a valid consideration. For instance the harman curve isn’t based on accuracy but average pretences focused just on frequency response for one brand of products which isn’t the most popular brand. The brands that focus on measurements are niche is practice so don’t conform to the overall consumer preferences. Other more popular brands do their own research and make other choices on what people prefer and are more successful. So in practice it is too complex to establish what is better or best for an individual.

That doesn’t mean that some measurements aren’t potentially useful for some individuals. Who ideally wants noise and distortion. But most people listen to music on the go and don’t know and don’t care about measurements and there is no evidence that these people are any less happy. They may well even be happier if they don’t know and don’t care about such things.

 

[Purité Audio]

Uh huh.
Keith

 

[solderdude]

And that is fine in theory but we don’t have published peer reviewed scientific research specifically combining snd linking specific measurements to how much people enjoy products in practice.

That's because of the variability of perception, taste, recording quality (the 'sound quality' of recordings is hugely variable).
Not because measurements are pointless or require to match perception with 100% accuracy (there certainly is a correlation).
Measurements are just indicators for signal fidelity.

We have lots of different measurements for a few brands and we have a lots of other factors that play into the consumer choice. So what actually matters is what is most enjoyable for a particular individual who are all different with different physiology and different preferences. Music is about enjoyment.

Yes, music is about enjoyment. Preference has a big part in this.
HiFi systems have a job... reproduce the recorded material (not music).
How much one wants to deviate from that is preference and that differs so you can't take that into consideration. Too much variables like variability of perception, taste, recording quality.

Besides people that enjoy the creative aspect or music can even enjoy that from the speaker of a phone and don't really NEED (near perfect) reproduction to enjoy those aspects of music.
They may enjoy music a bit more when fidelity is higher...

This is something entirely different as audiophiles which require a certain 'sound (quality)' to get enjoyment.
And even in the audiophile community there is a divide namely those that want the reproduction chain to be as good as possible and those that don't care about all that and are just looking for a sound they enjoy.
These 2 'groups' is what your gripe is with and the misunderstanding about the word 'sound quality' versus 'signal quality'.

The only proven we have is nyquist Shannon which says possible in theory and Shannon Whittaker which says we need an infinite sinc function to perfectly reproduce sound perfectly which isn’t possible in practice. Monty videos also state filters are difficult and we don’t appear to have measurements which capture how accurate the sinc function is applied.

A theorem is not the same as a theory.
The ears do not need infinite sinc function at all as ears are bandwidth limited.
There are many methods to check how accurate the sinc function is applied.
> 96kHz the sinc function can even be poor as there is hardly any recordings having relevant 'info' up to 50kHz and the ears don't detect.

We don’t have a weighting of all the different measurement factors. We don’t have measurements for all products. We don’t have scientific studies which link measurements to enjoyment. Only a few limited tests for certain aspects on average for certain brands.

The weighing has to be done by those interpreting the measurements. This is where knowledge of the public is lacking as well as all the required measurements.

There isn’t an e = mc2 or field equations for HiFi products.

There can't be why complain about it ?

Humans are different and subjective.

Yes they are which is why there is no 100% correlation with perception but there is 100% correlation to signal fidelity. The latter is what measurements are about. Not about perception.

So in practice it is too complex to establish what is better or best for an individual.

Yep but measurements are not about that, they only show measured performance and not about any of the other mentioned aspects.
This means measurements have value for those that can interpret them correctly and is only about signal fidelity to the recorded material.

But most people listen to music on the go and don’t know and don’t care about measurements and there is no evidence that these people are any less happy. They may well even be happier if they don’t know and don’t care about such things.

1: People don't have to care about measurements for measurements to still have value.
2: Does it matter to people who don't care about technical performance if some review says signal fidelity is lacking ? Do they really get upset and stop enjoying music ?

In the end... for people that care for measurements and understand them any (accurate and telling) measurements have a predictive value for signal fidelity (and thus potentially sound quality).
For people that don't care and/or understand measurements they don't have to look at it and just buy what they want and enjoy it just the same.


Measurements are valuable to those that understand them and are a great tool for furthering the quality of signal quality reproduction.
Period.

 

[RexrothPigeon]

And that is fine in theory but we don’t have published peer reviewed scientific research specifically combining snd linking specific measurements to how much people enjoy products in practice.

This has nothing to do with theory. Hifi's quality does not require a link to preference, since the job is to reproduce the source with fidelity. We can use tools to measure fidelity to source very, very, very well. Much more precisely than our ears can. There are of course also the Sean Olive studies about preference, but again, they're not needed to establish hifi equipment's objective quality.

So again, some people may not prefer high fidelity, of course. They may want distortion or inaccurate frequency reproduction or noise. And again, there is nothing wrong with that at all (de gustibus non est disputandum!). But the equipment they prefer is objectively worse at doing its job (accurately reproducing recordings).

 

[Beave]

I work hard for a living so I don’t have that sort of time but I’ve seen the vids and I’m aware of the conclusions and I’m completely willing to try products based on their findings.

But you have time to post on this forum 180+ times in the last six weeks? Just think, in that time, you could have read the book.

 

[voodooless]

The only proven we have is nyquist Shannon which says possible in theory

We only need one proof for that.. What more proof do you need than a perfectly mathematical proof.

Shannon Whittaker which says we need an infinite sinc function to perfectly reproduce sound perfectly which isn’t possible in practice.

Yes, so? We use an approximation that is so good that humans don't hear the difference.

Monty videos also state filters are difficult

He never said any of that.

we don’t appear to have measurements which capture how accurate the sinc function is applied.

Sure, we do. Amir does this every time he measures a DAC.

 

[Newman]

And that is fine in theory but we don’t have published peer reviewed scientific research specifically combining snd linking specific measurements to how much people enjoy products in practice.

Really...

 

[TTT15]

Im not dismissing measurements or research at all. I consider them useful information that is the best we have but dont see one proven definitive measurement for all components and all products that says that piece of equipment is the most accurate or more importantly enjoyable otherwise what is the point of being accurate.

I’ve not seen definitive peer published peer reviewed research which would constitute science linking measurements to people’s individual enjoyment for all products and components. Clearly the majority of consumers, retailers and producers subscribe to this view. That still doesn’t mean what we do have isn’t useful and shouldn’t be considered just that it is not definitive for all products and all individuals. Ultimately purchasing a consumer product is a subjective decision and the weight that someone puts on any of the various measurements among the many other factors is a subjective decision.

I’ve seen Monty speak about the difficulty with aliasing with reproduction filters and I’m not aware of measurements that specifically measure how accurate the interpolation filters are which I understand most DACs other than NOS DACs use.

On the ASR subjective objective survey I’m in the middle of the normal curve so my personal purchasing approach preference seems to tie in with the majority of ASR. For sure there are people who may wish to put more weight on specific individual measurements which is absolutely fine as far as I’m concerned although I’m not sure how they weight all the different measurements.

Others may prefer to purchase well measuring components which also meet all their other criteria and which they have also listened to and they subjectively enjoy. The majority of the population probably don’t consider measurements at all. All of which is fine as far as I’m concerned. I’m not sure there should be some kind of absolute dictatorship when it comes to personal consumer purchasing preferences so all positions are fine by me.

I do seem to prefer well measuring equipment myself but I don’t see one measurement which dictates whether I enjoy something or not. If others do then that is also good as far as I’m concerned but I certainly am not going to cast judgement on a person on their approach to consumer product specifications or consider them better or worse as a person.

 

[Mart68]

I do seem to prefer well measuring equipment myself but I don’t see one measurement which dictates whether I enjoy something or not. If others do then that is also good as far as I’m concerned but I certainly am not going to cast judgement on a person on their approach to consumer product specifications or consider them better or worse as a person.

If someone ignores or disregards facts and evidence that certainly paints something of a picture of them. The general population don't tend to be aware of any of that but that's really irrelevant. The general population don't own high fidelity sound systems or if some do, they went to a dealer and bought what he recommended. They use it, they are happy, they do not get actively involved in the hobby in ways such as posting on forums.

For me 'judgement' only comes into it when someone is telling me I'm an idiot for paying attention to facts and evidence. There is a vocal and active faction amongst enthusiasts that wilfully dismisses knowledge in favour of cargo-cult mysticism. Even that would not be an issue except that they tend to offer 'advice' based on that mysticism and so people looking for information about sound reproduction are misled in ways that cost them money, sometimes serious amounts of it.

 

[rdenney]

Oh, fer cryin' out loud. Research is the one thing we have in abundance, but you have to be prepared to find it and read it.

Many of the authors of such research participate in this forum, but you have to be prepared to take them at their word if you are unwilling to gather your own data.

People will listen to some snake-oil salesman or an English-major writer for a magazine or product advertising (and the exceptions to this generality know who they are) or gray-haired YouTube "expert", but they won't believe people whose work appears routinely in research journals such as the one published by the Audio Engineering Society. They won't believe the data from actual measurement and testing, or the explanations from real experts as to why those measurements are appropriately descriptive. They will lie to themselves about their own objectives ("I trust my ears so I won't do blind testing or review measured performance"). But they will believe jokers on YouTube whose only credentials are that they used to sell stuff in audio stores, a business that no longer exists (for the most part) perhaps because people stopped trusting them to present good-faith value propositions instead of hawking high-markup feel-good products and impressive-looking (rather than sounding) house brands. Or, they'll completely obscure the relative importance of measurements of gross effect with measurements of fine effect--they'll spend zillions on an amp and use them to drive highly-colored speakers that aren't faithful to the recorded signal in just about any real circumstance.

True, some of the regulars here have to remind themselves of that, too, and recognize that, for example, a DAC that improves from 115 dB SINAD to 120 dB SINAD represents no real improvement, even if they admire the achievement.

Rick "it's easier to believe charlatans than scientists, apparently, even when the scientists are not charlatans" Denney

 

[voodooless]

I’m not aware of measurements that specifically measure how accurate the interpolation filters are which I understand most DACs other than NOS DACs use.

Then your awareness is lacking...

And NOS is effectively also interpolation, just that it holds the sample value for the whole sample, resulting in a very poor reconstruction.

Here is a start for your awareness:

NOS vs. Digital Filtering DACs: Exploring filtering turned off, implications, fidelity and subjective audibility. (Recent BorderPatrol DAC chatter...)

A blog for audiophiles about more objective topics. Measurements of audio gear. Reasonable, realistic, no snakeoil assessment of sound, and equipment.

archimago.blogspot.com

Audiophile Myth #260: The Detestable Digital Filter Ringing and Real Music...

A blog for audiophiles about more objective topics. Measurements of audio gear. Reasonable, realistic, no snakeoil assessment of sound, and equipment.

archimago.blogspot.com

 

[TTT15]

Then your awareness is lacking...

And NOS is effectively also interpolation, just that it holds the sample value for the whole sample, resulting in a very poor reconstruction.

Here is a start for your awareness:

NOS vs. Digital Filtering DACs: Exploring filtering turned off, implications, fidelity and subjective audibility. (Recent BorderPatrol DAC chatter...)

A blog for audiophiles about more objective topics. Measurements of audio gear. Reasonable, realistic, no snakeoil assessment of sound, and equipment.

archimago.blogspot.com

Audiophile Myth #260: The Detestable Digital Filter Ringing and Real Music...

A blog for audiophiles about more objective topics. Measurements of audio gear. Reasonable, realistic, no snakeoil assessment of sound, and equipment.

archimago.blogspot.com

Thanks. I used the word not aware deliberately. I assumed that NOS just held the signal rather just drops off. I don’t really understand conceptually why people would be interested in NOS and it doesn’t interest me as doesn’t seem conceptually an accurate approach. I understand that different DACs do use different approaches and filters to the reconstruction process. I percieve that makes absolutely no difference in a modest speaker system or on the go in noisy environments but there does appear to be subtle differences in the reconstruction filters which are apparent on transparent headphone systems in a very quiet environment and I’m not sure what and how measurements capture the difference between different reconstruction methodologies.

 

[solderdude]

I’ve not seen definitive peer published peer reviewed research which would constitute science linking measurements to people’s individual enjoyment for all products and components.

That's because there is no relation between enjoyment and signal fidelity.
Why would anyone do scientific tests when there is no relation to begin with ?

I assumed that NOS just held the signal rather just drops off.

NOS means non oversampling.

What most people mean by it is filterless DACs.
These do not adhere to the sampling theorem (just like many of the selectable filters don't) and thus is 'broken'.
While far from ideal a filterless DAC usually works fine for >96kHz files as the ear's steep HF cutoff and transducer bandwidth limits will act as a filter.
Alas not for the crap above Nyquist which can cause (sometimes audible) issues.

ASR measurements (and those from other like Archimago) can tell you which ones to avoid.

 

[voodooless]

but there does appear to be subtle differences in the reconstruction filters which are apparent on transparent headphone systems in a very quiet environment and I’m not sure what and how measurements capture the difference between different reconstruction methodologies.

Frequency response. Young people might with a lot of practice be able to hear that diffrence. Statistically your probably not young, so there is no way you'll every hear the difrence.

 

[TTT15]

Sorry I’m not sure I understand. I listen to music for enjoyment so if there is no link to measurements I’m not sure what the point of measurements are. Personally I think audible noise and unpleasant distortion detracts from my enjoyment of music so the measurements are helpful. Likewise we have some indicative average preference curves. So I percieve a link between enjoyment which is my goal and measurements I just don’t see the links as absolute and definitive.

NOS doesn’t appeal to me on the grounds of accuracy but still not sure how we measure and rank that approach compared to other filters.

 

[Purité Audio]

That’s precisely why measurements are so useful, they allow you to discard poor designs, assuming you seek fidelity.
Amir measures the efficacy of the filters too, just purchase well designed equipment, again if your goal is fidelity.
Keith