Master Thread: Are Measurements Everything or Nothing?

[antcollinet]

HD was shown, for example by Geddes, to have little to no correlation to sound quality

Below a certain level perhaps. But try listening to music with a higher level - rarely sounds good.

More importantly - the same non linearities that create HD also result in IMD - which (when big enough to be audible) sounds universally bad.

 

[Heinrich]

Below a certain level perhaps. But try listening to music with a higher level - rarely sounds good.

More importantly - the same non linearities that create HD also result in IMD - which (when big enough to be audible) sounds universally bad.

Appreciated, of course. But as before, what is that level „for certain“ and how is it determined (scientifically)? Or in short, what is the engineers „good enough“, as I asked may times. If there is none (laughing out loud) how is the consequence of not being perfect evaluated other than subjectively?

In all modesty,I think this a killer argument. (Just kidding.)

 

[solderdude]

And ‚we‘ as objectivists (me included) should tell people to equalize to their individual (subjective) liking, and that shall be it. That‘s hard to grasp especially for audio enthusiasts. Just liking it isn‘t true, right

My suggestion would be to start from some recommended EQ or at least start with EQing some of the obvious errors and EQ by ear from there.
Also one can look at measurements to see if they are a 'worthy' candidate to begin with.

 

[Mart68]

Appreciated, of course. But as before, what is that level „for certain“ and how is it determined (scientifically)? Or in short, what is the engineers „good enough“, as I asked may times. If there is none (laughing out loud) how is the consequence of not being perfect evaluated other than subjectively?

In all modesty,I think this a killer argument. (Just kidding.)

The 'engineers' good enough' for THD was for a long time considered to be 0.1 percent (at 1 Khz) or below. Hence the Leak 'Point One' series of tube power amps. Roughly the point at which it was guaranteed to be inaudible.

I don't know if we've moved on from that. Seems to be the custom now to quote max amplifier power at 1 percent THD so maybe that is now the threshold.

Have to say, subjectively at least, I agree with Toole that distortion is never a good thing.

 

[Geert]

I still question how, with today‘s standardized measurements, the distortion of lossy codecs would be measured w/o a reference to subjective evaluation.

how is the consequence of not being perfect evaluated other than subjectively?

Subjective observations do matter, but if someone wants to sell them as a universal truth then they need to be done as a controlled experiment and the results need to be statistically relevant. I think that, in general ASR, supports that idea. In the end, F. Toole's work is also based on listening tests.

 

[voodooless]

I still question how, with today‘s standardized measurements, the distortion of lossy codecs would be measured w/o a reference to subjective evaluation. If there‘s detail missing, express it in percentage?

Maybe a reference to the HydrogenAudio TOS will answer:

TOS 8 - Hydrogenaudio Knowledgebase

wiki.hydrogenaudio.org

 

[Heinrich]

Subjective observations do matter, but if someone wants to sell them as a universal truth then they need to be done as a controlled experiment and the results need to be statistically relevant. I think that, in general ASR, supports that idea. In the end, F. Toole's work is also based on listening tests.

TTT15 doesn‘t seem to sell an individual truth. The issue is a religious belief in measurement, that rarely shines through. It won‘t be new, but science parameterizes nature. The more open parameters to be actually measured and not structurally deduced, the worse the model. But that only as a sidenote. Science imposes man-made ideas onto reality, and gets confirmation, but only so far.

With amps we are o/k below xyz % of HD, but still we long for less. Why is that?

Look, Purify as a well known amp brand claims hysteresis in the output of digi amps to matter a very lot. Their self proclaimed excellence is based on this. Snakeoil, how 2 measure, relevant? Purify as an outfit, seen as non-engineers, science deniers, the bad guys?

But again I get lost in so many examples. My stance is like accept honest subjectivists. Don‘t ever fall into religious belief on either side. Hifi is for fun only ;-)

ps on hydrogen audio, courtesy of @voodoless: I see their approach as isolating, while in perfect agreement with their aiming. Science needs to perfect hearing aids first. Stereo is no hearing aid, it is another experience altogether. In case science finds some interest, it may help those ever demanding perfectionists. But that‘s not what science is made for.

„We“ are lacking a definitive „good enough“, guys. CU

 

[rdenney]

The 'engineers' good enough' for THD was for a long time considered to be 0.1 percent (at 1 Khz) or below. Hence the Leak 'Point One' series of tube power amps. Roughly the point at which it was guaranteed to be inaudible.

I don't know if we've moved on from that. Seems to be the custom now to quote max amplifier power at 1 percent THD so maybe that is now the threshold.

Have to say, subjectively at least, I agree with Toole that distortion is never a good thing.

Agreeing with Toole's comment isn't a subjective thing, it's a conceptual thing. If you had two items to compare that were otherwise identical, and one had -40 dB (1%) distortion while the other had -80 or -100 dB distortion, anybody would choose the latter, whether they could actually hear it or not with any given playback material. I think that was Toole's point.

I run into this all the time. Engineers design solutions, but are not usually trained in defining objectives. I an a licensed engineer in five states and have two degrees, and I teach objectives-driven operations and requirements-driven design. This is a topic I run into a lot. Most of what I know about objectives, needs, and requirements I learned subsequent to my engineering training.

If you tell an engineer to design an amp with 0.1% distortion (-60dB), they'll do it. They could do that even 75 years ago. If you ask an engineer who is expert in amp design what that threshold should be, they are likely to work backwards from a design rather than work forwards from a use case and need that is based on objectives for operation. For that, you'd need an expert in psychoacoustics, who may or may not be an engineer, and a statistician, if you want to set an objective for a product line that meets the needs of your customer base. Not many engineers are statisticians. I was once roundly roasted by a room full of qualified engineers when I used the word "heteroskedasticity"--they had never even heard the term let alone knowing what it meant, and my use of it actually offended them.

I just wrote in another thread that I can easily distinguish a 1 KHz sine wave with -45 dB THD from the same signal with -100 dB THD, even with both played through an amp that might test at -60 dB THD into old bookshelf speakers. (I have tested this carefully.) But I cannot distinguish differences in THD below about -36 dB when comparing music samples in a proper ABX test through good headphones.

Even so, I would choose an amp that performs at 0.01% THD versus an amp at 1% THD--there's no reason not to these days--though I might be unconcerned about playing the music loudly enough to reach that 1% in practice. Not many loudspeakers can do better than that in any case.

Requirements flow from user needs, which are determined based on objectives. "Good enough" fulfills requirements, no more, no less.

Rick "much ado about nothing" Denney

 

[antcollinet]

Seems to be the custom now to quote max amplifier power at 1 percent THD so maybe that is now the threshold.

Always has been as far as I know, by many manufacturers. The argument being that max power will (should) only be used for very brief transient peaks - for which the distortion will be much less noticeable. I think the 0.1% (-60dB) is more realistic for a "continuous" level.

I personally aim for a headline -80dB (0.01%) and know I won't hear anything bad (Bearing in mind that is typically a 1kHz figure - and will be worse at higher frequencies.)

 

[TTT15]

Yes, it is shown in the distortion measurements.
When interpolation is not 'perfect' that would express itself as unwanted HF signals.
Just look at the plots of filters that have information well over that of the audible range.

View attachment 486273

Below a good and cheap DAC
View attachment 486274

When looking for the 'best possible filter inside a DAC':
View attachment 486275

When afraid for inter-sample overs simply lower the volume digitally by about 3dB before it goes to the DAC.

When distortion is low and there are no mirror images the filter is as perfect as can be.

Here you can clearly see how crappy all filterless DACs are and how poor performing the 'slow' filters are.

Good thing is that IF that is your thing you can upsample yourself with excellent filters and use a DAC at a higher sample rate.

In practice... when using a 'fast linear phase' filter it is as good as reconstruction can get and much, much better than any ear/brain can ever hope to achieve.

If signal fidelity is important stay away from R2R and stay away from any filter that is not fast + linear phase.


Below audible thresholds using music means it is inaudible.
That's all that matters, not if test results show absolute perfection.
You just have to know what the thresholds are.... which is the knowledge part and an interpretation.
You could always moan about the thresholds being a bit higher or lower and audibility of it with certain recordings but audibility thresholds that are known and even when you would be as strict as humanly possible is still magnitudes 'worse' than what measurements can show.

For music reproduction perfection is NOT needed. As good as possible is already severe overkill but rather its the hunt for best measurements.

Of course... this has nothing to do with preference or even 'liking' anything.

Thanks again for the as usual helpful answers. My understanding rightly or wrongly is that the band width limiting filter measurements are the counterpart to interpolation so if the band width filter is effective then the interpolation will be good as they are both a product of the sinc function. How do people weight the speaker measurements. I presume read Toole is a fair response but is there a summary somewhere.

 

[solderdude]

The reconstruction filters are an obligatory part of the process. What the filters do is calculate how to connect the data 'dots' which is the sample data.
R2R the DACs are using sample-and-hold, which is an incorrect method but the only practical solution as the taken samples are 'points' with no data between them and not values that do not change 99% of the time and only change values 1% of the time.

Klippel method does the weighing for you but only does so for a 'very good listening room' which not many people actually have.
This is only for expected tonality in a 'very good listening room'.

The most sound quality determining factor is tonality. That, however, is also room dependent and listening position and toe-in dependent as well as proper height dependent.

So... for speakers ... always listen to the speakers in your room on your gear.
For headphones... always audition.
For electronics... buy on functionality and see if they can do what you want them to do.
When you have one that measures good then you know what is fed to the speakers is as good as it gets.
When you don't like what you hear ... transducers and positioning.

The LAST thing I would ever worry about is the DAC, especially when it has fast linear phase filters and USB input and isn't a very old design.

 

[DSJR]

Science and technology moves on. Newton gets replaced by Einstein. Materials improve, storage improves, processing improves , knowledge improves etc There is invariably few absolutes especially when it comes to human interaction. None of that is to diminish the research and measurements that we do have at all. That is just the way it goes. The ASR survey shows that hard core measurement absolutists are in the minority tail of the normal like curve. Unfortunately in this world on many topics it is the people with the most extreme views who tend to be the loudest and most strident but that isn’t typically representative of the majority who tend to be more nuanced in their views.

Amply answered before this comment, but subjectivists listening to what they feel is a nice sound, is in my experience, very much a *multi sensed* experience and the ears or listener's hearing plays a very small part in this experience as a whole I feel.

 

[TTT15]

Oh dear, you are still at it. Still reciting from the Science Denier's Playbook.

• "Science can't prove anything." Translation: I can deny any evidence-based conclusion because it's all just theories and hypotheses.
• "You haven't measured the right things, enough things, or accurately enough." Translation: if I don't like what the evidence suggests, I can tap-dance around it.
• "Science hasn't found the right answers yet." Translation: science-based knowledge is constantly evolving so I can confidently assert that (whatever I choose to deny today) is quite likely wrong from tomorrow's perspective. Nyah nyah-nyah-nyah nyah!
• "Scepticism is in the nature of the scientific method, so my doubting the findings of science makes me more 'science-y' than people who accept them." Translation: even though I am an ignoramus on the science itself compared to actual researchers, my constant questioning of whatever I feel like denying today makes me look like a genius! Especially compared you plebs who are foolish enough to believe it.
• "Remember Einstein." Translation: someone might come along tomorrow and turn everything we thought we knew on its head. So I can deny absolutely anything I like and nobody can say I am definitely wrong.

I think you have tried at least three of the above. So far. Honestly, it doesn't wash. We can see right through it.

It usually turns out that people who come here carrying on in accordance with the above playbook are actually seeking confirmation of their sighted listening experiences, and looking for excuses to dispute and deny (and 'question') any audio science that seems to be invalidating their sighted listening experiences. "It's just so obvious to me, I can't possibly be imagining it, so the science is wrong. Let me count the ways (to deny it)." These people especially invest themselves in disputing the science that indicates that sighted listening doesn't work as a means to assess the attributes of the sound waves themselves. If they can deny that one, then they are freee.

You really need to read Toole. And to do so with a mindset of learning about the subject, and not just learning what 'questions' to ask so you can deny the bits you don't 'like'.

And you know what? You won't even find anything about high res vs non-high res sampling rates. Because that is considered to not even be a topic worthy of discussion among audio science researchers. And no, before you crank up your excuses, no it is not because they all work for "one company" that doesn't sell high-res audio gear. Do drop the conspiracy angle (hey, I should have added that to the above list, and yes you have already tried it on).

I fully agree that it would be good to read Toole at some point when I have time. That is generally your subjective interpretation of what I think though rather than what I actually think. Measurements and research are still good as far as I’m concerned. Why else would I buy well measuring equipment based on science and look to demo well measuring speakers. It’s correlating the objective measurements and research with all the other subjective human purchasing factors that is tricky. Surely the extent to which we weight particular measurements and all the other non sound purchasing factors is ultimately subjective. The decision to purchase HiFi or not is subjective. I’ve heard from some people that Sonos is perfectly good enough and no one needs HiFi so ultimately all our decisions are subjective. That doesn’t mean at all that the measurements or research is wrong or being denied. If you are saying that everyone should buy just off measurements rather than what they enjoy im not wholly convinced. Even if someone is buying off their perception which wouldn’t be supported by blind tests as there is limited blind tests available then that perception although it can’t be measured is their reality and the placebo effect is real.

 

[TTT15]

The reconstruction filters are an obligatory part of the process. What the filters do is calculate how to connect the data 'dots' which is the sample data.
R2R the DACs are using sample-and-hold, which is an incorrect method but the only practical solution as the taken samples are 'points' with no data between them and not values that do not change 99% of the time and only change values 1% of the time.

Klippel method does the weighing for you but only does so for a 'very good listening room' which not many people actually have.
This is only for expected tonality in a 'very good listening room'.

The most sound quality determining factor is tonality. That, however, is also room dependent and listening position and toe-in dependent as well as proper height dependent.

So... for speakers ... always listen to the speakers in your room on your gear.
For headphones... always audition.
For electronics... buy on functionality and see if they can do what you want them to do.
When you have one that measures good then you know what is fed to the speakers is as good as it gets.
When you don't like what you hear ... transducers and positioning.

The LAST thing I would ever worry about is the DAC, especially when it has fast linear phase filters and USB input and isn't a very old design.

Thanks once again although I thought r2r dacs also used oversampling unless they were NOS and then people would often use external upsampling. Not that I’m in the market for r2r just curious why the love for the likes of Holo May KTE. My dealer also told me long ago not to bother with another DAC and I have a fast filter and USB so that all seems good. I spent a few weekends auditioning different headphones at HiFi show and dealers and they measure well so that is fine. Demoing speakers at home isn’t an option unfortunately. Are the Koppel measurements on somewhere like spinorama and do they effectively give a weighted rating?

 

[Purité Audio]

And yet again, measurements simply allow you to choose if you wish, a well designed transparent product.
That doesn’t mean you will enjoy transparency.
The Klippel NFS is simply the most accurate method of measuring, every loudspeaker manufacturer should have one and publish its results.
Keith

 

[TTT15]

And I still don’t have an issue with that at all. That seems to be entirely reasonable to me and is how I’m proceeding. As stated previously I just have questions to facilitate my understanding for purchase decisions Asking questions is what i do to pay for the HiFi among other things . Some will be stupid, some irrelevant and some will lead to new understanding as Solderdude always helpfully provides . I fully appreciated that some other people might not be used to being challenged and will perceive it as a threat to their beliefs but that ironically is like people claiming they can hear differences that don’t exist, some people will choose to subjectively interpret something that doesn’t actually exist.

 

[Purité Audio]

I don’t believe anyone here finds your questions challenging, tedious perhaps.
Keith

 

[voodooless]

I fully appreciated that some other people might not be used to being challenged

Are you kidding me? At ASR everything gets challenged. We can do this all day long. This thread of (mostly) silly repetitive non arguments isn’t 811 pages long because nothing gets challenged.

and will perceive it as a threat to their beliefs

Proof will challenge the state of art, not belief.

 

[TTT15]

I don’t suppose otherwise and I’m absolutely all up for proof. I wish there was a lot more blind testing but that is difficult in practice.

 

[oleg87]

Thanks once again although I thought r2r dacs also used oversampling unless they were NOS and then people would often use external upsampling. Not that I’m in the market for r2r just curious why the love for the likes of Holo May KTE

Yes, they generally do, though unlike delta-sigma converters it's not inherent to the approach.

It's an expensive boutique device so it's not hard to guess what the "love" is for, and it seems like a well-engineered converter, but so are ESS/AKM ICs that cost less than the sandwich I'm having for lunch.