Master Thread: Are Measurements Everything or Nothing?

[JSmith]

Klippeled

Gold... if one has been klippeled does that mean they've been given a klippeling.

Measurements only make sense if you can buy the exact same as what has been measured.

You may be assuming here that device measurements are static or constant, whereas they are only static at one point in time. For example Amir tends to take a screenshot of the best figure, but the measurements will fluctuate. Many of the same devices tested here, elsewhere and by users often do have similar results, but it's never exactly the same and there are many variables involved, like signal stability, window timing, thermal drift etc. But as long as in the same ballpark, all is well and considered to "match".

JSmith

 

[Deleted member 88295]

You may be assuming here that device measurements are static or constant, whereas they are only static at one point in time. For example Amir tends to take a screenshot of the best figure, but the measurements will fluctuate. Many of the same devices tested here, elsewhere and by users often do have similar results, but it's never exactly the same and there are many variables involved, like signal stability, window timing, thermal drift etc. But as long as in the same ballpark, all is well and considered to "match".

View attachment 483280


JSmith

No I mean, company sends in product for testing and then sells a revision (read cost savings) instead. What good are measurements in that case?

 

[JSmith]

No I mean, company sends in product for testing and then sells a revision (read cost savings) instead. What good are measurements in that case?

In many cases they would be caught out. Don't forget members send items in for testing just as much as manufacturers, or the device is simply purchased retail.


JSmith

 

[solderdude]

No I mean, company sends in product for testing and then sells a revision (read cost savings) instead. What good are measurements in that case?

Measurements show technical performance in some parameters for the device under test.
It is only valid for that tested device but a good indicator for expected performance.

It may be broken, it may be cherry picked, it may be representative, it may be a review sample, it may be early or late production, it may be a crappy assembled device, it may be revision x.x. we generally don't know.

Usually differences between revision performance is small so there is very little chance performance degrades so much that it becomes audible.

 

[Curvature]

Measurements show technical performance in some parameters for the device under test.
It is only valid for that tested device but a good indicator for expected performance.

It may be broken, it may be cherry picked, it may be representative, it may be a review sample, it may be early or late production, it may be a crappy assembled device, it may be revision x.x. we generally don't know.

@Raiju Occasionally we see pair deviation measurements from some reviewers that have access to multiple samples.

 

[krabapple]

Measurements only make sense if you can buy the exact same as what has been measured.

This presumes there is typically a large enough unit-to-unit variance to be audible and dispositive.

Do you believe that's true for all audio devices that get measured?

 

[Deleted member 88295]

This presumes there is typically a large enough unit-to-unit variance to be audible and dispositive.

Do you believe that's true for all audio devices that get measured?

No I fully understand that there will variance in measurements in units with identical components and layout, and yes there will even be variance in the same unit measured twice.

And with what and how it is measured also plays a role.

But let me give an example, the 3e audio a7 was tested here with a SINAD of 102. But you cannot buy the exact same one that was tested, you can only buy a revision.

There is a test of the retail one that gets a SINAD of 96 (if I did the math right)

Now I do not know how accurate the measurements in the video are, I do trust what is measured here but like I said you cannot buy that exact one.

So it gets a bit confusing.

 

[krabapple]

Your use of the phrase 'the exact one' is what confused me.

You don't need to get 'the exact one', you need to get 'the same model and version' -- assuming the revision is likely to make an audible difference.

 

[solderdude]

Different analyzer.... different numbers... yet the actual performance may be similar but the used analyzer in the video may not measure as well as the AP .

Besides ..the difference between 96 and 102 is inaudible vs inaudible.

 

[pma]

Admirable patience.

 

[Deleted member 88295]

Different analyzer.... different numbers... yet the actual performance may be similar but the used analyzer in the video may not measure as well as the AP .

Besides ..the difference between 96 and 102 is inaudible vs inaudible.

Ok so I looked at their review of the Aiyima A70. Here it scores 94.42dB (Ch1) and 95.61dB (Ch2), there it scores 95.97dB. That is close enough for me to say their numbers for the 3e a7 could be true.

And the difference between 96 and 102 being inaudible is not what this is about.

 

[solderdude]

And the difference between 96 and 102 being inaudible is not what this is about.

Indeed it is about the differences in analyzers and measurement circumstances.

 

[rdenney]

Inderdaad it is abortus the differences in analyzers and measurement circumstances.

Sometimes it’s best to wait until the next morning rather than typing while asleep. Ask me how I know

My better HP 8903b (I have two) measures the total distortion of its own test-signal oscillator at -97 dB (1KHz, 2V RMS), which is better than its specified performance. That was the state of the distortion-measurement art 30 years ago. Doing better required a spectrum analyzer but that lacked precision and really only worked for harmonic distortion.

Analyzers have come a long way, particularly since they started using 24-bit digital sampling and source oscillation. The current Keysight (vice Agilent/HP) U8903A uses digital source and measurement at 32 bits depth to reach somewhat better than -100 dB, but the AP is, of course, better still. But noise levels that low are hard to measure and hard to keep isolated without meticulous protocols.

Rick “would not take measurements of performance below -100 dB as repeatable gospel without those protocols” Denney

 

[pma]

Rick “would not take measurements of performance below -100 dB as repeatable gospel

They are repeatable. The threshold of uncertainty is rather below -120dBr.

 

[solderdude]

Sometimes it’s best to wait until the next morning rather than typing while asleep. Ask me how I know

I was typing from a new phone and it had Dutch 'autocorrect' engaged.
Will fix it on my phone and tell it to not autocorrect but only make suggestions.

I won't ask you how you did not know it was Dutch autocorrect...
It is the middle of the day here...

There are plenty of people using an (external) sound card for acquisition which is the limiter of measured performance.

 

[rdenney]

They are repeatable. The threshold of uncertainty is rather below -120dBr.

But, as Solderdude attempted to say, that depends on the analyzer and the protocol. The issue was inconsistencies in analyses in the -95 to -105 dB range, as I saw it, and even with state-of-the-art analyzers (which most people cannot afford and do not own) there are mistakes in the setup that can cause differences like that.

Rick "whose 'external sound card' measures at a little better than about -100 dB" Denney

 

[Mneff]

Before getting into FIR Vs IIR it's useful to grasp linear Vs minimum phase.

When you make a change to the frequency response for example by blocking DC or having a roll off at high frequencies, you change the phase of the signals. So phase changes are inevitable in a bandwidth-limited system.

Thanks. That was helpful. Not. Mr. Hyper technical. If it’s not at a kindergarten level, forget it. A lot of people on this site seem to be EEs or at least have an engineering background. I don’t so maybe I shouldn’t be on it.

 

[Mneff]

I look at measurements first and seperate the ones that meet the minimum spec for audibility. Than the work starts. Features, reliability, cost, appearance, ease of use, bugs, WAF etc.
With modern electronics the measurements are usually the least of your concerns.

Yes. That’s the way I use this site.

 

[Jim Taylor]

As one of my favourite dealer said to me there isn’t better in HiFi just individual preferences so you have to decide for yourself

That is incorrect. There is both "better in Hi-Fi" on one hand and personal preferences on the other hand.

Better in Hi-Fi is the accumulated result of decades of scientists creating more accurate microphones, more accurate recording equipment, more accurate electronic circuits and more accurate loudspeakers. Because of their hard work, we can (if we wish) enjoy a closer approximation of the original event ... enjoying the original time and again, as it were.

But no one is saying that you (or I, or anyone else) has to PREFER accuracy. I happen to prefer it, but many people don't. Some people like constrained bass, and some prefer reduced treble. Some like distortion, and some like compressed dynamic range. There's nothing wrong with that.

So Hi-Fi (high fidelity, or accuracy) and preference are two totally different things. They may be closely interconnected, and on the other hand may not be interconnected at all. My personal opinion is that much of the antagonism we see in audio derives from conflating these two different (and non-related) aspects.

 

[Jim Taylor]

Thanks. That was helpful. Not. Mr. Hyper technical. If it’s not at a kindergarten level, forget it. A lot of people on this site seem to be EEs or at least have an engineering background. I don’t so maybe I shouldn’t be on it.

Don't give up. I came to this site 5 years ago, and I felt lost, like you do now. But I read books (simple ones), watched videos (again, simple ones) and tried to fathom the posts on this site posted by professionals in the audio business.
Slowly, I got to the point where I understood more. I'm no brain, and I don't know a great deal, but I've gotten to the point where I am satisfied with my insight into audio problems. At least, I know what they're talking about.

And I must say ... it feels damn good!