Master Thread: Are measurements Everything or Nothing?

[BDWoody]

Problem with most audiophiles is that they don't get to audition many products side by side in similar conditions ( but their high ego won't let them admit it ). So the list of measurements can be a "more reliable guide" then those mostly "managed" subjective reviews on many channels and websites. But again the " high ego" of audiophiles will claim that every amp sound same, every dac sound same, measurements don't relate to sound, measurements are inaudible etc etc. Tell me which would you rely on more the data which is irrefutably measured or mere play of words ?

I'm not really sure what that all means, but telling someone they are at the wrong place if they use the information offered here in a way that you may not 'get' misrepresents the site.

I moved your post out of that review thread and here where it is more appropriate.

 

[Pokee1016]

Thank you for the thorough and detailed review. I'm a little late to the party but came upon this when researching audio streamers. I understand that there is little room for subjectivity in the real of actual measured performance, it seems that the perspective on this has settled into two camps, one that swears on measure performance and the other that takes the actual audio experience into account.
There can be no doubt that the measured performance of any given audio unit is crucial to the performance of said component however I am curious about the output of this device and it's performance in the analog realm The design and performance of this section is absolutely crucial to what you are listening to.
And Merry Christmas, we met and I did some work for your company Madrona Digital a while back (2008?). I admire what you do and appreciate your thorough insight.

 

[escksu]

Thank you for the thorough and detailed review. I'm a little late to the party but came upon this when researching audio streamers. I understand that there is little room for subjectivity in the real of actual measured performance, it seems that the perspective on this has settled into two camps, one that swears on measure performance and the other that takes the actual audio experience into account.
There can be no doubt that the measured performance of any given audio unit is crucial to the performance of said component however I am curious about the output of this device and it's performance in the analog realm The design and performance of this section is absolutely crucial to what you are listening to.
And Merry Christmas, we met and I did some work for your company Madrona Digital a while back (2008?). I admire what you do and appreciate your thorough insight.

As long as you take humans out of the picture, meaurements is pretty much everything and straight forward. ITs easy to quantify performance.

But, once you put human inside, it makes things extremely complication and impossible to measure. Human perspective, what pple like/dislike cannot be measured. Its based on the person's own perception/feelings/external influences etc etc (too many factors to list). Thats why I have been saying what measures well and what a person like can be very different. Its perfectly normal. This applies to more than audio too, thats life.

Even for music itself. Some likes/hate certain genres, even singers etc.. Thats why there are fan clubs. I prefer classical over pop/rock. My wife like pop has has a few fav singers (I don't know how they are, never heard of them). And no, she don't like classical. Thats life.....

Of course, not everyone agrees with me... thats life too....

 

[BoredErica]

Split the difference and try 3ohm?

I just want to add, a few speakers among the top 14 passive speakers sorted by preference rating with sub dip below 4ohms. R3 is a great example of a very well measuring speaker which is also popular for its aesthetics. Goes down to 3ohms. 2ohms seems pretty out there. Not a single one of the top 14 went below 3ohms, let alone 2.

Just my 2 cents

 

[Pokee1016]

As long as you take humans out of the picture, meaurements is pretty much everything and straight forward. ITs easy to quantify performance.

But, once you put human inside, it makes things extremely complication and impossible to measure. Human perspective, what pple like/dislike cannot be measured. Its based on the person's own perception/feelings/external influences etc etc (too many factors to list). Thats why I have been saying what measures well and what a person like can be very different. Its perfectly normal. This applies to more than audio too, thats life.

Even for music itself. Some likes/hate certain genres, even singers etc.. Thats why there are fan clubs. I prefer classical over pop/rock. My wife like pop has has a few fav singers (I don't know how they are, never heard of them). And no, she don't like classical. Thats life.....

Of course, not everyone agrees with me... thats life too....

Agreed that personal perspective makes it difficult to nail down performance of any given component and that's why there are reviewers (good and bad) for these components just as there are for albums. All that they can do, and i find it useful when it comes from somebody who is competent in his craft, is compare the actual sound signature to other products or use descriptively colorful words to do the same. I also agree that every reader will interpret those words differently nonetheless hi-fi reviews have been around for decades. This new school of measured performance only tend to disrespect and even condemn those who dare use descriptive language when reviewing a product. That said, people like myself enjoy reading somebody else's personal perspective on not only the sound but the appearance, history etc. I've been in the business 40 years and have even had a short stint with Amir doing some work for him. At the time I was designing product for Mark Levinson, the man not the company. We would often use words to describe the results of a circuit or component change BEFORE doing critical listening to confirm that our observations were accurate. Things like "dark sounding" or "transparent" made sense and could be confirmed. My original post is just stating that the final output section of any DAC will contribute to the sound signature and I can look at measured results all day long, fully understand what to expect, but occasionally be surprised at what I was hearing vs what the measured results told me I should be hearing. I know this is often referred to as confirmation bias but my point is that this is audio and although everyone hears something else (just look at the shape of our ears) I find it useful to add some descriptive dialogue to the review to make it more interesting and a good reviewer can nail the sound signature a majority of the time to where I know what to expect when I actually sit down and listen

 

[rdenney]

Uh huh.

As you know, we used to measure things in the audio business routinely, although not always the right things. We still do in commercial sound. One of the things we can measure is whether those things you heard were not just confirmation bias.

But now, with tools like Audio Precision analyzers, making very sophisticated measurements of the equipment is much easier, and much easier to correlate it to subjective observation. But the first thing that must happen before subjective observation can be instructive is applying enough control to those observations to validate them as being based on actual hearing and not prior knowledge. It’s the only way to reliably trust our (read: someone else’s) ears.

Recognizing these biases and insisting on controlling for them is not some new thing. It just challenges the current dominant paradigm in (particularly high-end) audio.

Once one claims that the output of a DAC defies its measurements (which, by the way, include the audio output stage) without identifying why, we lose any basis for arguing against rank stupidities, like audiophile power cords and speaker cable risers. If ears can hear things that aren’t measured without being subjected to controlled verification, then it’s open season for charlatans.

Rick “measurement is not new anywhere except non-professional audio” Denney

 

[escksu]

Agreed that personal perspective makes it difficult to nail down performance of any given component and that's why there are reviewers (good and bad) for these components just as there are for albums. All that they can do, and i find it useful when it comes from somebody who is competent in his craft, is compare the actual sound signature to other products or use descriptively colorful words to do the same. I also agree that every reader will interpret those words differently nonetheless hi-fi reviews have been around for decades. This new school of measured performance only tend to disrespect and even condemn those who dare use descriptive language when reviewing a product. That said, people like myself enjoy reading somebody else's personal perspective on not only the sound but the appearance, history etc. I've been in the business 40 years and have even had a short stint with Amir doing some work for him. At the time I was designing product for Mark Levinson, the man not the company. We would often use words to describe the results of a circuit or component change BEFORE doing critical listening to confirm that our observations were accurate. Things like "dark sounding" or "transparent" made sense and could be confirmed. My original post is just stating that the final output section of any DAC will contribute to the sound signature and I can look at measured results all day long, fully understand what to expect, but occasionally be surprised at what I was hearing vs what the measured results told me I should be hearing. I know this is often referred to as confirmation bias but my point is that this is audio and although everyone hears something else (just look at the shape of our ears) I find it useful to add some descriptive dialogue to the review to make it more interesting and a good reviewer can nail the sound signature a majority of the time to where I know what to expect when I actually sit down and listen

I too, like to read about someone's perspective too. I personally do not believe that measurements alone can tell us everything.

I am more into cars than audio and i would say its not too different and its even more focused on subjective experiences than measurements. We can measure things like weight, engine power, spring rate etc etc etc...but all these cant tell you the feel of a car. But when professional drivers get behind the wheel, their opinions about the car is what really matters.

Sadly, most of the folks here dont understand that.

 

[rdenney]

I think you may be missing the point. I don’t think people object to expert subjective reviews, if they are consistent with measurements. Amir provides these, and once in awhile is surprised by the results. The question, though, is about instincts. Amir’s natural reaction to being surprised is to “science the **** out of it” (to quote one such example), to figure out why. The alternative is to jump on the anomaly as evidence that measurements are futile.

It’s been a long time since I read car reviews, but those I remember were careful about the distinction between subjective observations and measured performance. And measured performance included specific tests (lateral acceleration, stopping distance, etc.). All the reviews I read explained any difference in subjective observations and measurements, as I recall. The instinct always seemed to me one of seeking out the reasons for any difference.

I’ve never seen a car review yet that asked people to ignore measurements in favor of disagreeing subjective impressions. But we are asked to do that routinely in the audio world.

Rick “who reads subjective reviews, too” Denney

 

[alitomr1979]

Objectivists vs. subjectivists?

I have one question for hardcore objectivists, and it is not rethorical or guiding. It is honest curiosity.

How can one measure audio quality beyond frequency response and SINAD? What I am referring to is the fact that even when instruments can reproduce a particular note, that note has a different sound, that is characteristic of each instrument. Yes, you want a speaker and components with the ability to reproduce the source content without adding anything, that is a very interesting proposition, but what about the ability to reproduce particular sounds the way instruments sound in real life? How can you measure that?

My partial objectivism (“measures are really important but never tell the whole picture”)when it comes to audio is because I am ignorant of what to actually look for/ measure.

I look forward to your thoughts.

Merry Christmas to all!!

 

[Doodski]

How can you measure that?

There is no secret sauce. No VooDOO... The math has been present for a long time that explains the electrical energy in both AC and DC forms. The means to measure and see that energy has been around for a pretty significant time. None of this is magical thinking. It's all very real and can be explained mathematically in a instance of time as a instantaneous value.

 

[NTK]

Objectivists vs. subjectivists?

I have one question for hardcore objectivists, and it is not rethorical or guiding. It is honest curiosity.

How can one measure audio quality beyond frequency response and SINAD? What I am referring to is the fact that even when instruments can reproduce a particular note, that note has a different sound, that is characteristic of each instrument. Yes, you want a speaker and components with the ability to reproduce the source content without adding anything, that is a very interesting proposition, but what about the ability to reproduce particular sounds the way instruments sound in real life? How can you measure that?

My partial objectivism (“measures are really important but never tell the whole picture”)when it comes to audio is because I am ignorant of what to actually look for/ measure.

I look forward to your thoughts.

Merry Christmas to all!!

The whole idea behind frequency response analysis/measurements is that the device is assumed to behave as a linear time-invariant (LTI) system. When a system is LTI, we can apply the principle of superposition, which is: If the system response to input signal A is X, and the system response to input signal B is Y, then the system response to input signal A + B is X + Y.

Per Fourier's theorem, all signals can be decomposed into the summation of a series of sinusoidal waves (Fourier series). Thus, the response of an LTI system to any signal is the total sum of its response to each individual component of the signal decomposed into its Fourier series. Since the individual response to each frequency can be obtained from the frequency response, the response to an arbitrary signal can be calculated using the Fourier transform of the arbitrary signal (or equivalently using the impulse response derived from its frequency response). Therefore, mathematically, frequency response alone is sufficient to fully characterize the response characteristics of an LTI system.

No system is truly LTI, and SINAD is one way (of infinite number of ways) of measuring its nonlinearity (i.e. how much the system deviates from being LTI when operating under the test conditions). Also, the system can only be LTI when it is operating within certain limits, which are also something we want to measure.

 

[alitomr1979]

There is no secret sauce. No VooDOO... The math has been present for a long time that explains the electrical energy in both AC and DC forms. The means to measure and see that energy has been around for a pretty significant time. None of this is magical thinking. It's all very real and can be explained mathematically in a instance of time as a instantaneous value.

I understand how you can measure the actual note, but not how you can translate that into how an actual instrument sound. The same note from a piano and from a trumpet sounds very different. How do you measure those differences? Is it possible?

 

[Doodski]

I understand how you can measure the actual note, but not how you can translate that into how an actual instrument sound. The same note from a piano and from a trumpet sounds very different. How do you measure those differences? Is it possible?

The note is comprised of multiple frequencies arranged as the fundamental frequency and the harmonics frequencies. At any instance of time that can be explained mathematically and graphed.

 

[alitomr1979]

The note is comprised of multiple frequencies arranged as the fundamental frequency and the harmonics frequencies. At any instance of time that can be explained mathematically and graphed.

The note is comprised of multiple frequencies arranged as the fundamental frequency and the harmonics frequencies. At any instance of time that can be explained mathematically and graphed.

Ok. I understand this. My question is: what set of metrics I need to observe in order to be able to infer how faithfully a speaker is able to reproduce a note from a piano and the same note from another instrument, say a saxophone?

What are those metrics?

I see discussions of frequency responses and SINADs and they definitely tell the story of how a speaker sounds (mostly in relation to another measured speaker) but those are definitely insufficient to know how those notes actually sound when coming from different instruments. No?

 

[dshreter]

Ok. I understand this. My question is: what set of metrics I need to observe in order to be able to infer how faithfully a speaker is able to reproduce a note from a piano and the same note from another instrument, say a saxophone

It seems you’re trying to think of a speaker like it is an instrument, but it’s a transducer that is meant to have no sound signature of its own. By recreating whatever frequency response it is fed, it reproduces the sound of the recording. The whole chain can capture the tonality of an instrument extremely accurately.

Where things break down is in capturing how sound radiates from an instrument. Let’s say you have a hypothetical acoustic guitar that radiates 75% of the acoustic energy in a beam projecting off the face of the guitar and 25% behind the guitar (not how a guitar actually projects sound btw), you can’t capture that in a recording and make the speaker do the same thing.

Instead you record it with a microphone which captures what is happening at the location of the microphone. It’s still convincing and is similar to what you would hear if listening from the location of the microphone. But it’s impossible to make a speaker make the same sound as a guitar - or any other instrument.

 

[Doodski]

Ok. I understand this. My question is: what set of metrics I need to observe in order to be able to infer how faithfully a speaker is able to reproduce a note from a piano and the same note from another instrument, say a saxophone?

What are those metrics?

To fully analyze and explain requires a multi-disciplinary approach. The root from my perspective is that standards are in place, measures are calibrated, and references are made from those. Accordingly measures are made about hearing parameters, transducer, electronics and such and they are designed to meet or exceed the hearing capabilities of many/most humans.

I see discussions of frequency responses and SINADs and they definitely tell the story of how a speaker sounds (mostly in relation to another measured speaker) but those are definitely insufficient to know how those notes actually sound when coming from different instruments. No?

Need more complete information.

 

[Inner Space]

I see discussions of frequency responses and SINADs and they definitely tell the story of how a speaker sounds (mostly in relation to another measured speaker) but those are definitely insufficient to know how those notes actually sound when coming from different instruments. No?

The way we tell the difference between two notes of equal pitch and intensity is called timbre. It's the grab-bag of stuff that says one is a piano and one is a saxophone. Harmonic structures that drift differently over time, mostly. All of it is frequency response and amplitude. Conventional metrics get you most of the way there. Less often measured, but important, would be cabinet silence, and pair matching.

 

[dshreter]

The way we tell the difference between two notes of equal pitch and intensity is called timbre. It's the grab-bag of stuff that says one is a piano and one is a saxophone. Harmonic structures that drift differently over time, mostly. All of it is frequency response and amplitude. Conventional metrics get you most of the way there. Less often measured, but important, would be cabinet silence, and pair matching.

I mentioned it above, but the one big piece not captured is the way sound radiates from an instrument —- its own spinorama if you will, and much like a speaker an instrument is more than frequency and amplitude. It makes it very difficult to truly recreate the sound of an instrument with just a couple speakers as information is fundamentally lost in the recording process. As far as I know there’s no such thing as a sonic hologram.

 

[alitomr1979]

I mentioned it above, but the one big piece not captured is the way sound radiates from an instrument —- its own spinorama if you will, and much like a speaker an instrument is more than frequency and amplitude. It makes it very difficult to truly recreate the sound of an instrument with just a couple speakers as information is fundamentally lost in the recording process. As far as I know there’s no such thing as a sonic hologram.

I was actually referring to timbre, as @Inner Space described. And I still don’t see what specific measures I need in order to make valid inferences about the precision at reproducing timbre certain speaker has.

Today I have been switching from the OG LS50 To the R3 and with their similarities, a trumpet from Chris Botti sounds different in both. Often they sound quite similar, but timbre varies. What measures tell me about those variations?

“They are quite good at it” is a subjective claim.

 

[JSmith]

what about the ability to reproduce particular sounds the way instruments sound in real life? How can you measure that?

Binaural recording - Wikipedia

en.wikipedia.org

Binaural recording is a method of recording sound that uses two microphones, arranged with the intent to create a 3-D stereo sound sensation for the listener of actually being in the room with the performers or instruments. This effect is often created using a technique known as dummy head recording, wherein a mannequin head is fitted with a microphone in each ear. Binaural recording is intended for replay using headphones and will not translate properly over stereo speakers. This idea of a three-dimensional or "internal" form of sound has also translated into useful advancement of technology in many things such as stethoscopes creating "in-head" acoustics and IMAX movies being able to create a three-dimensional acoustic experience.

JSmith