Master Thread: Are Measurements Everything or Nothing?

[J.M. Noble]

Care to explain how that is less dynamic than music?

My signature line is coming in handy today.

 

[dshreter]

I suspect that there’s some misunderstanding of what a multitone signal implies for the operation of amplifiers and speakers.

Multitone sounds to the ear like a steady noise and by intuition the amplifier and speakers are operating in a steady way. In reality, and this corresponds to the time domain chart just posted, it is much more like a torture test as far as the amp and speakers are concerned.

In contrast orchestral music sounds to the ear much more complex, but in terms of what the electronics must do is much simpler.

 

[MarkS]

Once again, it does not matter if multitone is or is not an adequate substitute for music, because we can do properly controlled blind listening tests on music.

 

[Newman]

Once again, it does not matter if multitone is or is not an adequate substitute for music, because we can do properly controlled blind listening tests on music.

...and when such listening tests on music are made, they tend to be less revealing than appropriate test signals...

 

[RickS]

Looks as though we had a Thanksgiving troll. Perhaps his alias was a obvious indication, but was warned and failed to respond. His account bas been spam cleaned. When I have a chance will do a more complete sanitization.

 

[MattHooper]

it doesn’t matter what was used in the creation of the record or which equipment was used by the various engineers who worked on it, all we have is the artefact itself.
Keith

This is a sensible approach.

I don’t think it’s the only sensible approach… but it certainly one of them.

 

[Purité Audio]

What are the others?
Keith

 

[RickS]

...and when such listening tests on music are made, they tend to be less revealing than appropriate test signals...

Also much easier to do a multi-tone test well than it is to do double blind listening tests!

 

[MarkS]

Also much easier to do a multi-tone test well than it is to do double blind listening tests!

Yes, but this will not convince anyone who believes that humans can detect subtleties in music that measurements do not capture.

 

[dlaloum]

Yes, but this will not convince anyone who believes that humans can detect subtleties in music that measurements do not capture.

Which brings us to a related discussion...

What is the purpose of the measurements? - If it is merely to understand the technical behaviour of that which is being measured, then in this case the multi-tone is perfectly fine, but if the purpose includes a goal of educating people who most likely cannot understand the ramifications of the multitone test... then the measurements need to be chosen and tailored to that goal as well...

Science communicators are often scientists too... but their actual job is different - and this falls squarely into that "communication" domain, albeit with a strong science link.

 

[kyuu]

What is the purpose of the measurements? - If it is merely to understand the technical behaviour of that which is being measured, then in this case the multi-tone is perfectly fine, but if the purpose includes a goal of educating people who most likely cannot understand the ramifications of the multitone test... then the measurements need to be chosen and tailored to that goal as well...

What exactly is your proposal here? What measurement would satisfy those who reflexively reject measurements in favor of "trusting their ears"?

The problem with science communication to those who have no interest in taking any time to learn about anything is that you can only simplify to certain extent before you're just not really communicating the science anymore. Good science communication is also often a full-time job for an entire team. If someone wants to start up something like the Kurzgesagt or PBS Spacetime Youtube-channels but for audio topics, that'd be great. That's not really the purview of this site, however.

 

[MarkS]

That is why I prefer to emphasize listening tests over measurements.

 

[dlaloum]

What exactly is your proposal here? What measurement would satisfy those who reflexively reject measurements in favor of "trusting their ears"?

The problem with science communication to those who have no interest in taking any time to learn about anything is that you can only simplify to certain extent before you're just not really communicating the science anymore. Good science communication is also often a full-time job for an entire team. If someone wants to start up something like the Kurzgesagt or PBS Spacetime Youtube-channels but for audio topics, that'd be great. That's not really the purview of this site, however.

The topic of this thread - the value of measurements - has a direct bearing on science communications... it is one of the valued things we can achieve via the measurements...

And I agree science communication is often a full time job - and that's part of the point I was making.

 

[MattHooper]

Which brings us to a related discussion...

What is the purpose of the measurements? - If it is merely to understand the technical behaviour of that which is being measured, then in this case the multi-tone is perfectly fine, but if the purpose includes a goal of educating people who most likely cannot understand the ramifications of the multitone test... then the measurements need to be chosen and tailored to that goal as well...

Science communicators are often scientists too... but their actual job is different - and this falls squarely into that "communication" domain, albeit with a strong science link.

As I’ve pointed out before, one issue is that many laymen audiophiles are not experienced in correlating measurements to the subjective results. In other words not able to look at big suites of measurements and determine “ what that sounds like.”

This is why they tend to communicate with one another using subjective language “ this is what that speaker sounded like.”

On the other hand, the type of people who are drawn to measurements, and to spend lots of time correlating measurements to their subjective effects, tend to be “ engineer types” who have little time for subjective language. “ I have the measurements, that tells me how something sounds, why would I need additional words?”

And so this can open up a little bit of a communication gulf between the engineer with the deep understanding of the measurements, and the audiophiles who want to know “ OK that’s how it measures but what does that mean for how it sounds?”

It means that the very engineer types who know the answer to the question tend to be the least interested in translating the measurements and sound to the subjective language that the others will understand.

In other words to bridge this gap, it’s not going to work as well, if you have a natural disinclination to describe sound or an allergy to putting measurements or sound into subjective descriptions.

That’s why it’s pretty hard to find somebody who is good at both. That’s one of the valuable functions I think Stereophile’s John Atkinson has served over the years, given he tries to explain measurements in terms of correlating to subjective impressions.

Erin of Erin’s Audio Corner started off, struggling with this. He was constantly littering his subjective portion of the reviews with caveats about how he wasn’t that good or that interested in all the audio lingo when trying to describe the sound. But the fact is in order to become a successful a communicator as he has become, he had to come up with ways of expressing sound/measurements in subjective descriptions. And he’s more comfortable with it now. And it’s clear from the comment section for his videos that his audience really appreciates not just the measurement portion of his videos, but Erin’s attempts to describe “ what that sounds like.”

For Amir’s reviews, the measurements are great and can be very helpful IF you know how those correlate to the subjective consequences. And many people attracted to ASR seem of the mind “ I’m not here for flowery descriptions just give me the numbers.” And Amir seems generally of the same mindset.

I do appreciate the subjective portion of Amer’s reviews, as much as he downplay their usefulness, and personally, I would prefer to see more detail in the descriptions.

 

[dlaloum]

As I’ve pointed out before, one issue is that many laymen audiophiles are not experienced in correlating measurements to the subjective results. In other words not able to look at big suites of measurements and determine “ what that sounds like.”

This is why they tend to communicate with one another using subjective language “ this is what that speaker sounded like.”

On the other hand, the type of people who are drawn to measurements, and to spend lots of time correlating measurements to their subjective effects, tend to be “ engineer types” who have little time for subjective language. “ I have the measurements, that tells me how something sounds, why would I need additional words?”

And so this can open up a little bit of a communication gulf between the engineer with the deep understanding of the measurements, and the audiophiles who want to know “ OK that’s how it measures but what does that mean for how it sounds?”

It means that the very engineer types who know the answer to the question tend to be the least interested in translating the measurements and sound to the subjective language that the others will understand.

In other words to bridge this gap, it’s not going to work as well, if you have a natural disinclination to describe sound or an allergy to putting measurements or sound into subjective descriptions.

That’s why it’s pretty hard to find somebody who is good at both. That’s one of the valuable functions I think Stereophile’s John Atkinson has served over the years, given he tries to explain measurements in terms of correlating to subjective impressions.

Erin of Erin’s Audio Corner started off, struggling with this. He was constantly littering his subjective portion of the reviews with caveats about how he wasn’t that good or that interested in all the audio lingo when trying to describe the sound. But the fact is in order to become a successful a communicator as he has become, he had to come up with ways of expressing sound/measurements in subjective descriptions. And he’s more comfortable with it now. And it’s clear from the comment section for his videos that his audience really appreciates not just the measurement portion of his videos, but Erin’s attempts to describe “ what that sounds like.”

For Amir’s reviews, the measurements are great and can be very helpful IF you know how those correlate to the subjective consequences. And many people attracted to ASR seem of the mind “ I’m not here for flowery descriptions just give me the numbers.” And Amir seems generally of the same mindset.

I do appreciate the subjective portion of Amer’s reviews, as much as he downplay their usefulness, and personally, I would prefer to see more detail in the descriptions.

We need a couple of florists added to our electrical workshop!

 

[J.M. Noble]

Speaking as a musician who has professional experience in audio and music-related technical jobs: the artist in me wins. If it sounds good, I don't care how it measures. If there's a problem, measuring/CAE etc. are helpful in figuring out why and possibly how to improve it. But they aren't the arbiters.

 

[RexrothPigeon]

Speaking as a musician who has professional experience in audio and music-related technical jobs: the artist in me wins. If it sounds good, I don't care how it measures. If there's a problem, measuring/CAE etc. are helpful in figuring out why and possibly how to improve it. But they aren't the arbiters.

Also speaking as a musician with similar professional experiences, I’m painfully aware that our sighted listening assessments are unreliable—we’re unable to assess “if it sounds good” because our lowly ears are at the mercy of our eyes and other sources of bias.

 

[J.M. Noble]

Also speaking as a musician with similar professional experiences, I’m painfully aware that our sighted listening assessments are unreliable—we’re unable to assess “if it sounds good” because our lowly ears are at the mercy of our eyes and other sources of bias.

Unless someone is paying me, I don't feel the need to make those kinds of judgments. Does it "sound good"? Cool. It's like guitars or other instruments: which one is the "best"? It depends, and I'm not fixated on chasing something that doesn't exist (i.e. "the best").

As it happens, people do pay me to listen to music as a live sound engineer (which necessarily entails a lot of artistic choices in mixing the show). I habitually use a SPL meter (can't really blast pink noise at the gigs I work, so no SMAART etc) but again it pretty much comes down to my taste tempered by what I think will please the audience.

Just like in audiophile circles, I see live sound people go off in the weeds with all the DSP power available in modern digital mixers. They chase "flat" response, etc., and kind of forget about the fact that they are more akin to a bandleader or director than a studio technician and that no measurement is going to tell them how it should sound. I walk into their mixer scenes from time to time and have to undo a lot of stuff to get the dynamics back into the music (to say nothing of the propensity for feedback).

I bought a Schiit Lokius for my home stereo. I use multiband parametric EQs all the time in my work, but a six-band glorified tone control gets the job done at home: I measure to get a starting point, and then I make it sound pleasing without regard to "fidelity", whatever that is. When people have paid me for PA system tuning (DJ-oriented joints), I get a baseline response that is kind of on speaking terms with "flat plus bass" and then have a conversation with the person paying me while listening to stereo playback. I give them snapshots of the result of our work and then encourage them to use it as a jumping off point and to not be afraid to use their taste.

But it pleases others to play with their stereo systems just like a lot of hot rodders love working on their cars more than driving them. That's cool too.

 

[RexrothPigeon]

Unless someone is paying me, I don't feel the need to make those kinds of judgments. Does it "sound good"? Cool. It's like guitars or other instruments: which one is the "best"? It depends, and I'm not fixated on chasing something that doesn't exist (i.e. "the best").

As it happens, people do pay me to listen to music as a live sound engineer (which necessarily entails a lot of artistic choices in mixing the show). I habitually use a SPL meter (can't really blast pink noise at the gigs I work, so no SMAART etc) but again it pretty much comes down to my taste tempered by what I think will please the audience.

Just like in audiophile circles, I see live sound people go off in the weeds with all the DSP power available in modern digital mixers. They chase "flat" response, etc., and kind of forget about the fact that they are more akin to a bandleader or director than a studio technician and that no measurement is going to tell them how it should sound. I walk into their mixer scenes from time to time and have to undo a lot of stuff to get the dynamics back into the music (to say nothing of the propensity for feedback).

I bought a Schiit Lokius for my home stereo. I use multiband parametric EQs all the time in my work, but a six-band glorified tone control gets the job done at home: I measure to get a starting point, and then I make it sound pleasing without regard to "fidelity", whatever that is. When people have paid me for PA system tuning (DJ-oriented joints), I get a baseline response that is kind of on speaking terms with "flat plus bass" and then have a conversation with the person paying me while listening to stereo playback. I give them snapshots of the result of our work and then encourage them to use it as a jumping off point and to not be afraid to use their taste.

But it pleases others to play with their stereo systems just like a lot of hot rodders love working on their cars more than driving them. That's cool too.

Nothing wrong with any of that, of course, just a gentle reminder that all the many people who post here to insist that they are "trusting their ears" simply are not, since they are not listening blind. That's the only way to use our ears for truly critical audio evaluation. For your and my listening at home? who cares! do what we want! But it's not generalizable, and no musicians I know want their live sound engineers to think of themselves as "band leaders" or "directors" (shit, few topics are more contentious in music than the matter of who is leading or directing the band hahahaha).

Anyway since blind listening is inconvenient/onerous/impossible in most circumstances, we need measurements to help us bridge the gap and counteract our biases.

(As for tinkering with our personal stereo systems, well, that's another reason to seek out uncolored systems so we can tinker to our preferences rather than being boxed in by the equipment's limitations.)

 

[rdenney]

The purpose of engineering measurement is to confirm that the system under test meets a spec, fulfills requirements and meets needs (as a part of an acquisition process); or that it attains objectives (when integrating into intended activity in subsequent use). All of these are tied to usage, and usage of audio equipment is mostly but not entirely in the hearing domain. Controlled subjective testing (note the word “controlled”) is as valid as measurement of technical performance, if tied to those objectives.

A multitone test, like most test protocols, is valuable because it stresses the system similarly (actually, more rigorously) to usage, but we actually know much more about it and can therefore determine the performance of the system more efficiently and with greater resolution.

But not everyone has the same use cases and objectives, and that’s where communication with nontechnical users should focus.

Rick “but this is a technical forum” Denney