Why is audio objectivism so frequently focused on all the wrong things?

[Sgt. Ear Ache]

That's pretty much what I said But, while you or I may not be interested in preferences of others, that's not true of everyone else. Floyd, et al, did in fact measure subjective preference through blind testing. And found common patterns. That's useful information, and something worth studying further, for example to determine the source of such preferences. Or to find other populations that may not share the same preferences. All of this is information that can feed into further discoveries about how we hear what we hear. After all, this is how science works.

Agreed, but again, that doesn't seem to me to be the primary goal of objectivism in the audio hobby...

 

[Fluffy]

What do you mean by "don't agree with"?

I don't agree that flat sounding speakers/headphones are universally preferred. And in preference I mean subjective preference – accuracy is a different matter. But the enjoyableness of a transducer isn't necessarily connected to a flat frequency response. See what I said earlier about the Harman target response for headphones.

 

[DonH56]

You should read Dr. Toole's book.

 

[MediumRare]

To be blunt, I don't care what somebody else likes, even if it's a large group. ... So why should I listen to what someone else likes?

Here's a real possible affirmative answer: While few people are in literal terms "identical", in practical terms we all fall into segments. If you could be told what "segment" you are in and what a statistically valid set of people also in your segment prefered, you would be more than halfway to finding choices that would thrill you. You may revel in the journey, the search, the idea you are unique. That's OK too.

 

[pkane]

There are two possibilities here:

1. Almost all people like the same thing (in the NRCC/Harman speaker testing, for instance, when they control for bass, they could predict preferences with 98% certainty based on the on- and off-axis response of the speaker), in which case that thing is objectively correct, and a target that everyone should shoot for.

2. Different groups of people like different things, in which case there is no objectively correct thing, and it's legitimately just a matter of taste.

If you think measurements have value, and that it's meaningful to speak of things as "better" or "worse" based on their measurements, then you're already implicitly believing in the first possibility, right? (Or just for the sake of thoroughness: alternately, you believe in the second possibility, but also believe based on no evidence that a) there's a group of people who prefer the thing that measures better according to these specific measurements, and b) you're in that group. But this doesn't seem like a likely constellation of beliefs, tbh.)

Not quite. Measurements are useful in determining how far or how close a device is to the 'ideal'. For audio playback, the ideal device is the one that reproduces the recording with no distortions. This has nothing to do with anyone's preference and everything to do with science, design, and engineering.

 

[GrimSurfer]

That's pretty much what I said But, while you or I may not be interested in preferences of others, that's not true of everyone else. Floyd, et al, did in fact measure subjective preference through blind testing. And found common patterns.

Done while under employment of Harman IIRC, hence the term "Harman curve". It was clearly market research, not that such things don't play a legitimate role in the audio business. But it can lead to things such as honing products to a certain age/income group based on listening responses* instead of a broader objective of determining what humans like.

[* Age and income, especially disposable income, are related. So too are income levels, where white collar workers usually make more than blue collar ones... AND the hearing acuity of a white collar worker or a professional is likely to be higher than a blue collar worker who has spent a great deal of time working in a construction or industrial zone. So groups' responses to sound could be mapped to specific products. Let's keep this in mind before extrapolating Toole's work beyond its intended bounds!]

You should read Dr. Toole's book.

Yes. And consider carefully the problem that particular research was trying to solve.

 

[MediumRare]

There are two possibilities here:

Honestly, you may be confusing yourself. Both of those statement are true but for different measurements, or for different levels of a given measurement.

Easy example: Some like no salt on their french fries, some like some, but nobody likes a tablespoon of it.

 

[MediumRare]

You can use it for preference. You should first use DBT to determine the level needed for a detectable difference, and then follow with a test for preference. Food industry does this all the time.

Especially when cost-optimizing a recipe. But beware creeping-meatballism!

 

[Mikey]

I don't agree that flat sounding speakers/headphones are universally preferred. And in preference I mean subjective preference – accuracy is a different matter. But the enjoyableness of a transducer isn't necessarily connected to a flat frequency response. See what I said earlier about the Harman target response for headphones.

For speakers, the evidence is strongly against you -- if you think that you prefer speakers that aren't anechoic-flat with smooth off-axis response, the odds are that you're just wrong and don't know your own preferences.

For headphones, I think it's fair to say that the science isn't as proven-out yet. But with what is there, even Sean Olive believes that different individuals will have different preferences: "The MOA tests have told us that the exact amount of preferred bass/treble will vary depending on age, listening experience, gender, and program material. Younger listeners and less experienced listeners generally preferred slightly more bass and treble than older, more experienced listeners. For senior listeners (55+ years old) we found they preferred on average even less bass but more treble than younger listeners. We believe this is related to hearing loss. This is an educated guess because we didn’t measure the hearing of the subjects (except Harman trained ones) and we need to test a larger sample. So the final target satisfies the majority of listeners, although it won’t be perfect for all programs and all people. We believe some degree of personalization is necessary to satisfy all listeners.”

 

[DonH56]

Done while under employment of Harman IIRC, hence the term "Harman curve". It was clearly market research, not that such things don't play a legitimate role in the audio business.

Ask him, he posts here. The research was actually done at the NRCC before his time with Harman.

 

[pkane]

Done while under employment of Harman IIRC, hence the term "Harman curve". It was clearly market research, not that such things don't play a legitimate role in the audio business. But it can lead to things such as honing products to a certain age/income group based on listening responses* instead of a broader objective of determining what humans like.

Was it? I believe the work started before Harman paid for it.

 

[Mikey]

Not quite. Measurements are useful in determining how far or how close a device is to the 'ideal'. For audio playback, the ideal device is the one that reproduces the recording with no distortions. This has nothing to do with anyone's preference and everything to do with science, design, and engineering.

Yes, but what's "ideal"? To go back to loudspeakers, for instance, one theory is that they should be time coherent, that all components of the waveform should arrive at your ear (or the measuring microphone) at the same time. This was something proposed by John Dunlavy, in particular, and it's a design philosophy he put into his speakers. And the measurements show that his designs worked in this regard: "Look at the SC-IV's step response in fig.4. Pretty ideal—the outputs of all the drive-units arrive at the microphone at pretty much the same time. This, by definition, is time coherence. As a result, the SC-IV is one of the only two loudspeakers I've encountered that can produce a good squarewave shape."

Conversely, Toole and Olive believe that time coherence is irrelevant. If you measure the step response of a Revel speaker designed to their philosophy you get a totally incoherent response: "Turning to the time domain, the F228Be's step response on the tweeter axis (fig.7) indicates that all of its drive-units are connected in positive acoustic polarity, with the tweeter's output arriving at the microphone before the midrange unit's, which in turn arrives before that of the woofers."

So now if you're going to say that the ideal is to reproduce everything with no distortions, then it's obvious that Dunlavy is correct, right? The difference is very measurable in really obvious-to-see ways.

But in fact, Toole and Olive's research shows that time coherence doesn't matter. It doesn't predict listener preference, and the human ear appears to be totally insensitive to it. Stereophile continues to collect this measurement for all the speakers it measures, and there are manufacturers who continue to optimize their designs for it, but the best evidence we have -- that is, blind preference tests, done in statistically significant numbers -- shows that it doesn't matter.

Which again is the very core of my argument: Measurement without a theory behind it can mislead as easily as it can illuminate. If you don't have a coherent theory of which measurements matter, at what levels, and the blind test data to back that up, you can't really present a meaningful set of measurements except by coincidence. Even if you're going for "accuracy."

 

[Fluffy]

For speakers, the evidence is strongly against you -- if you think that you prefer speakers that aren't anechoic-flat with smooth off-axis response, the odds are that you're just wrong and don't know your own preferences.

For headphones, I think it's fair to say that the science isn't as proven-out yet. But with what is there, even Sean Olive believes that different individuals will have different preferences: "The MOA tests have told us that the exact amount of preferred bass/treble will vary depending on age, listening experience, gender, and program material. Younger listeners and less experienced listeners generally preferred slightly more bass and treble than older, more experienced listeners. For senior listeners (55+ years old) we found they preferred on average even less bass but more treble than younger listeners. We believe this is related to hearing loss. This is an educated guess because we didn’t measure the hearing of the subjects (except Harman trained ones) and we need to test a larger sample. So the final target satisfies the majority of listeners, although it won’t be perfect for all programs and all people. We believe some degree of personalization is necessary to satisfy all listeners.”

As for speakers, I could definitely be wrong, because I have yet to investigate my true preference in that area. Currently I own speakers based on their affordability. I hope to delve deeper into finding out my preferences in speaker sound signatures in the future.

As for headphones, I appreciate his disclaimers, but I've seen the negative consequences of claiming that there is a generally desired target response. Their suggested response is not a bit off in bass and treble for me – it's completely off. In another thread there was a guy claiming that headphones that I like very much are in fact objectively bad because they didn't conform to their response at all. And in another discussion another person suggested automatically correcting headphones to the Harman target based on measurements, as a way to achieve better sound. People are taking those targets to heart way too much, and even recommending headphones they haven't heard based on how much they conform to the target response. I think this is getting a bit absurd, and even counterproductive. People should find their own target response, and not rely on an overall preference to determine what should sound good to them.

 

[Blumlein 88]

Serious question: Aren't microphones at the professional level "solved"?

Not the way everything else is. All the in between electronics can be so clean as to be transparent imparting no sound of their own. Some gear is made to intentionally alter that. Pure fidelity is possible except for transducers.

Microphones generally don't have perfectly flat frequency response. Even the best to my knowledge are not flat the way electronics are. They also have variable directional responses. And distortion at least at the extremes aren't below .1%.

https://www.bksv.com/-/media/literature/Product-Data/bp2032.ashx

Here is a data sheet on a B&K measurement microphone. FR is +/- 2 db. One can use calibration curves to improve on that. One would also need to correct for off axis effects and maybe use a shroud to even out diffraction related FR changes. 3% distortion at 146 db SPL max input. It would be lower at lower SPL's, but I don't know that it would be under .1% even then. Diaphragm resonance frequency of 11 khz where it has a 90 degree phase shift. This for a 1 inch size capsule.

Now I could spend many hours making a list of electronically transparent DACs etc which are reasonably affordable and available. Few microphones would equal the one I note above, and any that get close are very expensive. I couldn't come up with such a list to choose from that is very large at all. And that is before we talk about self noise levels. Plus in general recording people voice recordings using non-linear microphones on purpose.

 

[MediumRare]

Didn't you say you're in the wine business? I honestly don't see the connection to the audiophile world. Wine (or food and drinks in general) can be compared to music preferences, for which people usually accepts a wide variety of preferences. The studies done by Harman and others to determine preference of sound signature don't produce different preferences for different groups, but one over-arching preference to everyone – "flat sound is good", basically. And that is the thing I don't agree with.

And as for how many people are close to the mean, there's an easy counter argument for that regarding our case – audiophiles are not close to the mean to begin with. The very niche-ness of this hobby already puts us in a group that is probably far away from the mean in a lot of respects. There is no reason to expect that audiophile's taste reflects the general population taste, and vice versa.

I believe you just proved my argument: You place yourself (and even audiophiles in general) in a different segment from the mass. Here are some similarities between wine and audio:

- A product which depends on sensory perception in numerous ways
- A product for which emotional connection (even brand affiliation) are part of the product
- A product for which price and quality are only vaguely correlated
- A product which "experts" perceive differently from average people
- A product with a robust "critic" industry who make their living telling people what to like, but are paid by producers
- I could continue but I'm ready to move on....

Oh, I believe now you're in MY wheelhouse
No, microphones are far from solved. As in with speakers, there are many flavors and attributes that make some microphones preferable for certain applications or tastes.

What I mean is that microphones, AFAIK, have now been designed to accurately capture basically any sound we want and at an attainable cost. So the problem is not "no microphone has been invented to do the job we need them to do", so, i.e. that problem has been "solved".

EDIT: @Blumlein 88 has convinced me I was completely wrong about that!

I'm in agreement with your statement about the range of choices.

 

[pkane]

Yes, but what's "ideal"? To go back to loudspeakers, for instance, one theory is that they should be time coherent, that all components of the waveform should arrive at your ear (or the measuring microphone) at the same time. This was something proposed by John Dunlavy, in particular, and it's a design philosophy he put into his speakers. And the measurements show that his designs worked in this regard: "Look at the SC-IV's step response in fig.4. Pretty ideal—the outputs of all the drive-units arrive at the microphone at pretty much the same time. This, by definition, is time coherence. As a result, the SC-IV is one of the only two loudspeakers I've encountered that can produce a good squarewave shape."

Conversely, Toole and Olive believe that time coherence is irrelevant. If you measure the step response of a Revel speaker designed to their philosophy you get a totally incoherent response: "Turning to the time domain, the F228Be's step response on the tweeter axis (fig.7) indicates that all of its drive-units are connected in positive acoustic polarity, with the tweeter's output arriving at the microphone before the midrange unit's, which in turn arrives before that of the woofers."

So now if you're going to say that the ideal is to reproduce everything with no distortions, then it's obvious that Dunlavy is correct, right? The difference is very measurable in really obvious-to-see ways.

But in fact, Toole and Olive's research shows that time coherence doesn't matter. It doesn't predict listener preference, and the human ear appears to be totally insensitive to it. Stereophile continues to collect this measurement for all the speakers it measures, and there are manufacturers who continue to optimize their designs for it, but the best evidence we have -- that is, blind preference tests, done in statistically significant numbers -- shows that it doesn't matter.

Which again is the very core of my argument: Measurement without a theory behind it can mislead as easily as it can illuminate. If you don't have a coherent theory of which measurements matter, at what levels, and the blind test data to back that up, you can't really present a meaningful set of measurements except by coincidence. Even if you're going for "accuracy."

Speakers are complicated. But that's not what's being measured here on ASR. Look at my signature. The software in the link allows you to measure the difference between two waveforms. If one is the original and the other one is captured at the output of a DAC or amplifier, you can, very objectively, measure the difference between two devices and know how far away they are from ideal and what distortion they add to the original waveform. It's true: DeltaWave doesn't measure speakers

 

[GrimSurfer]

Ask him, he posts here. The research was actually done at the NRCC before his time with Harman.

Why is it called the Harman curve and not the NRC curve then? (Rhetorical question to which neither of us may know the true answer!)

I would want to know the research question and who it was ultimately for, because there are any number of ways it could go (commissioned by Harman but conducted by the NRC, conducted by the NRC then purchased by Harman, purchased under an exclusivity agreement and or an NDA, etc. Co-funded research, public private initiative, etc)

To fully appreciate the answer you need to completely understand the question it was intended to address. You also need to know the identity of the customer because that provides additional insight into the scope of the research.

This might seem overly inquistive but the results of that study are being used as the foundation for much of what audiophiles (and others) believe. It is mentioned by many at ASR with the the same convictions as one might use the term "gravity".

So understanding the context is important before one can consider it as a canon of audio.

[Nothing written here should be taken as an attack against the professionalism or motivations of Dr. Toole. Rather, it should be interpreted as a sign that the results of his research are being taken seriously enough to warrant the most careful and complete consideration possible.]

For @Floyd Toole: Please see the questions above, Dr., and understand that my curiosity pertains to trying to understand the background story of your research on speaker response prefs (which I find fascinating because of its foundational impact).

 

[Mikey]

Speakers are complicated. But that's not what's being measured here on ASR. Look at my signature. The software in the link allows you to measure the difference between two waveforms. If one is the original and the other one is captured at the output of a DAC or amplifier, you can, very objectively, measure the difference between two devices and know how far away they are from ideal and what distortion they add to the original waveform.

And the question remains: To what extent does this matter to the human ear, and is there a waveform that's preferred universally, or do individuals have individual preferences? Without knowing the answer to those questions, you don't know what's significant anymore than you do when looking at speakers' step responses.

 

[MediumRare]

For speakers, the evidence is strongly against you -- if you think that you prefer speakers that aren't anechoic-flat with smooth off-axis response, the odds are that you're just wrong and don't know your own preferences.
”

Based on my many years of working with consumer preferences, I find it completely inconceivable that each person perceives sound the same and hence would have the same preference in speakers. Add to that differences in the types of music listened to and different levels of attentiveness.

I mean, that's just obvious when you think about it, isn't it?

 

[Island_Kenny]

This is my fundamental disagreement. I think that the blind test is the thing that matters, because the ultimate goal of an audio playback chain isn't to make an analyzer happy, it's to make humans happy. The analyzer is only a proxy for what humans like, and it only has value if its measurements can be correlated in a meaningful way against human preferences.

If you're not doing that human-perception check on it, you can't actually know whether you're measuring the right thing. Look at the example of TIM in the early solid-state days, as I linked above, for instance: By using gallons of negative feedback, manufacturers were able to create amps that measured amazingly, but sounded bad; later on, they discovered that a thing they hadn't been measuring was correlated to that heavy use of negative feedback, and that's what was causing it to sound bad.

If you didn't take seriously the human-perception check there, you'd just go forward, making things that "measured great" via measurements that didn't capture all the relevant factors, but that were actually pretty bad. And if you were to castigate "subjectivists" for not liking your "great measuring" stuff, you'd be the one who ultimately had egg on your face, not them.

By training, engineers work on numerical measurements, and more often than not they have a very narrow focus and do not see the big picture of consumer experiences at all.