"Bias" of some members towards headphone measurements?

[preload]

For the most part ASR defines sound quality as fidelity, and according to scientifically derived standards (Harman) when there is not a linear relationship between input and output. This is useful because fidelity can be directly quantified, but as you note, you might want something else / more.

Not so fast. Fidelity and sound quality, while related, are absolutely not the same thing. Fidelity implies reproducing sound as closely as possible to the original. That's fine for SS devices, but very cumbersome to define and measure for loudspeakers (unless your living room is a replica of the mastering studio). Sound quality, OTOH, is a subjective, human-reported characteristic or judgement about a sensory input that is neurally processed.

Harman recognized this dilemma and in their research, the chose not to chase loudspeaker "fidelity" (whatever that means), but rather, listener preferences as their research target. They wrote about why they decided to study listener preferences to inform loudspeaker design. I would encourage anyone interested to read about it.

The data can take you pretty far but I don't think most members here believe the data is the destination.

You'd be surprised.

 

[markanini]

Not only using rock music, but their research on the effect of the style of music on people’s performance to determine the quality of speakers and identify the speakers or headphones they prefer most showed rock music performed the best.

View attachment 423585

SoundStage! Access | SoundStageAccess.com (GoodSound.com) - Dr. Sean Olive on 35 Years of “Fast Car”

"We had the statistical evidence that people are better able to do the tests—they come up with answers much faster and much more consistently and with much more discrimination".

I always start with familiar rock songs to quickly evaluate new audio gear. Awesome to see this has some data behind it.

 

[Salt]

Harman is an approximation to hearing preference -> Psychoacoustic.
Measurement is refering to DUT performance-> Electromecanicalacoustic.

So what (to do):

Same as ever: straigt forward to linear FR, and after this: EQ (that was not available in past decades) to what is desirable/needed.

There was a comment inside this bubble that commented something like "I don't want distorsion baked in" ... but want to do it my way
.....

 

[preload]

This doesn't follow from the above. Please give examples. Also, please extend this concept to measurements, like should there be rock and roll settings as well as orchestra settings for a Klippel? And if so, what would those settings and measurements be?

You're going to need to clarify what it is you don't understand. Perhaps you don't understand what Olive's chart (posted by Geert) is conveying? Or you don't understand the concept of generalizability of research findings to populations that weren't included in the study? (all of the Harman published research of loudspeaker listener preference used rock music only, as far as I've seen, there may have been exceptions)

 

[MAB]

You're going to need to clarify what it is you don't understand. Perhaps you don't understand what Olive's chart (posted by Geert) is conveying? Or you don't understand the concept of generalizability of research findings to populations that weren't included in the study?

I do understand, but you already knew that.
I don't understand you.
Give an example of a speaker that sounds good with rock (for instance) and sounds bad with another genre of music. It's what you are implying. Please demonstrate what you imply, rather than an unproven thought experiment that you are promoting.

edit: Also, since you make such sweeping generalizations, to be convincing it will be necessary to provide multiple examples or entire classes of speakers that somehow cannot perform well outside of rock, or whatever.

 

[preload]

I do understand, but you already knew that.
I don't understand you.
Give an example of a speaker that sounds good with rock (for instance) and sounds bad with another genre of music. It's what you are implying. Please demonstrate what you imply, rather than an unproven thought experiment that you are promoting.

An OBVIOUS example would be rap/hip-hop, where bass quantity and distortion characteristics of the speaker would have a disproportionate affect on listener preferences. I would not expect the loudspeaker preference rankings derived from rock music listening to be the same as those for rap/hip-hop.

Also, this is a science forum, and I "promote" scientific principles. If a research experiment did not study a specific population, you don't know if the findings apply.

 

[kemmler3D]

HOWEVER, this also unfortunately limited the generalizability of the research because it is then not clear that their conclusions also apply to non-rock music. That's the problem.

The reason for using rock music is it has more, denser, harmonic content and at higher sustained levels than other genres. This means you don't miss as many features of the speaker's performance by virtue of the music not having any content at a given frequency.

Rock music is good for evaluating speakers in the same way that unfiltered daylight is better for evaluating printers than LED or fluorescent.

To put it another way, other genres don't have anything rock doesn't, as a tool for evaluating speakers. And, I would say it doesn't impair the generalizability of the Harman results in any obvious way.

An OBVIOUS example would be rap/hip-hop, where bass quantity and distortion characteristics of the speaker would have a disproportionate affect on listener preferences. I would not expect the loudspeaker preference rankings derived from rock music listening to be the same at all.

Definitely arguable, but it's not like rock is devoid of content below 60hz. And, for what it's worth, the Harman research actually did find that bass extension / quantity is really important, so somehow they managed to get the right answer anyway.

Even speaking as a fan of hip-hop I'd say it's worse for evaluating speakers than rock, overall... but when I did evaluate speakers using music in my job, I actually used rock AND hip hop (and classical, and electronic) in my 'suite' of test tracks. The reason was specifically to test bass capability.

If a research experiment did not study a specific population, you don't know if the findings apply.

Fair point. I don't know if Amir or others argue that the Harman research is the final word. I think it's the best we've got for now and so it's the best bet for a standard.

 

[Jim Taylor]

In the context of what I was saying, it's my subjective preference for the perceived sound reproduction when listening to familiar material

Subjective preference - or subjective anything, for that matter - cannot be predicted. Subjective assessment is emotional, changeable, non-logical and non-quantitative. Science, OTOH, is based on quantification, reproducibility and logic, and rejects emotion.

Using science, measurements and logic to quantify subjective "feelings" is like fish singing opera.

 

[Peterinvan]

I have said it before: everyone's ears are different; everyone's listening room is different. Always audition the gear if possible before buying.

When looking for new equipment to buy, I see the ASR measurements as a way to highlight any glaring issues with the gear that would put me off (like the 6KHz spike on the Sennheiser 800). So I can skip the audition of such gear. We are lucky to have a great store here in Vancouver for auditioning headphones, earphones, DAPs, and amps.

 

[preload]

Subjective preference - or subjective anything, for that matter - cannot be predicted. Subjective assessment is emotional, changeable, non-logical and non-quantitative. Science, OTOH, is based on quantification, reproducibility and logic, and rejects emotion.

Using science, measurements and logic to quantify subjective "feelings" is like fish singing opera.

I'm not sure where you got your definition of science from. Science simply requires the ability to observe and measure. You absolutely can quantify and measure human preferences. Or are you saying that the published research by Olive in AES were not "science"?? Good grief

 

[kemmler3D]

Sound quality, OTOH, is a subjective, human-reported characteristic or judgement about a sensory input that is neurally processed.

Yes, I should have been more clear - "sound quality" in that subjective sense isn't measured at ASR or anywhere else. Amir measures for fidelity when that is possible and measures for compliance with Harman when it is not, because it happens to be the most research-backed standard.

I assume if a more rigorously defined standard was developed, he'd switch to measuring speakers / headphones against that.

 

[Jim Taylor]

I'm not sure where you got your definition of science from. Science simply requires the ability to observe and measure. You absolutely can quantify and measure human preferences. Or are you saying that the published research by Olive in AES were not "science"?? Good grief

Primarily, I'm contrasting the unemotional and rigorous qualities of science with your statement, " ... it's my subjective preference ...". Please forgive my sometimes imperfect way of communicating my thoughts.

You had said, " ... The quicker people get over the fact that FR measurements of speakers/headphones aren't as predictive of SQ as they wish they were, ... "

Since you have defined SQ as " ... my subjective preference ...", the statement now becomes ...

"The quicker people get over the fact that FR measurements of speakers/headphones aren't as predictive of subjective preference as they wish they were, ..." and I had pointed out that you cannot tell the degree other people wish for something.

Therefore, I have a suspicion (that's all it is ... just a suspicion) that the statement should have been, "I can't get over the fact that FR measurements of speakers/headphones aren't as predictive of my subjective preference as I want them to be.".

That is a totally valid reaction.

Still, the statement of Toole and Olive that listeners prefer neutral reproduction over non-neutral reproduction applies. You or I can use FR graphs, dispersion characteristics (if discussing speakers), distortion results and compression characteristics to tell what is more neutral and what is less neutral.

The problem is, as I said, using that information to predict your subjective preference as accurately as you wish. I'm not sure whether a long period of time to train in correlation between measurements and preferences would yield fruit or not. Measurements don't change with time, mood or weather ... preferences can, and do.

So I hope you can see why I considered subjective preferences as an indefinite, constantly moving target, contrasted with scientific measurements, which are well-defined and reproducible across time and distance.

 

[preload]

So I hope you can see why I considered subjective preferences as an indefinite, constantly moving target, contrasted with scentific measurements, which are well-defined and reproducible across time and distance.

You seem to be making the argument that loudspeaker preferences change over time? If so, I'm not sure I agree with that opinion, but I see what you're saying.

 

[preload]

To put it another way, other genres don't have anything rock doesn't, as a tool for evaluating speakers. And, I would say it doesn't impair the generalizability of the Harman results in any obvious way.

You seem to be saying that even though the published Harman research that correlated listener preferences with analyzed measurements were only performed on rock music, the results should be generalizable "without impairment" to other musical genres, such as rap/hip-hop. If that's the case, then it's difficult for me to continue the conversation because we're not applying the same accepted principles of scientific research interpretation. I don't want to waste your time.

 

[Jim Taylor]

You seem to be saying that even though the published Harman research that correlated listener preferences with analyzed measurements were only performed on rock music, the results should be generalizable "without impairment" to other musical genres, such as rap/hip-hop. If that's the case, then it's difficult for me to continue the conversation because we're not applying the same accepted principles of scientific research interpretation. I don't want to waste your time.

Did you mean to say "the same accepted principles of data interpretation"? Which principles might those be? After all, as long as the signal has the necessary bandwidth, do speakers discriminate between music genres?

Somewhere on this site is a list of the recordings used in Dr Toole's work. Perhaps someone would be so kind to reproduce the list and post it here, so we can see whether there is some sort of deficiency. It would be interesting.

 

[NTK]

Did you mean to say "the same accepted principles of data interpretation"? Which principles might those be? After all, as long as the signal has the necessary bandwidth, do speakers discriminate between music genres?

Somewhere on this site is a list of the recordings used in Dr Toole's work. Perhaps someone would be so kind to reproduce the list and post it here, so we can see whether there is some sort of deficiency. It would be interesting.

The Harman playlist was discussed in this thread starting with this post.

Should I be here, I don't think I'm an audiophile

An audiophile is someone who would recoil at the thought of using regular cables and interconnects rather than thick expensive audiophile cables and interconnects as they swear there's an enormous sonic difference and can always identify tiny nuanced details in the sound that no one else hears...

www.audiosciencereview.com

This is just another variation of the complaint about measuring gear using testing tones — i.e. "we don't listening to sine waves/sweeps".

 

[Jim Taylor]

The Harman playlist was discussed in this thread starting with this post.

Should I be here, I don't think I'm an audiophile

An audiophile is someone who would recoil at the thought of using regular cables and interconnects rather than thick expensive audiophile cables and interconnects as they swear there's an enormous sonic difference and can always identify tiny nuanced details in the sound that no one else hears...

www.audiosciencereview.com

This is just another variation of the complaint about measuring gear using testing tones — i.e. "we don't listening to sine waves/sweeps".

Thank you, @NTK !

I notice that it says here, "While no single song is sufficient to fully evaluate a loudspeaker or headphone, certain tracks are particularly well-suited for the evaluation of specific attributes. For example, tracks with broadband, spectrally dense instrumentation are used for spectral judgements, while dynamic tracks with percussion and low bass are used for testing dynamic range and distortion."

I can see that the choice of recordings goes far beyond petty affection for style or genre.

 

[Old_School_Brad]

I'm not talking about EQ'ing a bad speaker and ending up with sound like a Revel Salon.
I'm talking about EQ'ing resonances and ending up with worse performance. Or no better, just a different form of bad.
That's the point, many speakers will sound worse after an attempt (for instance) to flatten the on-axis performance.

Well, that “different form of bad” might still be an improvement over the original. Whether EQ makes things worse or better depends on how it’s applied. Poorly executed EQ -like blindly trying to flatten the on-axis response, can definitely cause problems, especially if it introduces off-axis issues or other artifacts. But when EQ is applied thoughtfully, targeting specific problem areas without overcorrecting, it can make a meaningful difference.

For instance, while EQ can’t completely eliminate resonances, it can reduce their audible impact at the listening position. It’s not a perfect fix, but it’s often better than leaving those issues untreated. The key is using EQ in a way that enhances the overall sound without creating new problems.

I think you’ve made an important point about flattening the on-axis response. The real goal should be to optimize the sound at the actual listening position, where it truly matters, especially with objectively poorly measuring with uneven frequency responses or poor dispersion, and in typical reflective rooms, EQ can play a big role in improving what you hear by addressing peaks, dips, and room-related effects.

 

[preload]

Did you mean to say "the same accepted principles of data interpretation"?

Nope, when I wrote "accepted principles of scientific research interpretation" that's exactly what I was referring to. In this case, it's the principle of generalizability.

After all, as long as the signal has the necessary bandwidth, do speakers discriminate between music genres?

The question is NOT about speakers discriminating between music genres, but rather whether the relationship between loudspeaker measurement analysis and listener preference is maintained with non-rock recordings?

Consider the rap/hip-hop music example, where recordings are bass-heavy and where vocals are mixed with compression and aren't exactly reference quality:
Speaker A: 6.6" woofer with very smooth midrange/treble response and smooth directivity
Speaker B: 15" woofer with excellent bass, but a "batwing" response in the midrange/treble?

How might Speakers A and B fair with rock vs rap/hip-hop? Same relative ranking? or different?
Think about it.

 

[SuicideSquid]

You seem to be making the argument that loudspeaker preferences change over time? If so, I'm not sure I agree with that opinion, but I see what you're saying.

I think the argument being made is that our subjective preferences are always in flux and necessarily hard to pin down, partly because they're purely subjective and partly because there's a large body of evidence suggesting that our preferences for everything are affected by all kinds of things and are constantly in flux. You can be served two identical meals at a restaurant and you'll still likely perceive them as different based on who served them to you, the plate and arrangement, the time of day, your mood, etc. As every hi-fi salesman knows, you can change how a person perceives a loudspeaker by priming them with certain words before they listen to the speakers.