Master Complaint Thread About Headphone Measurements

[Smaestro]

That would sound extremely muffled if that were the case.
Studio monitors should have a flat response on axis, just like hifi speakers b.t.w.
Its the music content (SPL) that, only on average, is similar to pink noise. For that you thus need speakers with white noise spectrum.
It is best to use speakers using pink noise (and then undo that slope to obtain the speaker response) because tweeters and midrange speakers can not handle the energy of white noise at high SPL and would burn. They don't when using pink noise as there is less energy in the upper bands.
Speakers in a room change that to a gradual downwards slope with some resonances (boosts and cuts) in the lower part of the frequency range but not in a pink-noise kinf of way.

I say... engineers should mix so that listeners in a room hear 'natural and powerful sound'.

Unfortunately they also have to take into account the usage of phone speakers, car audio, boom boxes, medium quality speakers and even all kinds of headphones.

Ah yes, excuse me. It makes no sense at all now that I think about it.

I guess the idea I tried to articulate is the opposite: mix on an inverse pink noise response, so that the audio (while working on it) follows a white noise spectrum. Under the assumption that this would improve the energy spread of frequencies. But pink noise is already the equal energy default, and white noise isn't. So yeah. Good thought experiment.

As a sidenote, about mixing for lower quality audio. In the end it's a net plus, that more people can experience music, regardless of quality. Worth it, IMO.

 

[Smaestro]

As for "mixing headphones", I suppose (I don't work in this field) it would be different if a headphone were in fact tuned to provide a colouration that aids in discerning aspects of the mix while not actually intending for that tonality to be heard by the consumer. But if those "mixing headphones" are indeed being used as a stand-in to speakers for guiding EQ adjustments, then tonal standardization would be necessary lest those mixing headphones perpetuate the Circle of Confusion.

I agree with your whole point.
As far as what I'd like from mixing headphones, is an additional emphasis on certain areas. For example:
I'd love it if headphones are a bit over sibliant (in the 7-9kHz).

Let's say a song would have a "perfect" amount of sibilance. The mixer tries to achieve this target (as fast as possible I'll add). However, the effect of missing that target in both directions is not equal. If you have a little bit too much sibilance, it can completely ruin the song, making it unlistenable. On the other hand, having not enough sibilance, does not ruin a record. There are so many songs that have a 'th' instead of an 's' sound, and while it's not making the song better, it also only makes it a little bit worse.

In my opinion of course.

Does this add to the circle of confusion? Depends on if the mixer is actually aware of this trait and uses it. If he is aware of the effect and uses it well, then it would not. And it would help in speeding up the process, as there usually isn't that much time reserved for mixing.

 

[splitting_ears]

I agree with your whole point.
As far as what I'd like from mixing headphones, is an additional emphasis on certain areas. For example:
I'd love it if headphones are a bit over sibliant (in the 7-9kHz).

Let's say a song would have a "perfect" amount of sibilance. The mixer tries to achieve this target (as fast as possible I'll add). However, the effect of missing that target in both directions is not equal. If you have a little bit too much sibilance, it can completely ruin the song, making it unlistenable. On the other hand, having not enough sibilance, does not ruin a record. There are so many songs that have a 'th' instead of an 's' sound, and while it's not making the song better, it also only makes it a little bit worse.

In my opinion of course.

Does this add to the circle of confusion? Depends on if the mixer is actually aware of this trait and uses it. If he is aware of the effect and uses it well, then it would not. And it would help in speeding up the process, as there usually isn't that much time reserved for mixing.

That's a fair point, although I don't see why headphones should have a sibilance boost built in. We don't expect that from speakers, do we?
It also means you're stuck with that 7-9kHz boost even when you're not working. Hardly a plus in my book.

If you need to zoom into certain spectrum areas, why not create a few EQ presets to see how the music holds up in different scenarios? Limited bandwidth, bass boost, treble boost, etc? That should be easy to set up in any DAW.

 

[Smaestro]

That's a fair point, although I don't see why headphones should have a sibilance boost built in. We don't expect that from speakers, do we?
It also means you're stuck with that 7-9kHz boost even when you're not working. Hardly a plus in my book.

If you need to zoom into certain spectrum areas, why not create a few EQ presets to see how the music holds up in different scenarios? Limited bandwidth, bass boost, treble boost, etc? That should be easy to set up in any DAW.

I fully agree. I had the same thought eventually, but you beat me to it.

I do this already with a couple of other monitoring presets that I can toggle with a single click, among which:

Mono
LR channel swap
Left channel only, and Right channel only (on both ears)

Phone (low and high pass, with some resonance peaks)
Bass energy only (low cut)
Fletcher Munson (adjustable for the 20 dB down from the upper line)
Cross feed (with switchable EQ, delay, speaker angle and room size)

Small effort to add some sibilance, there's no reason to have any of this baked in.


I did have another idea, written below, but I disprove the idea by the end. I'm leaving it up here because I typed it already and perhaps the thought experiment is useful for others.

Hypothesis: For audio engineers, a flat and smooth frequency response is more important than a bumpy but Harman accurate one. The logic being that a flat response can be EQ'd perfectly into Harman, while a bumpy approximation of Harman can hardly be improved any further.

So you rate headphones on how accurate they can reach Harman with an reasonable EQ, for example 8 band PEQ, rather than out of the box performance.That's what Amir already does, but, it's not really accurate. For example, a lot of headphones FR slope down towards the subbass. This downward curve is then usually corrected with a Butterworth or flat shelving filter, but this only brings the average subbass level up, it doesnt correct the slope. A better correction is to use a different shelving filter or add another bell at 5Hz for example.

But...... Ok on second thought the current review system is exactly what I'm proposing, only with perhaps different limits of how much complexity the EQ is allowed to have. So never mind haha, all is well, the current system is fine.

Only if the manufacturer gives instructions with their headphones on how to EQ, can this make sense. (Inaccuarate out of the box, accurate Harman with EQ.) Have any manufacturers done this so far? If not, I wonder why not. Its a pretty easy way to get more sales from the Harman crowd.

 

[solderdude]

The wheel has already been invented. Not all wheels are perfectly round and have proper bearings though and not all wheels look alike (except for being round)
An 'average' target has been arrived at that suits the majority of headphone users (consumers).
The majority means there is another group that has a different preference which can be because of various reasons.

The problem with suggesting any EQ is that such would have to be based on measurements and requires a good seal for instance and a certain way to wear the headphone.
It is the main reason why Oratory1990 for instance shows which bands to EQ to taste and because of limitations in the used measurement gear can only suggest EQ up to a certain frequency (where it already is inaccurate).

So whatever strategy one can think of has already been done and considered.

Personal taste is not the same as scientific correct and that requires ideal circumstances which aren't really present in real life and test fixtures only adhere to a standard they were designed to meet.
Ones ears may not meet that standard, in fact most ears won't so adjust to taste on reproduction and on the recording side 'learn' the sound of the headphone or EQ it towards the sound of the monitors in the control room at the mixing console.

Harman is not perfect nor are other targets nor will EQing to some target, obtained on some fixture using some protocol not automatically lead to the correct EQ on ones head (if only for product variance).

 

[Smaestro]

After delving into all the information of Dr. Toole and Dr. Olive and other information, kindly provided by Amirm and the members of this forum, I have formed a couple of key points regarding audio production. I'm curious to your thoughts.

1. Under the assumption that an audio engineer (mixing and/or mastering) works best on the dr. Toole recommended speakers+room (flat-in-an-anechoic-room, placed in a semi-reflective room, user at ideal location), it follows that the ideal average FR for mixing headphones should be Harman OR 2013, not OR 2018, as OR 2013 was measured to be equal to the ideal room+speakers.
   
   
2. Recommending Harman OR 2018 adds to the circle of confusion. Harman OR 2018, as used in the reviews on ASR, is the most common preference. That makes it valid recommendation to consumers from a buyers guide perspective. However, if the goal is to end the circle of confusion, then OR 2018 should not be used, as it deviates from the optimal room+speakers as determined by dr. Toole.
   
   
3. Using any other FR during audio mixing or mastering does not contribute to the circle of confusion, as long as the audio engineer is so used to that FR that his/her recollection of music is based on that FR. As music always gets mixed and mastered to fit its genre, the product will not have more or less of any frequency region than it's predecessors. Of course, the basic condition must be fulfilled that all normally audible frequencies are at indeed audible and not cut-off. However, there is another hard condition on this:
   
   
4. If the studio speakers+room deviate from the 'dr. Toole recommendation', then the headphones used should deviate from OR 2013 in an equal way. This to keep the studio and the headphones as similar as possible. The Engineer is free to swap around between different FR's if desired, but unnecessary differences should be prevented.

 

[solderdude]

You could consider that the lack of tactile feel, opposite speakers, might lead to a preference of a bit more bass and more relaxed treble.
Harman research is about preference and there is no single correct preference but in the end the choice was made to create some (highly smoothed) target that would satisfy most listeners. It is nothing more than that.

The fact that the curve was revised a few times should tell you enough that it isn't as dry cut as it seems.
Then there is production variance, seal issues, placement differences, deviating preferences, sometimes interaction with amplifiers.

3 & 4: requires a mixing or mastering engineer that really knows his headphone or knows how to EQ it.
Judging from the abundant usage of DR compression and tonal accuracy of many (pop and rock) recordings some of these guys are way of the mark.
Fortunately there are also a lot of excellent recordings of which we often do not know if headphones are used and in what stage/purpose of the mixing/mastering process.

 

[markanini]

It's easy to casually express doubts on the Harman targets. I've yet to see anyone provide concrete data to validate any concerns. Even my personal critique lacked any quality data, at best I had some criticism on methodology rigor of the Harman In-ear target, compared to the over-ear target. And that's not much. It took Sean Olive himself validating a rather divergent target as statistically equal to Harmans. I may be leaving out some nuance, but that's the lone substantial development after 5 years of discussion on the internet.

Harman research targets the Bruel & Kjaer 5128

For the first time, the Harman IE 2019 headphone response target gets tested on the leading test fixture for headphone measurements.

www.soundguys.com

 

[Smaestro]

It's easy to casually express doubts on the Harman targets. I've yet to see anyone provide concrete data to validate any concerns. Even my personal critique lacked any quality data, at best I had some criticism on methodology rigor of the Harman In-ear target, compared to the over-ear target. And that's not much. It took Sean Olive himself validating a rather divergent target as statistically equal to Harmans. I may be leaving out some nuance, but that's the lone substantial development after 5 years of discussion on the internet.

Harman research targets the Bruel & Kjaer 5128

For the first time, the Harman IE 2019 headphone response target gets tested on the leading test fixture for headphone measurements.

www.soundguys.com

So it looks more like the 'casual doubters' had a point all that time.

The people who dont like Harman mostly critique that the low shelf and high bell are too loud, or too smiley / V shaped.

The Soundguys curve, which has just that, scores statistically equivalent to the best Harman curve.

I've yet to see anyone provide concrete data to validate any concerns.

I dont really understand what you mean by this, because:

Harman research is about letting trained and untrained listeners express a preference for a curve. It carries value because of its large sample size and presumably well controlled test conditions.

The 'casually doubters' of Harman, are similarly both trained and untrained listeners who have done the same preference research, on themselves. The same research, but with n=1.

Genuine question, if you disregard these personal observations as 'not concrete data', what other concrete data are you expecting from forum users?

 

[markanini]

So it looks more like the 'casual doubters' had a point all that time.

The people who dont like Harman mostly critique that the low shelf and high bell are too loud, or too smiley / V shaped.

The Soundguys curve, which has just that, scores statistically equivalent to the best Harman curve.



I dont really understand what you mean by this, because:

Harman research is about letting trained and untrained listeners express a preference for a curve. It carries value because of its large sample size and presumably well controlled test conditions.

The 'casually doubters' of Harman, are similarly both trained and untrained listeners who have done the same preference research, on themselves. The same research, but with n=1.

Genuine question, if you disregard these personal observations as 'not concrete data', what other concrete data are you expecting from forum users?

I can't disregard personal data, otherwise I'd be EQ-ing my devices to a curve I don't prefer, so it's not valueless to me. Either way there's always something valuable about research that elevates it beyond the circumstantial, which comes down to documented repeatability. Imagine an alternate timeline where the outcome of Harmans research suited your personal taste perfectly. Would that change your attitudes, would you side with casual doubters that don't produce comparative data?

 

[Doodski]

I can't disregard personal data, otherwise I'd be EQ-ing my devices to a curve I don't prefer, so it's not valueless to me. Either way there's always something valuable about research that elevates something beyond the circumstantial. Imagine an alternate timeline where the outcome of Harmans research suited your personal taste perfectly. Would that change your attitudes, or would you keep siding with casual doubters?

I'm totally off in left field as per PEQ settings. So I appreciate what your comment states.

 

[Hugo9000]

@Doodski That frightens me every time you share it haha! Unless you have to correct for some seriously wonky headphones with those settings.

 

[Doodski]

@Doodski That frightens me every time you share it haha! Unless you have to correct for some seriously wonky headphones with those settings.

Yes, I've worked around it 6 ways past Sunday and I still get the same PEQ settings no matter how I try to approach the matter. My ears are apparently messed up but in life I have no hearing issues. I hear people fine, I hear noises well and am not hearing impaired or limited.

 

[CedarX]

So it looks more like the 'casual doubters' had a point all that time.

Casual doubters always have a valid point, but what they tend to forget is that it’s just a point, or a personal preference.

Too often I read: this [pick your name] target does not work at all for me and “many people” agree, therefore this target is absolutely wrong.

 

[Smaestro]

I can't disregard personal data, otherwise I'd be EQ-ing my devices to a curve I don't prefer, so it's not valueless to me. Either way there's always something valuable about research that elevates it beyond the circumstantial, which comes down to documented repeatability. Imagine an alternate timeline where the outcome of Harmans research suited your personal taste perfectly. Would that change your attitudes, or would you keep siding with casual doubters?

Yes, I think so, hard to say. Perhaps the idea of being validated by research will change my judgment. For now, I think I'd still align with the skeptics.

My skepticism hasn't got to do with my faith in the validity of the research. I agree that the Harman curves are well-researched and it's great work. I am fully behind in doing research the proper way. The recent Senselab research also shows that adapted Harman curves are still among the best scorers. Great to have peer review. So up to here, I'm not really a skeptic I think.

Yet, I can't shake my doubts about the applicability in the pro world:

First, as far as ideal FR goes, both you, I, and Doodski already agree that we like something a bit different.

My process is that I've given different curves a fair share of time, both in listening and working with. And I simply end up at a much flatter curve, about 4dB less of bass and pinna gain than OE 2018.

>> To address your edit, the point of 'comparative data':
I AB'd my reference track library with a number of curves with two different headphones.
If I were to share this data, I'd have to edit my reference tracks to be EQ'd differently, and then use Lacinto ABX to do a AB Shootout between my reference tracks, and post screenshots of the results. But what does that tell you more than me (or whoever) simply typing that I found a preference? It's a lot of extra effort for something that isn't peer-reviewable as it's a personal observation. Unless you mean something else and I misunderstood.



Second, there are the engineers I know of, who as far as I could find out, don't use Harman compliant headphones.
Example: in an interview with John Hanes he says:

"I'm using the PSB M4U1 Headphones. I like these in particular because they sound pretty similar to my ProAc speakers and they are not heavy. "

Hanes also specifically mentions to use them for detail and to check the bass. This is the FR of the M4U1:

Pretty remarkable curve considering the bass and detail comment...!!

Yet, John Hanes is a 13 Grammy-winning mix engineer and he works with Serban Ghenea (23 Grammys). IMO they are the most influential mix engineers in pop music in the last 20 years. Specifically, they mixed a lot of Pharells work. I don't know which songs Senselab 2023 research used, but one of them is a Pharell song. I guess that if a random well-produced modern pop song is used in the Harman research, there's a decent chance that they've had their hands on it.

Given these strong different preferences by people who, as far as I can expect, know what they're doing, I conclude that subgroup preferences for different curves exist.

Edit: This wouldnt show up in the Harman research I think, as even though 30+ headphones were tested, the results were summarised into 3 clusters of listeneres. Pro-outlier preferences could/would be lost in that .


>> So do I side with people who don't post comparative data? In general: I have seen enough gear debate on other forums to know that without ABX, a lot of things can be claimed. At the same time, I have seen people pass unlikely ABX tests, and even myself have passed ABX tests on DACs... a test which I want to redo with different samples as it is unlikely to be audible.

The point is that I know many people can and do test themselves well, and most people serious about their work do. And even if the test flatters their ego, they retest. That's why I put quite a bit of weight on professional opinions. Ok, there are also people claiming world-changing events from expensive power cables or musically timed clock jitter (real example). But they are recognizable because they can't explain themselves properly, never show ABX results or post samples, and if they do, the samples are botched.

As an aside: the most recent Senselab research has another curve that scores very high (HP3), which is said in this article to be very different. Along with a couple of others that scored great as well. I don't have an AES membership so I can't read what that curve is like, but I'm curious.

SoundGuys headphone preference curve validated in AES paper

A scientific study on headphone sound shows our response curve is among the most preferred!

www.soundguys.com

 

[Smaestro]

I think this long write-up is beside the point. I understand Harman is a well-researched preference, I have no qualms about that. The point I was responding to initially was this statement by Amir:

Those folks should do their own studies demonstrating which headphone allows them to make better mix decisions.

Until then, listeners prefer the extra bass because that is what "real" sounds to them. By mixing against that consumer target, then we get standardization of music creation and playback. Using an entirely different response for mixing/mastering music means that we don't hear what they heard. This, by definition is wrong.

The first bolded part is about that if the producing side of music has disagreeing opinions, then Amir states that they should do their own research. I fully agree with that.

The second bolded part, is not right. It doesn't make sense.

1) According to this statement, if the engineer has a very different preference than Harman, then apparently he should just suck it up and listen in a way he doesn't find enjoyable. I wonder how great recordings will turn out then.
2) It builds on a misunderstanding about the creation side of music:

All music is produced, mixed and mastered in comparison to its genre. The only time when you hear engineers say they use no references anymore, is when mastering engineers specifically say they've been using the same system exclusively for so long that they don't need it anymore. All engineers will always start with references, and basically only a very small subset grows skilled enough to not need it, and having been attuned to their system is mandatory for that.

In result: only if audio professionals tune their ears to a different system than the one they work on, will they create some sort of circle of confusion in their brain.
But to make an analogy: that would be the same as if an F1 driver is practicing for his race in a speedboat instead of their race car. If he botches his race because the car behaves differently, then that is fully a failure on the drivers part. Not the fault of the speedboat and racecar industry not agreeing on a single standard.

 

[Robbo99999]

One thing I have issue with is the assumption that "neutral" or "for mixing" is necessarily "sterile" or "analytical" etc. From my understanding, the idea is that if you have a sonic event and multiple people witness it, they will each by their own ear physiology etc. have their own unique tonal experience of that sonic event's spectra; if that sonic event is captured with neutral microphones and played back with neutral speakers in a manner that relays the sonic event with close to the same incidence as the original, ideally those same people who witnessed the original sonic event would be able to hear an accurate reproduction of that original event while still having their ears apply whatever the unique tonal differences. Hence, the point is to standardize recording and playback tonality so that if the audio engineers record a sonic event and modify it e.g. to have warmth or "fun", that sonic information will pass through the chain unmodified, to be received by everyone's unique physiology. Sure, everyone has different ears, but the intent is to present the same original sonic information before said individual ears apply their personal tonal changes; if someone's ears have more ear gain than the average, the solution for the goal of accuracy is not to give them headphones or speakers with relaxed ear gain since in the presence of the original sound event, their ears would have still possessed that boosted ear gain; otherwise, it is fine if such an ear gain relaxation allows them to hear that sonic event more pleasantly when they couldn't before with the live stimulus. I would want to preserve the original (corrected) spectral balance as far down the chain as possible before applying listener preference.

As for "mixing headphones", I suppose (I don't work in this field) it would be different if a headphone were in fact tuned to provide a colouration that aids in discerning aspects of the mix while not actually intending for that tonality to be heard by the consumer. But if those "mixing headphones" are indeed being used as a stand-in to speakers for guiding EQ adjustments, then tonal standardization would be necessary lest those mixing headphones perpetuate the Circle of Confusion.

I agree, but the only thing to keep in mind is that headphones react differently to anatomy than a couple of speakers placed in front of you, so in order to get to the ideal situation you're describing then that requires in ear measurements of a couple of ideal speakers in your ideal room and then also in ear measurements of your headphone in question - you would then match the headphone to the in ear speaker measurement using EQ. It's not really possible to create a headphone that is going to be completely natural for everyone, for these reasons. The idea of the Headphone Harman Curve is to approximate neutrality, that is if we equate user preference (from the Harman studies) with neutrality, which we know is true when it comes to speakers - Anechoic Flat........so yes Headphone Harman Curve is approximated neutrality which will sound better to some individuals than others, just by nature of the fact that not everyone has the same ideal neutral measured curve when it comes to headphones, and that's due to anatomical differences between people, which you can't get around completely by using a fixed target curve.

 

[Mr. Haelscheir]

I agree, but the only thing to keep in mind is that headphones react differently to anatomy than a couple of speakers placed in front of you, so in order to get to the ideal situation you're describing then that requires in ear measurements of a couple of ideal speakers in your ideal room and then also in ear measurements of your headphone in question - you would then match the headphone to the in ear speaker measurement using EQ. It's not really possible to create a headphone that is going to be completely natural for everyone, for these reasons. The idea of the Headphone Harman Curve is to approximate neutrality, that is if we equate user preference (from the Harman studies) with neutrality, which we know is true when it comes to speakers - Anechoic Flat........so yes Headphone Harman Curve is approximated neutrality which will sound better to some individuals than others, just by nature of the fact that not everyone has the same ideal neutral measured curve when it comes to headphones, and that's due to anatomical differences between people, which you can't get around completely by using a fixed target curve.

I am well aware and note the importance of in-ear measurements in regard to those claiming they've "EQed two headphones to the same target" and had them sound very different: https://www.audiosciencereview.com/...a-headphone-review.50705/page-57#post-1853760 (post #1,137) was a key post of mine. My first paragraph was very much based on speaker neutral with said sonic event being more easily "repeatable" between different individuals listening from the same vantage point; everyone is having the "same" sound being presented to their ears prior to their unique anatomy's filtering, yet once we involve headphones, the coupling happens to be such that each individual requires a different compensation curve to match their respective speaker response, as you say necessitating an approximation/averaging. I had covered in https://www.head-fi.org/threads/rec...-virtualization.890719/page-121#post-18027627 (post #1,812) how I found blocked canal measurements to be troublesome for the aim of producing personal compensations, whereby I am working on a probe mic solution.

 

[Robbo99999]

.... yet once we involve headphones, the coupling happens to be such that each individual requires a different compensation curve to match their respective speaker response, as you say necessitating an approximation/averaging.

Yes, good, that we agree on.

 

[JaguarIT]

I would like to complain about too many people complaining about EQ factoring in to recommendations.

It's very trivial to ignore the EQ-based conclusion and recommendations and make up your own mind based on the non-EQ data, if you so desire, and many, many users, including myself, find the EQ data highly useful! If nothing else I finally learned how to use parametric EQ.

I appreciate and benefit from all the objective data, with and without EQ and am self-directed enough to not be bothered by parts of a review that transmit information I don't care about. Heck, I can even manage to not read whole reviews I don't care about! I don't even visit most audiophile review sites, because I've been misled over and over with subjective opinions, not grounded in objective measurements, spouting hype and nonsense.

I love this site, thank you. Reminds me of the days of NwAvGuy, and Marlene's musings before they disappeared. Both spent a lot of time trying to get head-fi.org users to not be blinded by industry hype and actual lies and misrepresentations and their voices have been missed.

NwAvGuy fairly reviewed and subsequently destroyed (just a negative review, not real destruction, lol) lots of expensive gear by objectively proving it was actually garbage, made fraudulent claims, or offered zero audible benefit at great cost and helped me find objectively good gear at reasonable prices. Marlene's musings did the same and objectively proved that lossy hires have inaudible changes from lossless hires. Read that again - Marlene's musings objectively proved that lossy hires transcodes are completely transparent to their lossless hires source tracks with real measurements. Plus objectively proved that transcoding low-bitrate lossy to higher bitrate lossy is objectively worse than the source lower bitrate lossy though quantized noise summing. Marlene also helped Fiio redo their MP3 decoding code so it actually worked correctly. Therefore everyone that bought from Fiio after her involvement with them through head-fi, which was right during their first ever DAPs being released, directly benefited from improved MP3 playback and decoding. Literally improved sound for thousands to millions of users.

Anyway, this forum has already benefited me greatly by pointing me to great cheap gear that crushes much more expensive gear.

I wish there were more iem and headphone reviews, but I won't complain about that, because I am too concerned about my own gear getting messed up to volunteer it, so I choose to not be a hypocrite about that. Keep up the great work!

Thanks!