HBK Headphone Measurement Talks from Head-Fi and Sean Olive

[ADU]

Right, so the baseline of the Harman target is an in-room measurement of a Revel F208, in the Harman room, which had been equalized to a flat response at the listening position on an omni mic, measured with a GRAS 45CA, right? The original (well, "original" - 2013) Harman target was derived by applying a pair of shelf filters based on listener preferences to that baseline
View attachment 158700
This, clearly, generally works rather well, since it's the foundation of the Harman work, which predicts subjective preference well. However, because it's an in-room measurement, to achieve a smooth target, a fairly large amount of smoothing was required (@Sean Olive can correct me if I'm wrong, but people less directly connected have said it's in the 1/2-1/4 octave range). This obscures some finer HRTF features and slightly changes others (such as the shape of the rise into the "ear gain" region).

These small details don't matter as much as the general trend, which of course is why the Harman target is overwhelmingly preferred, bur being able to work with higher resolution data is handy because headphone response measurements typically show features that are much high Q than the in-room measurement's resolution can incorporate. With GRAS fixtures, the "pinna notch" feature around 9khz is the iconic example - B&K gear typically has a constructive (rather than destructive) feature in this same area. It would improve our targets to have these small details included, and with anechoic data, we can get very high resolution HRTFs to work without having room modes or reflections interfering.

So when you talk about "anechoic data" then, I guess you'd be specifically referring to free-field measurements made from different angles on a HATS rig, like the ones shown here?...

https://backend.orbit.dtu.dk/ws/portalfiles/portal/57349421/HRTF+measurements.pdf

That's not overly surprising, given that Theile's work underpins the DF target. However there is a quite substantial difference in sound between FF (a frontally located flat loudspeaker in an anechoic environment) and DF - and indeed attacking FF both on its inaccuracy under his paradigm of reproduction and its subjective non-preferability is what Theile 1986 is about.

Our thinking does seem to run along slightly similar lines, on some things. Imo though, diffuse field curves will only work as a neutral in-ear raw response curve after they've been corrected to better match the sound power response of speakers in a room. Others will probably argue that this is too simplistic an approach. But I think you can probably get a fairly good approximation for a speaker's raw in-ear response on the 5128 rig by simply combining the speaker's diffuse sound power curve with the 5128's diffuse field curve.

This probably won't capture all of the important details in the treble, especially in-between the resonant peaks at about 3, 8 and 16 kHz, as described in one of my other posts above. But I think it'll at least put you somewhere in the ballpark for the peaks in the treble. Which is certainly better than nothin at all. And probably about as good (or maybe even better than?) what the current Harman target can do on a GRAS rig.

Regarding "low frequency boost", I feel like I get into this dialogue about every 5 days at this point, but an anechoically flat loudspeaker's in-room power response doesn't show a Harman-esque bass shelf unless it maintains directivity control to an atypically low frequency, and then rapidly transitions to omnidirectionality. The Revels used track the general 1dB/oct or so slant pretty well without equalization in-room
View attachment 158707
but their slant is closer to the "B&K curve". As, of course, there's a degree of circularity in defining speakers as good based on anechoically flat response and power response that yields the preferred in-room power response (would we more prefer a non-anechoically-flat speaker with differing directivity but the same power response?).

Some actual in-room measurements of speakers might resemble the old B&K speaker curve. It is not as good a match though to the estimated in-room response curves of well-extended speakers in the currently available spinorama data though. Which tends to be a bit more withdrawn, rather than more forward in the midrange frequencies. And also more steeply sloped upward towards the lower frequencies, before the curve begins to roll off in the sub-bass.

With the 5128 DF + sound power approach that I've been talkin about though (probably ad nauseum at this point), you can use whatever response you want for the sound power curve. Including the SP response of a "grot-box", if that's what you like or need for your work. It's really entirely up to you. And if you wanted to do something similar for the GRAS, then you could use an approach like Robbo is doing with the in-room curves instead... There is more than one way of skinning a cat, as they say. And they're just different ways of arriving at a similar result imho.

If you have some other ideas on how to go about something like this though, in addition to measuring some actual speakers in a typical room (which is somethin I can't do, since I don't have a 5128 rig), I'd be interested to hear some of them.

Based on the spinorama data I've seen though (which is admittedly somewhat sparse in this particular area), I think the bass/sub-bass response on the current Harman target is probably not too far away from the sub-bass response of a loudspeaker that has a flat direct/on-axis response which is very well extended into the sub-bass frequencies using either calibration and/or a built-in sub. Or perhaps a separate sub-woofer. And that seems to be what Dr. Olive was sayin here as well.

That type of well-extend response obviously won't be everyone's listening preference though as a model for a headphone's response. So it's nice to have some other options available imo. Particularly for open dynamic headphones.

The impacts of shoulders are less than you might think in an environment with reflections, but yes, I believe the 43AG used in an earlier paper was placed in a mannequin for specifically this reason.

Good to know. (ASR doesn't seem to have a thumbs up icon, but that would go here.)

I'm unsure what "the other steps above" are, in this case - if you're referring to the Harman work in the period where in room measurements were being done (e.g. Listener Preferences for Different Headphone Target Response Curves and Listener Preferences for In-Room Loudspeaker and Headphone Target Responses), IMO the initial flattening of the speaker in situ response serves an interesting function in showing that the preferred EQ adjustments for both headphones and speakers are similar - that is, that preferred headphone frequency response and speaker power response are roughly equivalent.

Fwiw, I agree that in the context of the study they were doing, that it was interesting to see the room curve and the flattened speaker's in-ear response separated.

If you just want to see or measure the speaker's normal bass-biased in-room response at the ear, then there should be no need to separate its "room gain" from the rest of its (in-ear) response. Because you could just measure the speaker "as is" (with no EQ applied) to get its in-ear equivalent. To do something like that though, you'd probably need either a fairly expensive pair of fullrange speakers that are already well-extended in the sub-bass frequencies (if that's what you want to emulate). Or maybe a separate sub (or possibly some EQ) to accomplish the same thing, as Dr. Olive noted.

I guess it's possible that an approach like the one Harman used, with the flattened and separated in-room and in-ear responses might be easier for some folks though. (?)

I did try to re-edit my previous post on this, btw, to try to make it a bit more clear. But maybe the context was/is just too confusing to make sense of those remarks.

 

[ADU]

(would we more prefer a non-anechoically-flat speaker with differing directivity but the same power response?).

This is also an interesting question, which I'm probably not qualified enough to comment on. But based on what I have heard and read on this subject (mostly from Doctors Toole and Olive, and the other folks at Harman), I think a neutral in-room response probably requires or involves both a flat on-axis response, and also a timbrally similar (though generally more darkly-tilted) off-axis response (which would also include its overall sound power). And on a conventional speaker, it is somewhat hard for me to imagine how you could reasonably achieve one without the other, with good results. (?)

In this video, Dr. Toole seemed to stress the significance of the timbral similarity of the two response curves as being important to achieving a neutral in-room response (which also involves the speaker's directivity). The linearity and flatness of the direct response also seems to be of fairly paramount importance though in achieving reasonably precise and accurate in-room decoding of the recorded content. (Assuming the rest of the recording chain follows the same principles and is fairly linear, and mastered on transducers with a similarly neutral in-room response.)

 

[Robbo99999]

Howdy, Robbo. Since both Amir and Dr. Olive seemed to be slightly confused by the dip between 3 and 8k on the 5128 DCA Stealth plot, I thought I'd try to show and explain why that's not such an unusual feature to see on some other rigs than the GRAS systems that they're perhaps a bit more used to.

It also took me some time to figure out why the treble looked and behaved the way it did on the GRAS measurements, with the rather large notches at 9 and 15k, for example. Since I was more used to seeing measurements like the ones above from GoldenEars, Rtings and Inner Fidelity. So I suppose I sort of understand some of their confusion on this, comin at it from the opposite direction.

Oh ok, you're just trying to find out the specific reasons why they differ....on a practical level it doesn't matter though, there's either a valid target curve for a rig or not. There isn't really one for the 5128 but there is for the GRAS, it's not really interesting until the work is redone with the 5128, in the meantime people can try to come up with some calibration curves for the 5128 that approximate the Harman Target, it's not ideal and it's already been discussed a lot in this thread.

 

[Deleted member 16543]

For what it's worth, the wide dip also shows up with my binaural microphone, when measuring speakers at the listening position.

 

[DualTriode]

Hello All,

Considering the tools we have Dr. Olive at Harman has done a very good job. When you include age, gender, culture and all the other variables there is a reported 0.86 correlation coefficient between subjects’ psychological preference score and headphone frequency response curve measured on a standard test fixture.

In all the conversation here it is accepted that HBK HATS 5128 does not have a published target curve. The 5128 has different pinna and couplers.

What is ignored here is that the GRAS 45 CA used to derive the Harman Target Curve is not the typical GRAS 45 CA-10 sold today. The Pinna and coupler are the new improved versions. The pinna most likely has a different frequency notch and different pinna amplification. The current new improved version coupler has different frequency response due to different resonance and damping. The GRAS 45 CA-10 is not the headphone test fixture used to derive the Harman Target Curve. It could be as far off the Harman Curve as the 5128. There is no data.

Thanks DT

 

[ADU]

Hello All,

Considering the tools we have Dr. Olive at Harman has done a very good job. When you include age, gender, culture and all the other variables there is a reported 0.86 correlation coefficient between subjects’ psychological preference score and headphone frequency response curve measured on a standard test fixture.

In all the conversation here it is accepted that HBK HATS 5128 does not have a published target curve. The 5128 has different pinna and couplers.

What is ignored here is that the GRAS 45 CA used to derive the Harman Target Curve is not the typical GRAS 45 CA-10 sold today. The Pinna and coupler are the new improved versions. The pinna most likely has a different frequency notch and different pinna amplification. The current new improved version coupler has different frequency response due to different resonance and damping. The GRAS 45 CA-10 is not the headphone test fixture used to derive the Harman Target Curve. It could be as far off the Harman Curve as the 5128. There is no data.

Thanks DT

A very good point, DT.

I think this might potentially help to explain why some headphones with a supposedly neutral response can measure noticeably different than the Harman target on the GRAS systems used by reviewers and other graphers. Though I'm sure some of those differences could probably also be attributed to the rather excessive smoothing on the Harman curve (as I think someone else already mentioned).

If Harman did their in-ear testing and measurements of speakers with the GRAS coupler and their custom pinna in a mannikin, that could also effect some of the measurements versus using the same coupler and pinna on a rig with a flat plate (like the reviewers are using).

To be totally consistent with the Harman research and some of those in situ measurements they used to help derive their headphone target (which were apparently based more on the shape and response of an actual human head and body), the reviewers probably should be using not only the same couplers and pinnae, but perhaps also the same type of mannikin head that Harman used for those tests. At least for all of their over-ear headphone reviews and measurements. Because the coupling and response of the over-ear HPs could be a bit different on the flat surface of a regular 45CA rig versus the same pinna and coupler attached to the more rounded surface of a mannikin.

With over-ear/circumaural headphones, you can't really have it both ways (ie flat plate and mannikin head) and expect to get the same result. Because the interfacing, leakage effects, and the orientation of the headphone's drivers to the ear canal will be different on the two types of fixtures, potentially altering or effecting the response in the lower frequencies.

To put this sort of in a nutshell, there appear to be many inconsistencies between the potentially more realistic equipment, methods and conditions that Harman used to derive their GRAS-based headphone approximation curve, and the GRAS systems with flat plates which are currently in more wide use by various reviewers and graphers. Which I think could raise some significant questions about the accuracy of their results, if they are are simply blindly following Harman's recommended target response curve or rating system, without making at least a few adjustments to take some of the differences in their gear into account.

Perhaps some of these inconsistencies might also explain some of the different preferences for a neutral response in the lower frequencies vs. the Harman target's response in that area as well. (?)

 

[ADU]

If you are currently using a flat plate system like the GRAS 45CA to do all of your headphone measurements and reviews, then one possible answer to the above dilemma imo is to use the Harman target simply as a general starting point or guidepost. And then to use your own skills, instincts, and experience as both a reviewer and listener to determine what makes the best sense in the way of a neutral, or ideal, or accurate, or appropriate target frequency response for your particular setup and users.

Listening to and comparing the responses of a variety of different headphones (like Tyll used to do for his reviews) might be one way of honing some of those analytical and listening skills. Listening to some of the opinions and impressions of some other reviewers, users, and listeners who you trust that have used or listened to the same headphones might be another way. Comparing the sound of the headphones to some good speakers in a room might be another. And more formal (or even informal) blind listening tests or comparisons of your own could be another.

I personally would like to see more reviewers comparing their headphones to the sound of a good neutral, properly set up pair of loudspeakers. Probably also with a sub-woofer, for comparisons with the closed-back HPs. Though I understand that may be a bit difficult for some folks. If you are serious about this kind of thing though, then that's probably something you should try to do, at some point. There probably isn't an easy way to do actual blind listening comparisons between the two. But I think even some informal comparisons might help some reviewers to get a little better handle on what a neutral response should or might actually sound like.

It seems a bit ironic, but maybe the best advice one could give to a new headphone reviewer is to spend perhaps a bit more time listening to some good loudspeakers, while doing your HP reviews.

 

[DualTriode]

Hello @Sean Olive , @amirm , anyone that knows,


Are there any Binaural Room Scanning, BSR, products with head position tracking headphones showing up in the market place?

This BSR technology has been in the Harman AES research papers for 13 plus years.

Seems that computer eye tracking monitors are next.

AI and AR stuff?

My car knows what I am looking at, it yells at me and tell me to keep my eyes on the road.


Thanks DT

 

[Deleted member 16543]

My car knows what I am looking at, it yells at me and tell me to keep my eyes on the road.

You sure it's not your wife?

 

[GaryH]

While I agree with this sentiment for the average headphone user I don't think it is needed for experienced listeners.

That's quite the claim, that 'experienced listeners' are better at identifying good, bad and the best sounding headphones without A/B comparisons. A prerequisite of that would be the ability to better distinguish headphone coloration/transparency. Presumably you must be able to do well on Harman's How to Listen program, which was designed to train listeners to do just that - what level can you get up to?

 

[solderdude]

I believe Harman research showed that experienced/trained listeners are better at correctly identifying good sound quality.
It would be strange if training/experience would not lead to this otherwise why would there be a listener training program ?

let me ask you a question... when you hear an excellent sounding system would you recognize that or would you insist on AB-ing that with a reference system before you can tell if it really sounds good ?
When you hear crappy speakers or headphones... can you not tell they are crappy or have to drag in a reference system to tell it sounds crap ?

I have no interest in the how to listen program and to see which level I can get to.
I prefer to just listen to well recorded music on enjoyable headphones that have a good sound quality and care less about other peoples opinions of what they think of me, what I do or like or don't do.

 

[GaryH]

I believe Harman research showed that experienced/trained listeners are better at correctly identifying good sound quality.

Trained yes, specifically defined by Harman as passing at least level 8 of all tests on their How to Listen program. And that's exactly what you could demonstrate to yourself and everyone here, that you are in fact a trained listener as defined by Harman, and so your claim that you are better at identifying good sound quality has some reliable, verifiable backing in scientific research.

Please don't think I'm picking on you or anything, this is an issue with pretty much every reviewer. But also I would think particularly pertinent not only to readers of your headphone blog reviews but to the customers of your 'Kameleon' passive filters, which, due to the limitations of your flate-plate measurement rig, are developed partly by subjectively estimating the frequency response of headphones 'how you perceive them', by using as you put it "references, ears, parametric EQ and many years experience". The only 'reviewer' I know of who is an actual trained listener by Harman's standards of passing level 8 of How to Listen is Oratory, and he isn't really a reviewer at all, just giving the occasional subjective opinion of headphones on Reddit here and there, focusing his efforts on a mountain of objective measurements and EQ profiles instead.

If reviewers want their subjective judgements to be some kind of useful data point, they must in some way expect or at least hope their readers trust in their ability to adeptly discern good sound quality. In the same way that we require measurement rigs to conform to industry standards that demonstrate the accuracy and reliability of their readings, if subjective reports are to have any utility to readers, these 'measurement rigs' i.e. the reviewers ears should be subject to provable standards of accuracy too, and being a Harman level 8 trained listener is as good a standard for this as I can see. So calling all reviewers, @Resolve , @metal571 , @crinacle , @antdroid , Jude (I know you're watching ), care to give Harman's How to Listen a go and post your results?

 

[Resolve]

Trained yes, specifically defined by Harman as passing at least level 8 of all tests on their How to Listen program. And that's exactly what you could demonstrate to yourself and everyone here, that you are in fact a trained listener as defined by Harman, and so your claim that you are better at identifying good sound quality has some reliable, verifiable backing in scientific research.

Please don't think I'm picking on you or anything, this is an issue with pretty much every reviewer. But also I would think particularly pertinent not only to readers of your headphone blog reviews but to the customers of your 'Kameleon' passive filters, which, due to the limitations of your flate-plate measurement rig, are developed partly by subjectively estimating the frequency response of headphones 'how you perceive them', by using as you put it "references, ears, parametric EQ and many years experience". The only 'reviewer' I know of who is an actual trained listener by Harman's standards of passing level 8 of How to Listen is Oratory, and he isn't really a reviewer at all, just giving the occasional subjective opinion of headphones on Reddit here and there, focusing his efforts on a mountain of objective measurements and EQ profiles instead.

If reviewers want their subjective judgements to be some kind of useful data point, they must in some way expect or at least hope their readers trust in their ability to adeptly discern good sound quality. In the same way that we require measurement rigs to conform to industry standards that demonstrate the accuracy and reliability of their readings, if subjective reports are to have any utility to readers, these 'measurement rigs' i.e. the reviewers ears should be subject to provable standards of accuracy too, and being a Harman level 8 trained listener is as good a standard for this as I can see. So calling all reviewers, @Resolve , @metal571 , @crinacle , @antdroid , Jude (I know you're watching ), care to give Harman's How to Listen a go and post your results?

Well, in the case of Crin, and myself, we both publish data from standardized measurement rigs (GRAS 43AG-7 for both of us), and we don't merely rely on our ears. Jude uses the 5128, in addition to a 45CA (I believe?), and I know Ant has a fixture of some kind as well, although I'm not sure what specifically he's using at the moment. So while I generally think it's a good idea to do these kinds of listening tests, it would be wrong to assume we're simply relying solely on our subjective experiences, which of course are fallible. Moreover, requiring it of us doesn't make much sense given that the contentious elements of our subjective reports generally wouldn't be validated nor invalidated by scoring well on Harman's How to Listen. Simply put, I'm not exactly sure why you're calling on the folks who do rely on industry standard or compliant measurement systems to do this - if anything it would make more sense to ask that of reviewers who don't use these tools. But, speaking personally... I enjoy this kind of stuff so I'll be giving it a go when I have a moment.

 

[GaryH]

Well, in the case of Crin, and myself, we both publish data from standardized measurement rigs (GRAS 43AG-7 for both of us), and we don't merely rely on our ears. Jude uses the 5128, in addition to a 45CA (I believe?), and I know Ant has a fixture of some kind as well, although I'm not sure what specifically he's using at the moment. So while I generally think it's a good idea to do these kinds of listening tests, it would be wrong to assume we're simply relying solely on our subjective experiences, which of course are fallible. Moreover, requiring it of us doesn't make much sense given that the contentious elements of our subjective reports generally wouldn't be validated nor invalidated by scoring well on Harman's How to Listen. Simply put, I'm not exactly sure why you're calling on the folks who do rely on industry standard or compliant measurement systems to do this - if anything that would make more sense to ask that of reviewers who don't use these tools.

As this is a science-based forum, there aren't likely to be many subjective-only reviewers on here. If you know of any, feel free to tag them. I'm aware you and Crinacle do industry standard measurements, and I think that's great and adds very useful and much appreciated data for the community. I think we can all agree though that the end goal is to be able to correlate these objective measurements with subjective experience, but to do that you need both to be accurate and reliable to a verifiable standard. You have that on the objective side, now I'd say it's time to look more closely at the subjective side. Listener training via How to Listen would help towards that ultimate goal of correlating the subjective with the objective, as Harman's research has shown. No it wouldn't validate or invalidate subjective reports, but it would increase their probable accuracy and reliability and so trust in them which could lead to further investigation of their potential objective correlates.

But, speaking personally... I enjoy this kind of stuff so I'll be giving it a go when I have a moment.

Brilliant, looking forward to seeing your results

 

[solderdude]

Please don't think I'm picking on you or anything, this is an issue with pretty much every reviewer. But also I would think particularly pertinent not only to readers of your headphone blog reviews but to the customers of your 'Kameleon' passive filters, which, due to the limitations of your flate-plate measurement rig, are developed partly by subjectively estimating the frequency response of headphones 'how you perceive them', by using as you put it "references, ears, parametric EQ and many years experience".

I got this for you (2014) and that was really easy to pass and means nothing but was fun to do back then.

You avoided my question and did not answer it but instead insisted on how unreliable everything I do and say is based on your 'objective' views and demand proof (even from real reviewers) or at least feel I owe this to people who visit my website to publish test results of which you later say:

No it wouldn't validate or invalidate subjective reports

So even if I plaster my 'Harman level accomplishments' all over every evaluation of headphones it doesn't invalidate nor validate the subjective part to which ALL reviewers who put a headphone on their heads are guilty as they know what they are listening to and have measured it as well.
Every hard core industry compliant objectivist is going to ignore what I do anyway because it is wrong. My Harman level is not going to change any of this either. I don't mind. There are many folks doing industry standard measurements which I also look at.

My measurements are NOT acc. to any industry standards and partly my subjective findings.
I am open about this and my reasons are publicly explained.
Does that invalidate the, over 40 years in this audio business accumulated knowledge ? A Harman test would be the only thing that 'validates' this to you and others who need validation ?
Besides.. do you really think I don't look at industry measurements and also take that (up to 6kHz) into consideration ?

I'll ask you again and hope you answer my question this time.

When you hear an excellent sounding system would you recognize that or would you insist on AB-ing that with a reference system before you can tell if it really sounds good ?

When you hear crappy speakers or headphones... can you not tell they are crappy or have to drag in a reference system to tell it sounds crap ?

 

[MayaTlab]

I think we can all agree though that the end goal is to be able to correlate these objective measurements with subjective experience, but to do that you need both to be accurate and reliable to a verifiable standard. You have that on the objective side, now I'd say it's time to look more closely at the subjective side. Listener training via How to Listen would help towards that ultimate goal of correlating the subjective with the objective, as Harman's research has shown.

I think that this may be a misunderstanding of both the value that Harman sees in trained listeners and of what “subjective” means in the context of Harman’s research vs. audio reviewers.

From what I understand - and please correct me if I'm wrong, after reading a few of Harman’s articles, trained listeners have a number of characteristics that are useful in listening tests :
- when presented with a scale, they seem to make more extensive use of it, scoring lower than untrained listeners poor loudspeakers / headphones, so they show more discrimination,
- when presented twice with the same loudspeakers / headphones, they tend to score them more consistently than untrained listeners.
Some of that is visible in that article for example :
https://www.aes.org/e-lib/browse.cfm?elib=19436

Coincidentally, some of the articles mention that the trained listeners had their hearing evaluated.
It’s also quite likely that they have more experience with Harman’s listening test format and are part of a roster of "regulars", and, speculating here, more experience listening to “decent speakers in a decent room”.

I think that, for the most part, the entirety of a listener’s subjective experience is contained, in Harman’s research, within the score that person ascribes to loudspeakers / headphones. There is no such thing as “slam”, “detail”, “soundstage”.
When it comes to "correlating the subjective with the objective" in this context, the score is the subjective. Since trained listeners have a number of characteristics that make them useful when it comes to scoring audio equipment, there is value in being trained for that purpose.

On the other hand, what usually falls under the "subjective" area for audio reviewers, is usually described using typical audio jargon such as "detail", "soundstage", "punch", "warm", or if you're What Hifi even more flowery terms such as "poise" and "deft musicality".

Now actually Harman's research also occasionally asked listeners, trained or not, to describe in a few words their experience, beyond merely scoring it, but I don't think that it's ever been the main thrust of the research and an area that's been extensively explored.
Unless I'm mistaken, there is no evidence that listeners trained with Harman's program would spontaneously use the same terminology more often and consistently than untrained listeners to describe the same measurable difference when exposed to it.

That also seems to me more like a research in semantics than in acoustics if I'm honest.

What I would really like to see a lot more of when it comes to headphones evaluations and "correlating the subjective with the objective", however, is extensive in-situ, on head measurements, as I believe that it's a far more potentially fruitful area to explore, particularly when juxtaposing the research that's been done so far on the audibility of peaks / dips and the sort of unforeseen variations one may experience, at least for some headphones, which may occasionally be largely audible.
If anything Rtings' consistency measurements at lower frequencies on five real human subjects should already be enough justification to systematically perform on-head measurements of the actual lower frequencies' response as part of a headphones' review.

Coincidentally, that's also something @Resolve happens to do, to a degree. Take it however you like it, but apparently that's one way he assessed that what he would describe as "punch and slam" is not correlated with on-head FR below a few hundred hertz. That's already a start in trying to "correlate the subjective with the objective", or in that case rather to not correlate it.

If I may, I am far less interested in knowing whatever level a headphones reviewer achieves on Harman's How To Listen than on getting some data on how the actual sample they measured on their test rig and they evaluated actually measured on their head, and comparing it with other headphones.

Too bad that it's not straightforward to do past 1kHz and from my limited "random dumb dude on the internet tries to use microphones in his ears" experience even more so past 7kHz or so.

 

[GaryH]

I got this for you (2014) and that was really easy to pass and means nothing but was fun to do back then.

View attachment 161454

Not sure whether that means 100% correct or just 100% completed attempts. Looks like others got this with the former:

Regardless, as you said it means nothing in terms of verifiable scientific research, as unlike How to Listen's trained listeners I'm not aware of any equivalent public research on Philips' 'Golden Ears' listeners. (In fact internally Philips' tests are reportedly stricter, with a higher 'Platinum Ear' level, so the version of the test released to the public may well have just been a marketing ploy to get audiophiles to feel good about themselves, and in turn Philips, the ones giving them this prestigious 'award' ) If you did in fact achieve Golden Ear level (which would put you in the top ~13% of participants from the numbers on your screenshot, although many probably didn't bother attempting more than the basic level so there could be more), that at least suggests you might do well on How to Listen though. Only one way to know for sure

So even if I plaster my 'Harman level accomplishments' all over every evaluation of headphones it doesn't invalidate nor validate the subjective part

No it doesn't, but as I said above would it make it more probable that your subjective judgements are accurate and reliable, and so increase trust in them. This is not an all-or-nothing affair, it's a continuum.

You avoided my question and did not answer it

I ignored those questions because they're irrelevant to the original discussion, which was about you saying that you can identify the 'best headphones you've ever heard' (describing it with nebulous terms like "effortless" and "disappearing" no less) without an A/B comparison with any of the 'over 100' headphones you've listened to (at least one of which would have been your previous 'best headphone you've heard'), and claiming this ability to discern good/bad sound quality is easier for 'experienced listeners'. I then asked you to provide verifiable evidence of this supposed ability by taking the How to Listen test. You refused.

When you hear an excellent sounding system would you recognize that or would you insist on AB-ing that with a reference system before you can tell if it really sounds good ?

When you hear crappy speakers or headphones... can you not tell they are crappy or have to drag in a reference system to tell it sounds crap ?

What I would do has no relevance to your unsubstantiated claim of your supposed ability, but I'll answer anyway to make you happy. And the answers are yes and yes I would A/B them with a reference system, if I really want to make sure. And I would definitely want to make sure if 1) I was claiming it was 'the best headphone I'd ever heard' out of over 100, not merely 'good' or 'crap', 2) I were a reviewer (who many people listen to no less) whose reviews can have a significant effect on the reputation of a headphone model / company in the eyes of my readers, and especially 3) if I am selling devices that are in any way developed using my subjective judgements of the headphones with and without those devices attached, to ensure I would not be selling unintentional snake oil to my customers.

 

[solderdude]

There is no pleasing you is there ?

And it is good to read you can also hear if something sounds good or not, seems I am not the only one .

Acc. to you I cannot post my opinion untill I prove to you (and to those visiting my website) which has no review of the HE-1, by means of taking the Harman course and revealing that all important level that I am qualified to publicly post that the HE-1 was the best I heard. Only then my opinion is valid.

You live in a strange world.

 

[GaryH]

I think that this may be a misunderstanding of both the value that Harman sees in trained listeners and of what “subjective” means in the context of Harman’s research vs. audio reviewers.

From what I understand - and please correct me if I'm wrong, after reading a few of Harman’s articles, trained listeners have a number of characteristics that are useful in listening tests :
- when presented with a scale, they seem to make more extensive use of it, scoring lower than untrained listeners poor loudspeakers / headphones, so they show more discrimination,
- when presented twice with the same loudspeakers / headphones, they tend to score them more consistently than untrained listeners.
Some of that is visible in that article for example :
https://www.aes.org/e-lib/browse.cfm?elib=19436

Coincidentally, some of the articles mention that the trained listeners had their hearing evaluated.
It’s also quite likely that they have more experience with Harman’s listening test format and are part of a roster of "regulars", and, speculating here, more experience listening to “decent speakers in a decent room”.

I think that, for the most part, the entirety of a listener’s subjective experience is contained, in Harman’s research, within the score that person ascribes to loudspeakers / headphones. There is no such thing as “slam”, “detail”, “soundstage”.
When it comes to "correlating the subjective with the objective" in this context, the score is the subjective. Since trained listeners have a number of characteristics that make them useful when it comes to scoring audio equipment, there is value in being trained for that purpose.

On the other hand, what usually falls under the "subjective" area for audio reviewers, is usually described using typical audio jargon such as "detail", "soundstage", "punch", "warm", or if you're What Hifi even more flowery terms such as "poise" and "deft musicality".

Now actually Harman's research also occasionally asked listeners, trained or not, to describe in a few words their experience, beyond merely scoring it, but I don't think that it's ever been the main thrust of the research and an area that's been extensively explored.
Unless I'm mistaken, there is no evidence that listeners trained with Harman's program would spontaneously use the same terminology more often and consistently than untrained listeners to describe the same measurable difference when exposed to it.

No misunderstanding. I'm aware that the correlation Harman has focused on between subjective and objective has been primarily unidimensional (actual preference rating correlated to predicted preference rating based on measurements). However, in addition to trained listeners demonstrating a "more discriminating and reliable judgment of sound quality than untrained listeners" as Sean Olive says in this blog post, he also says this:

A second reason for training listeners is that they are able to report precisely what they like and dislike about the sound quality using well-defined, meaningful terms.

This sounds like exactly the kind of thing reviewers would benefit from (and in turn their readers).

If I may, I am far less interested in knowing whatever level a headphones reviewer achieves on Harman's How To Listen than on getting some data on how the actual sample they measured on their test rig and they evaluated actually measured on their head, and comparing it with other headphones.

Why not both? In-ear mic measurements do have obvious issues though (which I've been over with you before), but like you I do appreciate Resolve's and Rtings' extra in-ear data for bass response across different listeners. What's really needed is a completely non-invasive way to measure the PRTF though - something like what the AKG N90Qs 'TruNote' auto-calibration technology does, playing an acoustic signal into the ear, listening to the reflections, and calculating that ear's particular resonances etc. Just throwing this out there, but maybe measuring the difference in response between the N90Qs calibrated on-head and on-HATS could provide some kind of 'PRTF error response' that could be added to measurements of other headphones.

 

[MayaTlab]

What's really needed is a completely non-invasive way to measure the PRTF though - something like what the AKG N90Qs 'TruNote' auto-calibration technology does, playing an acoustic signal into the ear, listening to the reflections, and calculating that ear's particular resonances etc.

Inasmuch as I agree, I'm not quite certain that we're there yet or any time soon. Have you tried the N90Q ?

In-ear measurements will have to make do in the meantime, and while they have their own limitations, as I've already explained to you, there are ways to cut through the noise and extract actionable results, even above 1kHz.

Sean Olive says in this blog post, he also says this:

That's good to know, but I am not aware of an actual article that evaluated that. If you're aware of it I'll take it.