HBK Headphone Measurement Talks from Head-Fi and Sean Olive

[amirm]

The only downside (or accuracy variation) that is involved with that statement is that not everyone hears Harman Target Headphones with the same tonality, due to people's different HRTF's and also some specific ear to headphone interactions that are not readily predictable.....

You would have the same issue in a live setting as well. We can't try to get better than that. This is why I mentioned once there, you can use tone control, etc. to deal with whatever is remaining than fighting what the true sound of some music is.

 

[DualTriode]

The only downside (or accuracy variation) that is involved with that statement is that not everyone hears Harman Target Headphones with the same tonality, due to people's different HRTF's and also some specific ear to headphone interactions that are not readily predictable.....but I certainly agree that it's a good standard to get behind, even if it's not true complete transparency, but in terms of headphones it is certainly the best target to get behind to help lesson "Circle of Confusion" issues, if we're assuming a lot of music is ultimately created & mastered (if correct terminology) on headphones as opposed to speakers. (But I should add that it also helps to lesson "Circle of Confusion" even if the music has been created using speakers rather than headphones, as we know it's designed largely to emulate Flat Anechoic Speakers in a good listening room).

Hello,

If I see anything that includes "circle of confusion" that is not optics or photography I stop reading and ignore.

"circle of confusion" has nothing to do with audio. "circle of confusion" is a term stolen from optics or photography.

Google it!

Thanks DT

 

[pozz]

Hello,

If I see anything that includes "circle of confusion" that is not optics or photography I stop reading and ignore.

"circle of confusion" has nothing to do with audio. "circle of confusion" is a term stolen from optics or photography.

Google it!

Thanks DT

I thought no one else had noticed. Same with "bookshelves". No idea what that has to do audio.

 

[joentell]

I, and I suspect most members here, are more interested in the quantifiable, verifiable science of, rather than the art of listening

Great, and on that quantifiable note, what was the highest level you could pass?

Lol.

Are we going to make this a competition now? What do I get if I win?

 

[GaryH]

Lol.

Are we going to make this a competition now? What do I get if I win?

Well it's no more of a competition than asking a reviewer what the capabilities of their measuring equipment are before looking at their measurements, except the equipment in this case is your ears. If you 'win' you get more trust in your subjective judgements, just as the owner of a GRAS 45CA, Klippel NFS, or Audio Precision APx555 gets more trust in their measurements (as long as they know how to use them properly). And of course just for taking part, being the (so far) only reviewer to post their results you'll get something expensive measuring equipment can't buy - respect

 

[joentell]

Well it's no more of a competition than asking a reviewer what the capabilities of their measuring equipment are before looking at their measurements, except the equipment in this case is your ears. If you 'win' you get more trust in your subjective judgements, just as the owner of a GRAS 45CA, Klippel NFS, or Audio Precision APx555 gets more trust in their measurements (as long as they know how to use them properly). And of course just for taking part, being the (so far) only reviewer to post their results you'll get something expensive measuring equipment can't buy - respect

"...Respect" I think you took that line from Vin Diesel in Fast & Furious.

 

[GaryH]

Fwiw, here is my attempt at explaining why I think over-ear headphones (and speakers) are generally better than IEMs for these kinds of generic frequency response tests...

Master Complaint Thread About Headphone Measurements

One thing that also needs to be taken into consideration on this is the users' ability to "self-correct" the sound of the different headphones on their own head, if they sound off, by shifting the headphones around a bit, or adjusting the seal, headband, and so forth. Which is something that's...

www.audiosciencereview.com

These are probably the most salient parts of the above post on this subject...





Best to read the entire post though to understand the context of the above statements.

Again, not supported by the data. Take a look at the results of this IEM study and this over-ear headphone study from Sean's presentation. The average preference rating given to the Harman target in the former was 68. The average given to Harman target in the latter was 67. If IEMs bypassing individual PRTFs was as much of an issue as you claim, the IEM Harman target (which lacks any even generalized fine-grain PRTF features) would not have scored equally as well (in fact marginally, although statistically insignificantly, better) as the over-ear Harman target (the latter of course convolves with the listener's PRTF when from a worn over-ear headphone).

 

[Robbo99999]

You would have the same issue in a live setting as well. We can't try to get better than that. This is why I mentioned once there, you can use tone control, etc. to deal with whatever is remaining than fighting what the true sound of some music is.

I agree, we can't get better than that at the moment. As a quick related aside, if anyone wants to use Tone Control and they have to use parametric filters, then they can use these three High Shelf Filters together (applying same Gain or Reduction (dB) to each of the filters) which will effectively be an almost perfect linear tone control tilt from 20-20000Hz: three High Shelf Filters with Q0.5 at 63Hz / 632Hz / 6324Hz.

Hello,

If I see anything that includes "circle of confusion" that is not optics or photography I stop reading and ignore.

"circle of confusion" has nothing to do with audio. "circle of confusion" is a term stolen from optics or photography.

Google it!

Thanks DT

It's a term that I think was coined by the Harman folks in relation to audio - it does have a meaning in audio circles, and it's undeniable that if music is created on bright speakers, then there's increased likelihood that it will sound dull when played back on flat reference speakers or even more dull if your own playback system has too much overall downslope in addition (as an example), so the phenomenon does exist - it's about trying to match tonality of the playback system to that of the creator of the music (within broad realms).

 

[preload]

Hello,

If I see anything that includes "circle of confusion" that is not optics or photography I stop reading and ignore.

"circle of confusion" has nothing to do with audio. "circle of confusion" is a term stolen from optics or photography.

Google it!

Thanks DT

Try changing your search terms to "circle of confusion audio." You would think people know how to use Google by now.

 

[preload]

I agree, we can't get better than that at the moment. As a quick related aside, if anyone wants to use Tone Control and they have to use parametric filters, then they can use these three High Shelf Filters together (applying same Gain or Reduction (dB) to each of the filters) which will effectively be an almost perfect linear tone control tilt from 20-20000Hz: three High Shelf Filters at 63Hz / 632Hz / 6324Hz.

What Q on the shelf filters? Wondering if there's a "standard." I remember in one of the Harman papers they used "2nd order" filters with a frequency of 2.5khz as their treble tone control, but I don't know how to translate that to a shelf filter + q setting on my PEQ. Perhaps someone knows.

 

[preload]

I think that this may be a misunderstanding of both the value that Harman sees in trained listeners and of what “subjective” means in the context of Harman’s research vs. audio reviewers.

From what I understand - and please correct me if I'm wrong, after reading a few of Harman’s articles, trained listeners have a number of characteristics that are useful in listening tests :
- when presented with a scale, they seem to make more extensive use of it, scoring lower than untrained listeners poor loudspeakers / headphones, so they show more discrimination,
- when presented twice with the same loudspeakers / headphones, they tend to score them more consistently than untrained listeners.
Some of that is visible in that article for example :
https://www.aes.org/e-lib/browse.cfm?elib=19436

Yep, that is what I recall as well.

 

[preload]

Well your opinion on IEMs is contrary to Harman's research, which found a 0.91 correlation between actual and predicted ratings given by listeners in blind tests, compared to 0.86 for both over-ear headphones and speakers. Anyway, as I said in my previous post which you've overlooked, Harman observed the same findings with over-ear headphones (238 listeners) concerning the improvement in reliability and discrimination of trained listeners compared to untrained. And as @pozz says, this all builds on Tools and Olive's similar findings of trained and untrained listeners with speakers. This is well-estsblished stuff by now.

I was just about to respond to @ADU by referencing the same studies and the same argument.

 

[ifloatoveryou]

If I had known doing this program would validate my subjective personal opinions and findings would instantly take away all doubt for those interested in audio I might even have gone through the trouble.
You seem to be the only one asking to do this (in a demanding way) so you can take what I, and now also industry insiders and reviewers as well, write more serious.

Well, the research shows trained listeners are better able to discriminate better between speakers than reviewers, and for untrained listeners in general for headphones. (There may be a study with reviewers evaluating headphones, but off the top of my head I do not recall one). Asking reviewers to see if they meet the qualifiaction of a trained listener, and if not then to train their ear, I think is reasonable. I am surprised Gary is the first to bring this up, I'll be the second to request this of reviewers. I'll tag @Resolve since he expressed interest in this and what I talk about below may be relevant to him as well.

There also seems to be some confusion regarding what the statistics of trained vs untrained listener rating headphones mean. Passing the program does not mean your subjective impressions are valid and no doubt can be cast upon the - this is an incorrect interpretation of research.

The differences between trained and untrained listeners describe everyone studied, and may not accurately describe an individual. Some untrained listeners will be be equally consistent in blind subjective assessments of headphones or speakers, just on average a random untrained listener is worse than a random trained listener. You may already meet the criteria for being a trained listener, and even if you don't, your subjective opinions if blind tested in some study may still be consistent, similar to that of a trained listener.

Just because someone is a trained listener does not mean their subjective impressions will be reliable. There will always be bias - no subjective impressions will be 100% valid and reliable. The science shows subjective impressions are more reliable when a listener is trained. This does not imply that every subjective impression by a trained listener reliable and valid, just that on average they are more reliable than untrained listeners. Nor does this imply that an impression by an untrained listener is necessarily invalid.

Of course, this is not specific to you. If you do not meet the criteria of trained listener, it does not mean every personal opinion and subjective impression on your website would suddenly become invalid. The science does however show that subjective assessment is more reliable if you are a trained listener. Seeing as this is a science oriented forum, and subjective assessments are a necessary part of reviews, I think it is good to ask reviewers to check if they meet the criteria for being a trained listener, and if they do not, it is good to ask them to train their ear as research participants have done.

 

[solderdude]

The confusion about the Harman research seems to be only with autodidact ADU.

Regarding me and what I do. The same applies to every other reviewer.
Even if I took the course and passed it with highest honors only a handful of ASR members would care about the test results.
99.9 % of the readers would not care nor know what it means.
Regardless, the nerdy ones will still insist my measuring method is completely flawed and I am biased. Which I, like everyone else, is even Harman program trained people.
Harman trained people are just a little better at detecting certain aspects than random folks with or without an interest in audio reproduction.

So whatever I do or don't do doesn't change anything for readers. It would only mean some nerdy guys on the internet may question my hearing a bit less.. well actually they won't they would still question everything I do and say as they have already made up their minds.

Also, even when I reach a high level at the Harman test people (well at least a few) it won't change my mind about the HE-1 I heard and compared to, to me, well known reference headphones which seems to have been the reason to drag me into the discussion.

 

[DualTriode]

Try changing your search terms to "circle of confusion audio." You would think people know how to use Google by now.

The Circle of Confusion concept has belonged to optics and photography since the 1860's. Floyd Toole ripped it off in his book in 2008.

Thanks DT

If you see circle of confusion in audio text stop reading and ignore.

 

[ifloatoveryou]

So whatever I do or don't do doesn't change anything for readers. It would only mean some nerdy guys on the internet may question my hearing a bit less.. well actually they won't they would still question everything I do and say.

Also, even when I reach a high level at the Harman test people (well at least a few) it won't change my mind about the HE-1 I heard and compared to, to me, well known headphones.

I agree with what you're saying, most readers aren't so interested in Harman or Toole's research and becoming a trained listener would not be some revelation that changes everything. I had forgotten this discussion stemmed from your evaluation of the HE-1 - I am not interested in whether or not it is your favorite headphone. I am interested in the reliability of audiophile reviews.

The research strongly suggests training your ears will improve your ability to evaluate what you are hearing. Seeing if your ears are trained, and training your ears if not, would likely make anyone's reviews better. It is true that only a handful of nerdy guys on the internet care about this.

One last thing: you provide a plethora of useful information and acknowledge the limitations of your approach - I am sure most audio nerds on the internet greatly appreciate what you do!

 

[Scgorg]

The Circle of Confusion concept has belonged to optics and photography since the 1860's. Floyd Toole ripped it off in his book in 2008.

Thanks DT

If you see circle of confusion in audio text stop reading and ignore.

So terms and/or words cannot have more than one meaning? Toole should have made a completely new term named "the cycle of uncertainty" instead?

Do you realize how ridiculous you sound?

 

[solderdude]

I am interested in the reliability of audiophile reviews.

I am too, that's why I do it my way which differs from those of many others.
I found most reviews of headphones,that I know pretty well, to be all over the place and overly positive filled with advertising material. These are the usual suspects most of the case. Its the main reason you can only find me on my own website and here.
There is so many commercialism involved and maintaining good relationships with manufacturers (that because of this will send them samples) that I don't trust any of it unless they find the same things I do.

I am not sponsored, I rarely receive samples (because I do not hype what manufacturers want hyped) and I do have references to compare to.
My measurement method differs and have my reasons for that. Overlay them with those from Amir and they are very close.
Besides I also look for other measurements and reviews (when available) as well and am quite well trained in audio and listening.
That's why I do what I do and how I do this. People are free to read what I write and like or dismiss it for whatever reasons they have.

One last thing: you provide a plethora of useful information and acknowledge the limitations of your approach - I am sure most audio nerds on the internet greatly appreciate what you do!

I get some positive feedback now and then but not by nerds.
I did get one hate mail once accusing me of snake-oil business (the passive filters which is the only thing I sell simply because I am the only one designing and making them).
Most nerds will not visit my site though as they know how I measure and don't agree.

 

[MayaTlab]

Asking reviewers to see if they meet the qualifiaction of a trained listener, and if not then to train their ear, I think is reasonable. I am surprised Gary is the first to bring this up, I'll be the second to request this of reviewers. I'll tag @Resolve since he expressed interest in this and what I talk about below may be relevant to him as well.

I would seriously encourage people who ask this to actually do it themselves (which I'm pretty certain GaryH never bothered to do). Perhaps this might help them put the importance of such test into perspective. Let it be repeated here : the only quantifiable advantage so far observed (or at least published) with trained listeners is that they are more discriminatory and more consistent when evaluating speakers and headphones within the format of Harman's research. Very useful in that context, no doubts about it.

As an exercise it's an interesting thing to do, but probably shouldn't be seen as the be-all, end all thing GaryH thinks it is. You can cheat your way through some of tests BTW (the noise one for example).
And there are other websites providing that service in a fairly similar format.

Besides, someone could be near-deaf anyway and still provide interesting observations when it comes to characterising a pair of headphones' behaviour with measurements alone. As long as that person is curious and pretty good at coming with experimental ideas to provide interesting data, I'm all, well, ears.
I appreciate the plurality of methodologies we're seeing when it comes to evaluating headphones (well as long as it's not What Hifi stuff) and I would absolutely pay attention to a review that provides interesting data even if the reviewer in question couldn't - oh the horror - qualify as a trained listener in How To Listen.

I've seen enough questionable results, even on "industry standard" test equipment, that should have been caught by listening tests unless the operators were in a rush, even though some of the operators in question probably tried their hand at How To Listen, that I'm taking this with a pinch of salt.

 

[DualTriode]

So terms and/or words cannot have more than one meaning? Toole should have made a completely new term named "the cycle of uncertainty" instead?

Do you realize how ridiculous you sound?

Floyd Toole must have thought that the words were his own. There were no references, credits or footnotes given.

I am sure that there is a photo text on his shelf that he forgot reading 40 years ago.