Headphones preference score ranking chart based on Harman target curve

[thewas]

https://github.com/jaakkopasanen/AutoEq/blob/master/results/RANKING.md

Headphones ranked by Harman headphone listener preference scores.

Tables include the preference score (Score), standard deviation of the error (STD), slope of the logarithimc regression fit of the error (Slope) for both headphone types and average of the absolute error (Average) for in-ear headphones. STD tells how much the headphone deviates from neutral and slope tells if the headphone is warm (< 0) or bright (> 0).

Keep in mind that these numbers are calculated with deviations from Harman targets. The linked results use different levels of bass boost so the slope numbers here won't match the error curves you see in the linked results.

Over-ear table includes headphones measured by oratory1990. In-ear table includes headphones measured by oratory1990 and Crinacale. Measurements from other databases are not included because they are not compatible with measurements, targets and preference scoring developed by Sean Olive et al.

 

[solderdude]

It's a nice list.
Doesn't include actual listening it seems.
It is merely ranked by deviations from a specific standard.
Closed and open mixed and comfort not taken into account.
I certainly would not buy any headphone based on this ranking for a plethora of reasons.

But.. it is a list ranking the HE-1 just slightly above the cheap K371.
Those that heard both probably won't agree with the ranking. There is a substantial difference between them. Not in tonal balance perhaps but in overall sound quality and other properties these do not even come close.

What would be the benefit of this preference rating ? preference for tonal balance only.

 

[thewas]

What would be the benefit of this preference rating ? preference for tonal balance only.

I see it as an alternative to the Olive loudspeaker score calculated in this forums for all measured examples which is also based just on tonality and thus inherently limited.
For advanced users like you or many others here of course more other aspects matter, but for a newbie who just wants to find a decent sounding headphone without EQing or modding it can be a good starting point (which I would wish to have had when I started the hobby), as comfort and other issues can be googled from other reviews and even tested at a local store.

 

[Fluffy]

What a bunch of nonsense… I hope no one takes this ranking seriously.

 

[solderdude]

This means a beginner should buy the $400 PSB M4U wireless NC headphone with the remarks in this review that it lacks details and dynamics, has a poor battery life and so-so noise cancelling.

I am now intrigued by the almost impossible to find A800 and am a bit afraid of the +10dB bass boost compared to the HD660S though.

Orpheus is out of every beginners budget but the almost similar ranked HD600 is not. Well over 10dB bass difference with the slightly bassier HD660S doesn't inspire much confidence that the HD600 has similar tonal balance. It find the HD600 bass-light, the HD660 bass decent.

The AKG K371 is nice for a beginner and even music afficionado on a wider budget and is cheap. If it were a great headphone, almost as great as the HE1 I would have kept it.
The very nice soundig Jade-II is ranked below the K371 ?
In fact most of the headphones are ranked below K371. Based on this it would be pointless to buy anything other than the A800 instead of the K371.
It's not something I would say to people looking to get top sound quality in every budget.

Besides, not everyone is looking for close to Harman (HD600 isn't for sure yet is ranked almost as high as the HE1).
Bass heads will find the K371 bass-shy.

It is a ranked list... not similar to speaker rankings IMO and not even to the not so telling SINAD list. But these are ranked lists too.

I think Goldenears did better with their 'sliders' and R'tings at least makes an effort to rate (on numbers only) for different use purposes.
Not everyone will have the same score-weighing thing either.

 

[thewas]

This means a beginner should buy the $400 PSB M4U wireless NC headphone with the remarks in this review that it lacks details and dynamics, has a poor battery life and so-so noise cancelling.

Jeez, everybody who reads the description immediately understands that this ranking is just an objective based approach only about the tonal characteristics which as we know play the biggest role in the sound quality perception, like Floyd Toole says if tonality is wrong then nothing else matters. Nobody will interprete properties like battery life, noise canceling etc. in such a list...

The very nice soundig Jade-II is ranked below the K371 ?

Of course at very high standards such an purely numerical approach won't be sufficient for individual nuances, but this is not what such indexes are for but to give a newbie a good startnig point for his own listening tests and avoid buying something which is tonality wise a desaster. I don't know how it is for you but I personally wouldn't want to hear any of the headphones of the list with a score below 50 without equalising it.

It is a ranked list... not similar to speaker rankings IMO

What exactly is its difference with the in this forum so much discussed Harman loudspeaker score?? They are both just based on the tonality in comparison to some predefined targets and thus limited but still will say with a quite high computable probability how many % of the people would prefer headphone A over B tonality wise. Like every statistic it won't work for everybody though, it is said that there exist even people who actually like the highs of a Ultrasone Edition 10

 

[Fluffy]

like Floyd Toole says if tonality is wrong then nothing else matters

I agree and disagree. Tonality can be fixed, to a point, and measuring to find potential places that needs fixing is beneficial. On the other hand, if tonality is perfect a headphone can still sound like crap if it has bad distortions, resonances, or some other defect that doesn't directly affect FR (like problematic sensitivity or impedance curve).

 

[thewas]

I agree and disagree. Tonality can be fixed, to a point, and measuring to find potential places that needs fixing is beneficial. On the other hand, if tonality is perfect a headphone can still sound like crap if it has bad distortions, resonances, or some other defect that doesn't directly affect FR (like problematic sensitivity or impedance curve).

As I had written already in #3 nobody here is talking about EQing headphones, but for 99% of the people worldwide who don't. And yes, with EQ option this list would completely change, someone could try to compute a reasonable metric for it like for example Sonarworks does.
And of course there can be exceptions like very high distortions levels although they are luckily rather rare for reasonably quality headphones. As said this is just an automated metric on one (but most important) characteristic and nothing else and it also limited but just writing

What a bunch of nonsense… I hope no one takes this ranking seriously.

is just pure polemic unless you make a better one.

 

[solderdude]

Jeez, everybody who reads the description immediately understands that this ranking is just an objective based approach only about the tonal characteristics which as we know play the biggest role in the sound quality perception, like Floyd Toole says if tonality is wrong then nothing else matters. Nobody will interprete properties like battery life, noise canceling etc. in such a list...

If that is true then why are the A800 (overly bassy/warm) and HD600 (lean/bright) so closely matched in rating. It does not make sense. It's a nonsensical list.

If one wants to use this as buying advise then they should know the list is rather pointless. A newby should not buy on technical ratings but go to a shop where they demo stuff and see what kind of sound they want. Some want muddy bass monsters others want lean and bright.
This absolutely is not shown in the list. That makes the list nonsensical. Besides... try to obtain the A800 for instance.

I don't believe this list is truly helpfull guiding anyone to a 'good sounding' headphone. It's a list with some kind of ranking but of little to no practical usage. Just like the top left of the SINAD range ranking list. It is a list which ranks 2 combined properties but have no bearing on all the other properties that are far more inportant when selecting a DAC. Simply saying to a newby, just select the DAC most towards the top of the ranking list may lead to purchasing something they do not specifically need.

I have heard/measured/compared quite a few headphones and do not rank them because one cannot rank them. One can make a list based on some metrics but that won't say anything about whether or not it is useful to a newby nor initiate.

I think Rtings does a better scoring and even that one is found 'stupid' by a lot of folks. I know these are just opinions but at least they rank on more than just an aspect. I think the speaker ratings are far better than this list.

Do you actually rank the headphones you own (without EQ) in a similar way as the discussed list ? I am sure you don't. You know too much about headphones and their (not-EQ'ed) sound.

 

[lashto]

@solderdude @Fluffy
I think you guys might be a bit too critical here. You make very good points but of course this is not a perfect list and of course it is not the only thing to check when buying a HP.
It is just one useful list.

I'd be curious to see a similar list with HPs ordered by how well they respond to EQ. Not sure if AutoEQ data can be used for that, but an 'EQ-friendliness' list might be even more useful.

 

[MZKM]

Closed and open mixed and comfort not taken into account.

Comfort is indeed very important. The weight of the headphone, the clamping force, the pad feel, and the pad ventilation are all important.Also, as someone who wears glasses, the amount of headphones that can’t fit my glasses is too dame high (ear cups/pads are too narrow, pushing my glasses forward).

 

[MZKM]

RTINGS is the best resouce for headphone measurements in my opinion; but their scoring and weightings are heavily debated, so go off the measurement data.

 

[thewas]

If that is true then why are the A800 (overly bassy/warm) and HD600 (lean/bright) so closely matched in rating. It does not make sense. It's a nonsensical list.

Its just an objective metric based one only one aspect and of course limited but I wouldn't at all say nonsense as according to the Harman experiments it has a significant correlation to the listeners preferences.

A newby should not buy on technical ratings but go to a shop where they demo stuff and see what kind of sound they want. Some want muddy bass monsters others want lean and bright.
This absolutely is not shown in the list.

Going to a shop is in my experience the worst a newbie can do as he quickly gets overwhelmed and saturated by the sound impressions and often leads in buying a strongly v-shaped headphone that tires him on the long run. Also there has been quite some research done on many different listeners groups (age, country, experience) and the only significant statistical difference in tonal preferences are that newbies tend to prefer a couple of more dB at both ends of the audible range.

 

[Fluffy]

RTINGS is the best resouce for headphone measurements in my opinion; but their scoring and weightings are heavily debated, so go off the measurement data.

And preferably the raw measurement, because the compensated charts are very confusing.

 

[solderdude]

@solderdude @Fluffy
I think you guys might be a bit too critical here. You make very good points but of course this is not a perfect list and of course it is not the only thing to check when buying a HP.
It is just one useful list.

I'd be curious to see a similar list with HPs ordered by how well they respond to EQ. Not sure if AutoEQ data can be used for that, but an 'EQ-friendliness' list might be even more useful.

The problem is it would only EQ correctly on the specific test rig it is measured on and measure incorrectly on any other brand/model test rig.

It is a list for sure. I seriously question the usefulness when a newby wants a headphone with unobtanium and unaffordable right next to cheap.

And yes, indeed not all headphones EQ equally well. Some that have dips and peaks that are EQ'ed out still sound poor but in a different way.
My opinion is: the less EQ is needed the better the EQ'ed result usually is.

You can't EQ a HD201 into a HE-6. There are limits and aspects besides tonality. All aspects need to be good for good sound. Not only tonality.
The A800 tonality differs immensly from the HD600 yet ranked closely. The list cannot possibly be useful for tonal accuracy.

I amy be too serious about this though... The majority of headphone buyers just buys one on looks or reviews telling how great they are and the best you can get.

 

[solderdude]

Its just an objective metric based one only one aspect and of course limited but I wouldn't at all say nonsense as according to the Harman experiments it has a significant correlation to the listeners preferences.

It looks as though this list is not based on tonality aspects (looking closer at it as it shows 'tilt' as well but rather ranks at the least amount of peaks and dips measured on a specific coupler.
This explains ther ranking.
That would mean that while tonally different the ranked headphones need less 'aggressive' EQ to sound 'linear' but yet tonally different.

I don't think one should base buying decisions on this chart alone. Might as well simply advise people to buy a HD600 for open and not portable duties and K371 for closed (and portable) duties.
The PSB M4U 8 perhaps for those wanting wireless while not wanting excellent noise cancelling.

I could well go along with these recommendations but think there are better alternatives.


You didn't answer my question though...

Do you actually rank the headphones you own (without EQ) in a similar way as the discussed list ? I am sure you don't. You know too much about headphones and their (not-EQ'ed) sound.

 

[PierreV]

Looking at some of the things we have in the house (I have a few others including a few of the "sport" in-ears listed)

Samsung Galaxy Buds 81
Westone UM Pro 50 9
Apple AirPods -26
Apple EarPods -51



I have no words...

 

[solderdude]

I certainly don't rank the ones I know in a similar way to the published list. Maybe that's why I am of the opinion it is a worthless list.

That said the HE-1 is on the top of my list and never heard the A800 nor PSB.

 

[lashto]

The problem is it would only EQ correctly on the specific test rig it is measured on and measure incorrectly on any other brand/model test rig.

It is a list for sure. I seriously question the usefulness when a newby wants a headphone with unobtanium and unaffordable right next to cheap.

And yes, indeed not all headphones EQ equally well. Some that have dips and peaks that are EQ'ed out still sound poor but in a different way.
My opinion is: the less EQ is needed the better the EQ'ed result usually is.

You can't EQ a HD201 into a HE-6. There are limits and aspects besides tonality. All aspects need to be good for good sound. Not only tonality.
The A800 tonality differs immensly from the HD600 yet ranked closely. The list cannot possibly be useful for tonal accuracy.

I amy be too serious about this though... The majority of headphone buyers just buys one on looks or reviews telling how great they are and the best you can get.

let's not get too serious . You make a very good point in many places that HP measurements & choices are not an easy task, just felt that you were a bit too critical here.

Anyway, IMO a HP with excellent EQ response would be golden. Measurements and target curves are far from perfect but they continuously evolve and you'd be able to 'update' that kind of HP forever.

 

[thewas]

It looks as though this list is not based on tonality aspects (looking closer at it as it shows 'tilt' as well but rather ranks at the least amount of peaks and dips measured on a specific coupler.

Because it is based on the Harman research which has found that local peaks and dips are more dentrimental to the perceived sound quality as different smooth tilts.

I don't think one should base buying decisions on this chart alone.

Who ever said so here??

You didn't answer my question though...
Do you actually rank the headphones you own (without EQ) in a similar way as the discussed list ? I am sure you don't. You know too much about headphones and their (not-EQ'ed) sound.

Sorry, didn't see it, I personally don't see very big variations to the tonality of the few headphones I have but I haven't tested as many as you as I am a a loudspeaker listener. Of course personal taste and indidual HRTF play also a significant role and that Harman target is also questionable but its at least one objective approach, hopefully more and better ones will come in the future.
By the way you didn't answer my question either.
What exactly is its difference with the in this forum so much discussed Harman loudspeaker score??

And yes, indeed not all headphones EQ equally well. Some that have dips and peaks that are EQ'ed out still sound poor but in a different way.
My opinion is: the less EQ is needed the better the EQ'ed result usually is.

Which actually underlines the importance of this metric

You can't EQ a HD201 into a HE-6. There are limits and aspects besides tonality.

Of course, although they usually aren't as big as most of us think when we agree to the results of such research http://seanolive.blogspot.com/2016/04/a-virtual-headphone-listening-test.html

All aspects need to be good for good sound. Not only tonality.

But tonality is by far the most important and as Toole says, if that is wrong than nothing else matters (oops, the tonality of the Metallic song isn't great either )