• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

headphones ratings relative to peers?

guzkiy

Member
Joined
Jan 25, 2022
Messages
13
Likes
6
Hi All!
I am new here but I am fascinated by the lack of bs on this forum (as oppose to some other audio forums - names do not matter).
The measurement system makes total sense to me except one thing - there is no subjective rating of the headphones relative to the peers in the same price range.

Say Amir has 'recommended' label on Sony MDR-ZX110 that costs 9$ and the same 'recommended' on Dan Clark's Stealth that goes for 4K$. I am not in a position to spend 4K$ on Stealth but that does not mean I should go for Sony MDR-ZX110 either. Unless I am missing something, there is no measure how cheap/reach a certain device is for it's price category. That would help with the decision making like 'I have XXX$ cash to spend - what are my choices?'.

Looking at wine rating (Wine Enthusiast)... They are normally given within the price range. There is *almost* no doubt '20XX Château Margaux' 350$ tastes nicer than unknown 'Argentina Red Blend' for 15$, but the latter may have 92 points rating (>90 is good) within the price category and the former may have lesser rating just because there are many nice wines in that price range.
Is there something similar with headphones?
 

_thelaughingman

Major Contributor
Forum Donor
Joined
Jan 1, 2020
Messages
1,324
Likes
1,943
I think you'll have a hard time finding a subjective rating on headphones due to the complexity of how other's perceive sound from headphones. But, there will be many here that can help you narrow down a pair of headphones based on your preference of music or tonality.
 
OP
G

guzkiy

Member
Joined
Jan 25, 2022
Messages
13
Likes
6
Tastes differ in headphones for sure. But the same is true for wine. Still, there is a system for wine (far from perfect of course).
 

_thelaughingman

Major Contributor
Forum Donor
Joined
Jan 1, 2020
Messages
1,324
Likes
1,943
Tastes differ in headphones for sure. But the same is true for wine. Still, there is a system for wine (far from perfect of course).
I agree, there is a lack of standardized rating system on reviewed headphones. I can recommend, headphone rankings by Crinacle who has a comprehensive list of headphones that he has measured and attached a subjective review to each based on tonality and couple other factors. Hopefully this helps.
 
  • Like
Reactions: ADU

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,386
Location
Seattle Area
I do rank headphones on 1 to 4 "panther" scales but relative to each other, they are less usable. I wish there was a simple answer but there is not.
 

ADU

Major Contributor
Joined
Jul 29, 2021
Messages
1,584
Likes
1,086
I agree, there is a lack of standardized rating system on reviewed headphones. I can recommend, headphone rankings by Crinacle who has a comprehensive list of headphones that he has measured and attached a subjective review to each based on tonality and couple other factors. Hopefully this helps.

Crinacle's list is an interesting one. I don't think his criteria is totally objective though, as you've said. And he doesn't really separate things out by price. There are some very expensive headphones near the top of his list. But I think that's because he feels they do a better job of checking off more of the boxes on sound quality (in his opinion). And there are also some comparatively inexpensive headphones that rank much higher on his list than many of the more expensive models.

He shows the prices of all the headphones though. So you can sort of parse things out in a similar price bracket that way... if that's what you want.

Tyll Hertsens would do something a little more like what the OP is asking, both in his reviews and also in his headphone "Wall of Fame" for Inner Fidelity. Tyll got out of the headphone reviewing and measuring business a number of years ago though. So his lists aren't really up to date anymore. (Most of his reviews can still be found here, and also here though, for those who wish to check them out.)

There are other popular review sites that will also give their top picks in different categories and price brackets. How reliable these others are though, I wouldn't know.
 
Last edited:

ADU

Major Contributor
Joined
Jul 29, 2021
Messages
1,584
Likes
1,086
Resolve on another popular headphone review site will also compare the headphones he's reviewing to other similar models, often in a similar price range.

Since he primarily reviews the higher cost mid and hi-fi head and earphones, his reviews are geared a bit more towards the audiophiles. So I don't always agree with him on everything. He has come a long way though from where he began. And I'm always somewhat curious to hear his opinions, since he uses a combination of both measurements and his subjective impressions. I believe he also uses a GRAS system which is similar to the one used by ASR for headphone measurements.
 
Last edited:

Earfonia

Active Member
Joined
Feb 21, 2019
Messages
278
Likes
398
Location
Singapore
Hi All!
I am new here but I am fascinated by the lack of bs on this forum (as oppose to some other audio forums - names do not matter).
The measurement system makes total sense to me except one thing - there is no subjective rating of the headphones relative to the peers in the same price range.

Say Amir has 'recommended' label on Sony MDR-ZX110 that costs 9$ and the same 'recommended' on Dan Clark's Stealth that goes for 4K$. I am not in a position to spend 4K$ on Stealth but that does not mean I should go for Sony MDR-ZX110 either. Unless I am missing something, there is no measure how cheap/reach a certain device is for it's price category. That would help with the decision making like 'I have XXX$ cash to spend - what are my choices?'.

Looking at wine rating (Wine Enthusiast)... They are normally given within the price range. There is *almost* no doubt '20XX Château Margaux' 350$ tastes nicer than unknown 'Argentina Red Blend' for 15$, but the latter may have 92 points rating (>90 is good) within the price category and the former may have lesser rating just because there are many nice wines in that price range.
Is there something similar with headphones?

I did headphone reviews in the past in HF and compare the headphone under review with other headphones. But there are few problems:

1. There are probably thousands of headphones models in the market. Let say I compare headphone A to headphone B and C. The chance is very rare that most of the readers have headphone B and C so the comparisons are useful for them. Most of the time the readers have no idea of all the headphones in the comparisons and the comparisons are not useful for them.

2. It takes days or weeks to do objective comparisons with various genres of music. It makes reviewing process a lot longer and time is always the limiting factor for all of us.

3. Price tag is not an indicator of sound quality. We all know that. So it is not always useful to compare headphone at similar price brackets. Sometime we can get better one at lower price bracket if we trust our ears and measurement, instead of our eyes looking at the price tags. Don't buy audio gears by price bracket, set your requirement and get the cheapest possible equipment that still meet your requirement.

4. We all should learn to read measurement results and estimate the sound quality. That's exactly what measurements are for.

I'm not against comparisons, and I always try to do comparisons in my review when time permit. But as mentioned above, comparisons may not always be useful.

My 2 cents ;)
 

ADU

Major Contributor
Joined
Jul 29, 2021
Messages
1,584
Likes
1,086
Unless you have a personal fortune tucked away somewhere, it can also be difficult to keep some of the more expensive and harder to acquire HPs around long enough to use for direct comparisons with other HPs that you're reviewing. I believe Resolve has had some issues with this from time to time.

That was the purpose of Tyll's "wall of fame" though. And somehow he was able to convince the manufacturers to let him keep their headphones on the wall, at least for awhile, so he could do some other comparisons with them. Having a headphone on the wall became sort of like a seal or badge of approval though, that I think alot of mfrs appreciated.
 

Thomas_A

Major Contributor
Forum Donor
Joined
Jun 20, 2019
Messages
3,422
Likes
2,407
Location
Sweden
If there is data file for the reference (Harman) I could try making SD calculation for residuals ranging 80 Hz-8 kHz, alternative 40 Hz-16 kHz as a score. Or just correlation coefficient.
 
Last edited:

jae

Major Contributor
Joined
Dec 2, 2019
Messages
1,208
Likes
1,508
Frequency response and low distortion are the biggest predictors in perceived sound quality. Generally speaking, most "good" headphones will have acceptably low distortion these days minus a few exceptions, so something like FR will be what you should focus on in most cases. Qualities like headphone style or driver type/performance, comfort, price, build quality etc. are generally going to secondary things, but depending on your use case and and personal preferences some of those attributes may need to be weighted accordingly.

When it comes to FR, everyone has their own preferences but one of the best "objective" standards when it comes to preference we have is the Harman target curve. Do some research on the forums or elsewhere to understand what this is and how it was derived, is used, and its limitations and caveats. A large majority of listeners will prefer this FR target, and even those that do not tend to have their preferences very close to it. So a good place to start may be headphones that conform well to that out of the box, and you can find such a list -> https://github.com/jaakkopasanen/AutoEq/blob/master/results/RANKING.md here, along with measured FRs from various contributors. Willingness to use EQ may be another compromise that could potentially allow headphones lower down the list that aren't as objectively good "on paper" in terms of harman compliance to perform superior to ones that might have a better FR out of the box. Regardless, Harman is a good starting point to either aim for or to begin your tweaking through EQ, and there are other target curves you can explore as well.

The headphone that you find most comfortable, most durable, fits most within your budget, or has that "je ne sais quoi" will rarely be the one that is the best overall sounding or has the best response. So it's a balancing act when picking something to check the most boxes and how a headphone is "rated" overall is a very personal affair, to a point where I would not even worry too much about how other people rate headphones. Comparison to other headphones can be helpful, but the most important thing you can take home from any review is objective data to reconcile your own experience and needs with a purchase decision.
 
OP
G

guzkiy

Member
Joined
Jan 25, 2022
Messages
13
Likes
6
I did headphone reviews in the past in HF and compare the headphone under review with other headphones. But there are few problems:

1. There are probably thousands of headphones models in the market. Let say I compare headphone A to headphone B and C. The chance is very rare that most of the readers have headphone B and C so the comparisons are useful for them. Most of the time the readers have no idea of all the headphones in the comparisons and the comparisons are not useful for them.

2. It takes days or weeks to do objective comparisons with various genres of music. It makes reviewing process a lot longer and time is always the limiting factor for all of us.

3. Price tag is not an indicator of sound quality. We all know that. So it is not always useful to compare headphone at similar price brackets. Sometime we can get better one at lower price bracket if we trust our ears and measurement, instead of our eyes looking at the price tags. Don't buy audio gears by price bracket, set your requirement and get the cheapest possible equipment that still meet your requirement.

4. We all should learn to read measurement results and estimate the sound quality. That's exactly what measurements are for.

I'm not against comparisons, and I always try to do comparisons in my review when time permit. But as mentioned above, comparisons may not always be useful.

My 2 cents ;)
@Earfonia :
I appreciate your 2 cents, obviously having done some HP review you know the situation better.
However, let me throw in a couple of thoughts:
1. Yes, a direct comparison of headphone model A to model B do not add much value to the reader. In the end of the day we are all after the best value for the money, so may not be even interested in more that one model. But where do I start looking for that model?
2. Agreed
3. Sorry, I agree to disagree here (with the statement we don't by gears by price). We do buy everything within the budget. So where is the system that tells me what *could be* the few best model to start my research from *within* my budget? It looks to me as a wide price bucket though. For instance, I am not going to try 9$ Sony MDR-ZX110 if I have 1000$ budget, and not because I am posh or something but because I know it would be a waste of time. I am going to start from $400 price tag (hifiman he400i?) and compare, and may change my budget as I go.
Another example (sorry it is Saturday - all kind of pleasures come to mind): When I try to choose wine in the restaurant - I do go from the price and ask 'what are the best red wines you have up to XXX$?'.

Yes, there are too many headphones on the market and owning/keeping them is an expensive business. On the other hand measurements take no space and no expiry date.
What if:
1. we select most important measurements (TD, Frequency response deviations from Harman, Group response deviations) and put some scores on those measurements.
2. Get a single 'total' score for the HP as geometric mean of those weighted factors. The lower the total score - the closer the HP to an 'ideal' one.
3. If we put it on a chart where on x axis we have price and on y axis we have the single 'total' score from 3), and fit a line there through averages (linear regression score/price). We have a system - everything below the line is better value than on top of the line.
4. As someone looking for a new pair of HP, I'd start from the middle of my budget, and try models that are below the line. Outliers below the line deserve immediate attention and those I'd start from. I can move up the price if not happy with the sound quality or down the price but still below the line.
I can do the math and the chart if someone has the data...
 
OP
G

guzkiy

Member
Joined
Jan 25, 2022
Messages
13
Likes
6
@jae: great link. I had something similar in mind - see my prev post.
I agree 'personal ranking' is the thing that matters, but there are 100s of HP on the market - as someone new to the HP audio I'd rather start trying from a few good ones. My ears tend to agree with the measurements to a big degree, so a system like in my prev post would provide a starting point (not necessary the final recommendation)
 

Earfonia

Active Member
Joined
Feb 21, 2019
Messages
278
Likes
398
Location
Singapore
....
3. Sorry, I agree to disagree here (with the statement we don't by gears by price). We do buy everything within the budget. So where is the system that tells me what *could be* the few best model to start my research from *within* my budget? It looks to me as a wide price bucket though. For instance, I am not going to try 9$ Sony MDR-ZX110 if I have 1000$ budget, and not because I am posh or something but because I know it would be a waste of time. I am going to start from $400 price tag (hifiman he400i?) and compare, and may change my budget as I go.
Another example (sorry it is Saturday - all kind of pleasures come to mind): When I try to choose wine in the restaurant - I do go from the price and ask 'what are the best red wines you have up to XXX$?'.

Yes, there are too many headphones on the market and owning/keeping them is an expensive business. On the other hand measurements take no space and no expiry date.
What if:
1. we select most important measurements (TD, Frequency response deviations from Harman, Group response deviations) and put some scores on those measurements.
2. Get a single 'total' score for the HP as geometric mean of those weighted factors. The lower the total score - the closer the HP to an 'ideal' one.
3. If we put it on a chart where on x axis we have price and on y axis we have the single 'total' score from 3), and fit a line there through averages (linear regression score/price). We have a system - everything below the line is better value than on top of the line.
4. As someone looking for a new pair of HP, I'd start from the middle of my budget, and try models that are below the line. Outliers below the line deserve immediate attention and those I'd start from. I can move up the price if not happy with the sound quality or down the price but still below the line.
I can do the math and the chart if someone has the data...

Yes agree, that's another approach.
Ratings / scores highly dependent on the criteria we set for the ratings / scoring. And the criteria is the tricky part here. Unlike speaker, with headphones and In-Ear Monitor, it is dependent on the individual HRTF. We don't know how far the differences of HRTF between individuals. Do we have more or less similar HRTF or vastly different? What I read and watch from Harman's / Sean Olive research, it seems HRTF between individual have similarities and not vastly different, and that's a good news. But more study need to be done.

Trying to set a realistic rating is exactly what I'm trying to do with In-Ear Monitor now. In the past I reviewed IEMs, Headphones, and DACs, but I'm not happy with my own reviews, not objective enough. The last few years I just focus on In-Ear Monitor and IEM measurement. Headphone measurement is tricky and complicated. There is leak factor from hair and wearing position. And I don't have big enough space for noise isolation container for headphone. IEM is small so I have built noise isolation container for IEM measurement.

For now this is my rating criteria, basically when all ratings are green it indicates that the IEM is good quality and worth considering. But even when it is all green, it doesn't mean the sound signature will suit everyone preference.

M13 Earfonia Rating Criteria.png


I just started reviewing IEM again this year. Hopefully the ratings is useful.
 
Top Bottom