FR Chart Observations - And a way to find the best sound signature for 'almost' everyone

[KR8NUX]

In my attempt to find my preferable FR for headphones, I went on various sites to explore and found that RTINGs had a decent way of documenting them. I know that since they might have had only one unit at hand, their charts might not be representative of an aggregated FR curve for their devices, but at least, its a starting point. I would also be looking at FRs from Innerfidelity or SBAF to correlate and explore further.

Getting to the main point - Post conducting some surveys with Audiophiles and general consumers, I would attempt (or am hoping to, if I get time from class and work) to use them as a training set in Machine Learning, to find a better suited FR Curve. I know audio listening tastes are subjective, but the beauty of ML/DL is that given enough training sets and responses we would come closer to the ideal FRC, arguably more accurate than Harman's on-ground testing approach. The reason I say this is because the sample set would be much larger than whatever a normal blind study/Focus group study can realistically have. If any of the subs subscribers are working in ML, I would love to collaborate because I am just a novice with limited skills and the project would greatly benefit from proper experts.

So, after going through the entire lot, I am posting a few that caught my eye. I looked for the ones which are typically flat, and the ones near the Harman Curve. I can say that the others on their list can be probably safely ignored.
The ones that caught my attention, for being "FLAT" are :
1More Triple IEM- https://www.rtings.com/headphones/1-2/graph#432/2031 - Quite nice for a relatively cheap set of IEMS
Bose Quietcontrol 30 -https://www.rtings.com/headphones/1-2/graph#407/2031 - Amazingly flat, treble seems to be bad
Bose QC 25 https://www.rtings.com/headphones/1-2/graph#237/2031 - Still looks great
Bose Soundlink On Ear -https://www.rtings.com/headphones/1-2/graph#313/2031 - Still flat
Bose Soundlink On Ear 2 -https://www.rtings.com/headphones/1-2/graph#322/2031 - As the above, slight channel imbalance
Bose Soundsport Wireless - https://www.rtings.com/headphones/1-2/graph#358/2031- Still looks good to me
More Bose - almost all of them look quite good with being flat.
Audeze Mobius -https://www.rtings.com/headphones/1-2/graph#689/2031 - Closed back seems good
Amazonbasics -https://www.rtings.com/headphones/1-2/graph#395/2031 - Kinda Amazed by this
LCD2 -https://www.rtings.com/headphones/1-2/graph#691/2031 - Seems good, midrange hump, probably better vocals.
Anker Soundbuds - https://www.rtings.com/headphones/1-2/graph#570/2031 - Another unexpected one
Anker Soundcore - https://www.rtings.com/headphones/1-2/graph#634/2031- And another one
Astro A50 - https://www.rtings.com/headphones/1-2/graph#501/2031- Great for a pair of gaming cans
Audiotechnica M50X https://www.rtings.com/headphones/1-2/graph#295/2031- Poor channel matching, something up in the mids.
M70X- https://www.rtings.com/headphones/1-2/graph#374/2031 - Better than the last
MSR7- https://www.rtings.com/headphones/1-2/graph#478/2031 - Finally better channel matching by AT
B&O H6 -https://www.rtings.com/headphones/1-2/graph#540/2031 _ Seems to be the only flat one from B&O
Brainwavz HM5 - https://www.rtings.com/headphones/1-2/graph#594/2031- Very flat, again something up with mids
Beats X- https://www.rtings.com/headphones/1-2/graph#436/2031 - Contrary to typical hate ?
Beats EP On Ear- https://www.rtings.com/headphones/1-2/graph#522/2031 - Again, very good ?
Beats Solo2Wireless - https://www.rtings.com/headphones/1-2/graph#388/2031- Still seems OK ?
URBeats - https://www.rtings.com/headphones/1-2/graph#347/2031 - Seems like what people slam beats about ?
BeyerDynamics DT770 - https://www.rtings.com/headphones/1-2/graph#440/2031 - Nothing to see here, kinda expected
DT880-https://www.rtings.com/headphones/1-2/graph#431/2031 - The above, open back changes lows, as expected
DT990PRO- https://www.rtings.com/headphones/1-2/graph#329/2031 - Seems better than the other 2.
DT1990 PRO- https://www.rtings.com/headphones/1-2/graph#628/2031- WOW. (But treble)
DT1770 PRO-https://www.rtings.com/headphones/1-2/graph#714/2031- Similar but something off again in the same low mids.
B&W P5 Wireless - https://www.rtings.com/headphones/1-2/graph#366/2031 - Best from B&W
B&W P7 - https://www.rtings.com/headphones/1-2/graph#348/2031 - Seems like a slight downgrade
Corsair HS60 - https://www.rtings.com/headphones/1-2/graph#686/2031 - Another decent gaming pair
ER4R -https://www.rtings.com/headphones/1-2/graph#621/2031- Looks great
Harman Kardon NC - https://www.rtings.com/headphones/1-2/graph#259/2031 - Doesnt seems to be following their curve that much
Hifiman Ananda - https://www.rtings.com/headphones/1-2/graph#670/2031 - What's on with the channel imbalance
Sundara - https://www.rtings.com/headphones/1-2/graph#589/2031 - Great planar
Edition X -https://www.rtings.com/headphones/1-2/graph#352/2031 - Ok but treble
HyperX Cloud Alpha - https://www.rtings.com/headphones/1-2/graph#627/2031- Another decent gaming pair
JBL Everest Elite 700 - https://www.rtings.com/headphones/1-2/graph#447/2031 - I like this a lot, would probably get this one personally
JBL Free - https://www.rtings.com/headphones/1-2/graph#591/2031 - Yet another good one
JBL Endurance Sprint -https://www.rtings.com/headphones/1-2/graph#676/2031- good but wassup with the treble
Jaybird X2 - https://www.rtings.com/headphones/1-2/graph#330/2031 - best looking one from Jaybird.
KZ AS10 - https://www.rtings.com/headphones/1-2/graph#698/2031 - Looking Nice for an El-cheapo IEM
Logitech G533 - ://www.rtings.com/headphones/1-2/graph#505/2031 - Another great gaming pair !
Mee X6 Plus - https://www.rtings.com/headphones/1-2/graph#537/2031 - Seems fine, bad channel imbalance
Marshall Major 2 - https://www.rtings.com/headphones/1-2/graph#394/2031
NAD VISO HP50 - https://www.rtings.com/headphones/1-2/graph#582/2031 - Great !
Oppo PM3-https://www.rtings.com/headphones/1-2/graph#344/2031 - Amazing Planar
Philips SHP9500 - https://www.rtings.com/headphones/1-2/graph#371/2031- Eerily similar to the HD600 - see https://www.rtings.com/headphones/1-2/graph#371/2031/325
Philips Fidelio NC1 - https://www.rtings.com/headphones/1-2/graph#251/2031 - Nice !
Samsung GearIconX - https://www.rtings.com/headphones/1-2/graph#567/2031 - Flatter than I expected
Samsung Level U Pro - https://www.rtings.com/headphones/1-2/graph#372/2031 - Also same
Sennheiser HD800S - https://www.rtings.com/headphones/1-2/graph#290/2031 - I guess this is the analytical target ?
Sennheiser HD600 -https://www.rtings.com/headphones/1-2/graph#325/2031 - Kinda opposite of HC ?
HD 598C -https://www.rtings.com/headphones/1-2/graph#409/2031- The flattest I found from Sennheiser
HD1 In Ear Wireless - https://www.rtings.com/headphones/1-2/graph#534/2031 - Sennheiser level channel matching
Momentum In Ear - https://www.rtings.com/headphones/1-2/graph#391/2031- Looks consumer friendly
Sony MDR 1A - https://www.rtings.com/headphones/1-2/graph#430/2031 - Looking good too
Sony MDR 7506 - https://www.rtings.com/headphones/1-2/graph#386/2031 - Kinda shows me why the audiotechnicas also have this drop in that mid
Sony WF 1000X https://www.rtings.com/headphones/1-2/graph#562/2031 - Looks good but the treble ?
Sony H900 -https://www.rtings.com/headphones/1-2/graph#617/2031 - Looks nice
Sony CH700 - https://www.rtings.com/headphones/1-2/graph#650/2031 - Very consumer looking.
Superlux HD 681 Evo -https://www.rtings.com/headphones/1-2/graph#523/2031 - Great for the price
Steelseries Arcticprogamedac - https://www.rtings.com/headphones/1-2/graph#616/2031 - Great gaming pair again
Vmoda Crossfade 2 Wireless - https://www.rtings.com/headphones/1-2/graph#445/2031 - Poor channel matching

We are already aware of the Harman Curve, so these might be "flat" but as per psychoacoustics
https://www.innerfidelity.com/content/harman-tweaks-its-headphone-target-response

I would love to hear the community's responses/critique/ideas on this.

 

[peder2tm]

Where would you get the data from?
As I understand it you would like to get training examples in the form of (frequency response, user rating) pairs.
Even with thousands of ratings you have to be very careful or you will end up with a result that just reflects the data selection bias.

 

[KR8NUX]

Yes, I agree, that's why I wanted to have a discussion to understand potential challenges which I might need to overcome. One way of sampling would be based on FR curves of popular devices - maybe get people to answer surveys that define their preference, what they like over another and maybe also find weighing biases about what 'part' they like.
I am trying to figure out an approach. The 'bias' part is actually an objective - the need is to find the curve that most people would bias positively towards, that is why run simulations first by normal curve fitting techniques and regression before I try any ML approach.

 

[DonH56]

Any such curve will be an average of listeners and thus may or may not make any given one listener happy. Over a large sample size you could learn to recognize outliers to come up with a reasonable target but of course the outliers won't like it...

As for an approach, you could define a fixed set of short examples (music, speech, whatever) and apply different EQ to them. Ask listeners to rate (order) the ones they prefer. Probably have to use a small set of short samples and curves then refine as you go. And you'll need to randomize the selections, probably repeating each one once or twice, to reduce listener fatigue (in this case boredom) and such.

Setting up controlled testing is a huge PITA so I have not done it in years. There are several folk here with extensive experience that can help identify the pitfalls and guide the testing methodology.

 

[solderdude]

The plots from Rtings are already compensated for the bass (Harman style)

IMO none of these headphones are truly 'ideal' and FR is important but there are also other properties.

Interested to see what you come up with.

 

[KR8NUX]

Any such curve will be an average of listeners and thus may or may not make any given one listener happy. Over a large sample size you could learn to recognize outliers to come up with a reasonable target but of course the outliers won't like it...

As for an approach, you could define a fixed set of short examples (music, speech, whatever) and apply different EQ to them. Ask listeners to rate (order) the ones they prefer. Probably have to use a small set of short samples and curves then refine as you go. And you'll need to randomize the selections, probably repeating each one once or twice, to reduce listener fatigue (in this case boredom) and such.

Setting up controlled testing is a huge PITA so I have not done it in years. There are several folk here with extensive experience that can help identify the pitfalls and guide the testing methodology.

Oh, but that is the primary research-driven scientific method driven way of doing things. Harman, True-Fi are already doing that and I am not looking to do any better than them with their extensive on ground research processes. As I mentioned, the beauty of ML is scale - I can take thousands of 'preferred' sound signatures to train the model, thereby coming to an arguably more reliable signature, which can keep evolving by running newer training sets and compensation biases. Machine learning works in a different way from the conventional approach, so we do not need to create large sample sets and go through standard regressions - I would just create training sets and assign weighted preferences and run them on certain algos that seem appropriate, and try around with different regressions to see which seems to work 'sensibly'.

Thanks for giving me more insight on how primary research works in the field, would help me pick out what to adopt as training models and look for better resources.

Plus, outliers would exist, we always presume the best we can take the model to is 99% accuracy. This is still better than 85-90% achieved with standard research techniques, because the machine would try to self learn what seems to be a better regression.
All this only if I get time from class and work

 

[KR8NUX]

The plots from Rtings are already compensated for the bass (Harman style)

If you are looking for a reference grade headphone that reacts well to simple EQ:
HD600, HD650, HD6XX, HD58X, HD660S to get a feel for proper tonal balance.
There are plenty of other headphones that perform 'better' in certain aspects and can also be compensated.

I recommend to only use some mild EQ and not try to EQ exactly acc. to graphs as all graphs are inherently incorrect. When you try to compensate for those errors things usually don't get better.
Use headphones that require an absolute minimum to 'correct' and don't use headphones that have sharp dips or peaks or multiples of it.

You can't polish a turd.

And remember... once EQ'ed 'flat' acc. to Rtings plots (or others) is NO guaranty you will like the sound.

Oh, I am not looking for headphones, I just compiled a list of the compensated 'flatter' ones from RTINGs and their tests. I know they are compensated, if you look at the charts I posted, I did post the flat ones taking into account the compensation. The RAW FR is not the charts I have put up
My main point of discussion was about trying to incorporate an ML/AI approach towards the most preferred sound signature, to find a possible 'upgrade' to the harman curve.

 

[solderdude]

Yes, I got that.. in the end.

Curious to see what you come up with. Whose 'neutral' is close to your found 'neutral' and how much that deviates from known 'target curves'.
Then it would be a question on which 'volume' (SPL, listening level) it is based and what kind of music is used and who's 'taste' is included in the evaluation.

I mean you can probably find an equal amount of people preferring the sound of the HD800 and the Nighthawk where both claim the sound is as neutral as it gets.

I had both the HD600 and SHP9500 and while both are bass shy they did not sound similar to me at all.
They don't measure the same either. There are differences of 5 to 10dB between 1kHz and 20kHz which are very audible.
But below 1kHz they measure quite the same on Sam's rig but different on mine, above 1kHz the SHP9500 the differences are enormous on my rig.

So ... yes I am interested in the results and how they compare to other rigs.
Rtings target is pretty good to my ears.

 

[KR8NUX]

Yes, I got that.. in the end.

Curious to see what you come up with. Whose 'neutral' is close to your found 'neutral' and how much that deviates from known 'target curves'.
Then it would be a question on which 'volume' (SPL, listening level) it is based and what kind of music is used and who's 'taste' is included in the evaluation.

I mean you can probably find an equal amount of people preferring the sound of the HD800 and the Nighthawk where both claim the sound is as neutral as it gets.

So ... yes I am interested in the results.

Thanks, I hope to get started on this at least before the holidays. Fingers crossed :/
Oh and the other aspects - Loudness, Transient response, etc, that's another ball game altogether. I don't even know if there is any largely verifiable and quantifiable dataset available for those and what people's preferences are.
For loudness, I mean, I can understand safe and unsafe hearing levels, but that's all.

 

[solderdude]

There is a rather substantial difference in tonal balance when listening at low listening levels (relaxed in the evening) and 'analytical' listening during the day.
The same headphone can sound very different with the same music between these circumstances.
That may be something to take into consideration.... Phon curves.

Start with FR first... at 'realistic' listening levels.

 

[Blumlein 88]

Yes, you need to solve what Harman in speakers called the circle of confusion. I'm not so sure they've gotten it right with headphones the way they did with speakers.

 

[SIY]

Yes, you need to solve what Harman in speakers called the circle of confusion. I'm not so sure they've gotten it right with headphones the way they did with speakers.

IMO, headphones are MUCH worse in this regard than speakers. In my soon-to-appear review of the miniDSP EARS, I coined the term "hypersphere of confusion."

 

[dc655321]

My main point of discussion was about trying to incorporate an ML/AI approach towards the most preferred sound signature, to find a possible 'upgrade' to the harman curve.

Interesting idea.

I have a little experience with machine learning, enough to know that the hardest part is often obtaining and cleaning the data.

Do you have plans for getting access to the numeric spectra? Most sites simply post plots, not the data. Could extract data from plots, but that's a sizable undertaking itself...
Any thoughts on normalization processes? Apples and oranges in a training set is fatal...

Initially, I would think an unsupervised technique may be insightful: a nearest-neighbor or clustering analysis to guide further studies.

 

[KR8NUX]

Interesting idea.

I have a little experience with machine learning, enough to know that the hardest part is often obtaining and cleaning the data.

Do you have plans for getting access to the numeric spectra? Most sites simply post plots, not the data. Could extract data from plots, but that's a sizable undertaking itself...
Any thoughts on normalization processes? Apples and oranges in a training set is fatal...

Initially, I would think an unsupervised technique may be insightful: a nearest-neighbor or clustering analysis to guide further studies.

Ok, so I was looking at how reddit user u/jaakkopasanen created the AutoEQ git library, and he has a library of over 1300 headphones now. I have been talking to him about how he has collected and cleaned the data, but I think most of them are already there in the git library he posted.
Going back to where it started and how I got the idea in the first place.
https://www.reddit.com/r/headphones/comments/a08bjq
He has posted it here. I am looking to start with his database to collect the headphone measurements. As for the approach towards the 'preferred curve', that, I would need to think further on. I wanted to approach it by using typically preferred headphone FRs (stuff that people generally like and is at the top of the list of most sought after ones), but I would brainstorm a bit further and take feedback from everyone on that. I am also severely limited by my Python capabilities, most of the stuff I do for ML is on MATLAB, because I am primarily in the IoT and building data space, so coding is not one of my biggest strengths.

Yes, an unsupervised nearest-neighbour is the first one I thought of. Great to finally get some validation and input from the community!

 

[Blumlein 88]

I fully agree with the hypersphere of confusion idea.

In speakers we have a fairly straightforward idea of measurements in an anechoic chamber or in an outdoor area free of obstructions.

We have nothing remotely approaching such a direct base level situation to build upon. And that is before you start placing phones on heads and all the variability of that. I don't like making such criticism without a suggestion for solving it, but that is the situation.

 

[dc655321]

I am also severely limited by my Python capabilities, most of the stuff I do for ML is on MATLAB, because I am primarily in the IoT and building data space, so coding is not one of my biggest strengths.

The weapons of choice are typically either Python, Matlab, or R. Pick your poison ;-)
My experience is with scikit-learn. In terms of learning resources, this book is a really good overview of scikit-learn and python as applied to machine learning.

 

[KR8NUX]

The weapons of choice are typically either Python, Matlab, or R. Pick your poison ;-)
My experience is with scikit-learn. In terms of learning resources, this book is a really good overview of scikit-learn and python as applied to machine learning.

Good resource! A used one is only 10$, I'm getting it.

 

[dc655321]

Good resource! A used one is only 10$, I'm getting it.

There is a 2nd edition out now (that was fast!). Can't vouch for that one.