• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required as is 20 years of participation in forums (not all true). Come here to have fun, be ready to be teased and not take online life too seriously. We now measure and review equipment for free! Click here for details.

Finding value in headphone measurements

Mad_Economist

Senior Member
Technical Expert
Joined
Nov 29, 2017
Messages
364
Likes
881
#1
With @amirm's experiment with the 5128 coming to a close, I thought it might be useful to take a moment to think about what makes headphone measurements useful to us and what we're looking for in them - particularly since Amir will be faced with the cost/benefit analysis of choosing a headphone test fixture if he does end up going that route.

I tend to separate headphone measurements into two overarching categories, each with its own sub categories:

The first is measurements which correspond to audible performance for end users, the "consumer utility" measurements. Frequency response in relation to a subjectively pertinent target response curve is the most obvious of these, as per Sean Olive's excellent work, it corresponds most significantly to subjective sound quality, but it's not the only one, and it's not entirely monodimensional. Sensitivity (I prefer to express this in dBSPL/V, personally), maximum linear output level, noise isolation from outside sound sources, leakage level outside the cups for a given level at the eardrum, and in the fairly niche cases where it comes up audible levels of distortion at conventional listening volumes are all significant to the subjective experience of using headphones.

Additionally, linear frequency response, as Amir's experiences in mounting headphones show, isn't just one curve - it's a spread of behavior under varying circumstances, and in my opinion there's value to capturing some of that breadth. A "typical"/"average" frequency response plot, showing reasonably charitable coupling and an average of multiple positions, is useful for comparing frequency response to a target curve. A plot of average and worst-case deviation from this average by frequency with positioning can be useful for showing the range of on-head behavior that can occur even on a single ear. Measurements with deliberately compromised seal - glasses, wigs, gaps of an intentional size under the pads, etc - can differentiate headphone tolerance of suboptimal wear, something which definitely occurs in real usage.

The second is measurements which can provide some technical or engineering interest, but do not directly have significance to audible behavior in most cases, a.k.a. "nerd bait". Examples here would include the electrical impedance of the headphone, most analysis in the time domain, nonlinear behavior at ludicrous extremes of output level (yes, I've played a 20hz sine wave at 120dBSPL before, no, I won't apologize), behavior beyond the audible band (I may be the only one, but I always got a chuckle out of Rin Choi forcing headphones to play 2hz sine waves), acoustic and electrical crosstalk, atypical measures of nonlinearity (coherence, swept IMD, etc), etc.

For my purposes, as a complete nerd, pretty much all properly conducted measurements have at least some interesting component - I don't buy that many headphones anymore, and I spent a lot of time tinkering with headphone design, so anything that lets you look inside another design is interesting to me. Sadly for me, Amir doesn't run this site for my sole benefit, so I thought I'd drop this here to ask: what do you want from headphone measurements? What do you go in looking for? What's the focus you're looking for from headphone analysis? Perhaps this might be helpful to Amir in deciding both what he's looking for from test equipment, and whether this headphone measurement malarky is worth his time at all! :D
 

solderdude

Major Contributor
Joined
Jul 21, 2018
Messages
7,123
Likes
13,926
Location
The Neverlands
#2
I just like to know how a headphone sounds (to me) that's basically why people buy them.
Multiple measurements can be used to confirm what is heard.
Measurements and listening can be used to make adjustments from a commercial product when it doesn't meet my preference.
Nothing as personal as headphones. They 'fit' you in comfort and sound or they don't. There will never be a headphone EVERYONE likes.
No matter how much research is done.
I would not rely on measurements only, certainly not from one measurement source, regardless how expensive the measurement gear is.
Having more measurements available increases the chance to find the actual flaws in a headphone (with listening as confirmation) and correct for this when corrections needed are only small.

Amir's desire to measure headphones (acc. to a standard) is helpful as yet another datapoint that can be used.
For selfish reasons I hope he does continue to measure and will only be interested in some plots.
Don't think its worth the money, not even worth 10k if the only goal is to just publish yet another set of measurements of headphones that have been measured extensively already. Not my decision and not my money.
 

pozz

Data Ordinator
Forum Donor
Editor
Joined
May 21, 2019
Messages
2,953
Likes
4,490
#3
Practicalities:
  • FR data for ideal and compromised seals over multiple runs and fittings.
  • Measurements of headphone response to EQ. For specific models, how possible is it to EQ a problem area without affecting the surrounding frequencies too much? For which frequency ranges is EQ most effective?
Nerdisms:
  • A general inquiry into headphone driver behaviour. To what extent is HF response just a tuned form of break-up? (This is likely better measured by a Klippel system than the 5128. But maybe some interesting conclusions will come out of discussions about particular models.)
  • A study of localization. Measure differences in FR and time arrival for each ear using tone bursts, uncorrelated sounds (diffuse, ambient noise) and correlated sounds (stereo and mono signals). Remove headphones and play signals over speakers to the dummy from different positions. Record ILD and ITD changes across the spectrum. I realize this is more like lab work than review work but it's damn interesting.
 

Jimbob54

Major Contributor
Forum Donor
Joined
Oct 25, 2019
Messages
4,139
Likes
3,916
#4
If I was lying I would say option A- but since I started lurking around ASR and HP measurements generally I have bought a few sets of not very cheap HP and not once looked at any measurements before clicking the button.

Is there was a "consumer " exec summary then a full on geek out after the fold I might dabble.
 

bobbooo

Major Contributor
Joined
Aug 30, 2019
Messages
1,369
Likes
1,722
#5
With @amirm's experiment with the 5128 coming to a close, I thought it might be useful to take a moment to think about what makes headphone measurements useful to us and what we're looking for in them - particularly since Amir will be faced with the cost/benefit analysis of choosing a headphone test fixture if he does end up going that route.

I tend to separate headphone measurements into two overarching categories, each with its own sub categories:

The first is measurements which correspond to audible performance for end users, the "consumer utility" measurements. Frequency response in relation to a subjectively pertinent target response curve is the most obvious of these, as per Sean Olive's excellent work, it corresponds most significantly to subjective sound quality, but it's not the only one, and it's not entirely monodimensional. Sensitivity (I prefer to express this in dBSPL/V, personally), maximum linear output level, noise isolation from outside sound sources, leakage level outside the cups for a given level at the eardrum, and in the fairly niche cases where it comes up audible levels of distortion at conventional listening volumes are all significant to the subjective experience of using headphones.

Additionally, linear frequency response, as Amir's experiences in mounting headphones show, isn't just one curve - it's a spread of behavior under varying circumstances, and in my opinion there's value to capturing some of that breadth. A "typical"/"average" frequency response plot, showing reasonably charitable coupling and an average of multiple positions, is useful for comparing frequency response to a target curve. A plot of average and worst-case deviation from this average by frequency with positioning can be useful for showing the range of on-head behavior that can occur even on a single ear. Measurements with deliberately compromised seal - glasses, wigs, gaps of an intentional size under the pads, etc - can differentiate headphone tolerance of suboptimal wear, something which definitely occurs in real usage.

The second is measurements which can provide some technical or engineering interest, but do not directly have significance to audible behavior in most cases, a.k.a. "nerd bait". Examples here would include the electrical impedance of the headphone, most analysis in the time domain, nonlinear behavior at ludicrous extremes of output level (yes, I've played a 20hz sine wave at 120dBSPL before, no, I won't apologize), behavior beyond the audible band (I may be the only one, but I always got a chuckle out of Rin Choi forcing headphones to play 2hz sine waves), acoustic and electrical crosstalk, atypical measures of nonlinearity (coherence, swept IMD, etc), etc.

For my purposes, as a complete nerd, pretty much all properly conducted measurements have at least some interesting component - I don't buy that many headphones anymore, and I spent a lot of time tinkering with headphone design, so anything that lets you look inside another design is interesting to me. Sadly for me, Amir doesn't run this site for my sole benefit, so I thought I'd drop this here to ask: what do you want from headphone measurements? What do you go in looking for? What's the focus you're looking for from headphone analysis? Perhaps this might be helpful to Amir in deciding both what he's looking for from test equipment, and whether this headphone measurement malarky is worth his time at all! :D
I'd agree with most of that, although I'd move electrical impedance into the 'measurements which correspond to audible performance for end users' group, especially for IEMs, which often have very low and wildly varying impedance with frequency, which can cause audible frequency response changes with all but ideal (very low) output impedance sources. Take the Campfire Andromeda for example, which has a ~9 dB increase in the treble with just a 10 ohm impedance source (which are not uncommon). And it's not just low impedance headphones that are affected, due to the fact that many stereo and AV receiver headphone outs (the latter very useful for e.g. watching movies without disturbing your household/neighbours at night) have huge output impedances (some up to 500 ohms!), so knowing the impedance curve of a headphone will tell the end user how and by how much its frequency response will be affected in these scenarios.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
30,319
Likes
89,619
Location
Seattle Area
#6
You know... as I packed the 5128 to send it back I got sad. Maybe it was because the thing almost looks human. :) In reality I think it was because as flawed as the measurements are headphones are, it gave me a personal insight I did not have about them. Nor had I gotten that from reading a bunch of graphs elsewhere.

When I measure things, I try to tease out a story. What is the device about? Why is it acting the way it is? What does it net out to? As much as there are headphone sites out there with extensive measurements, I miss this in them.

OP gives me hope that there is more we can measure to get that story. Even if we don't want to buy something, it is good to be informed about what is out there.
 
OP
Mad_Economist

Mad_Economist

Senior Member
Technical Expert
Joined
Nov 29, 2017
Messages
364
Likes
881
Thread Starter #10
I'd agree with most of that, although I'd move electrical impedance into the 'measurements which correspond to audible performance for end users' group, especially for IEMs, which often have very low and wildly varying impedance with frequency, which can cause audible frequency response changes with all but ideal (very low) output impedance sources. Take the Campfire Andromeda for example, which has a ~9 dB increase in the treble with just a 10 ohm impedance source (which are not uncommon). And it's not just low impedance headphones that are affected, due to the fact that many stereo and AV receiver headphone outs (the latter very useful for e.g. watching movies without disturbing your household/neighbours at night) have huge output impedances (some up to 500 ohms!), so knowing the impedance curve of a headphone will tell you how and by how much its frequency response will be affected in these scenarios.
I suspected that there might be contention on putting electrical Z in the "not audibly significant" section - my reasoning, at the risk of being controversial, largely boils down to a position that for amplifier-headphone impedance interactions to significantly impact frequency response, a minimum of one of the amplifier designer and headphone designer must have made a terrible mistake...
 
OP
Mad_Economist

Mad_Economist

Senior Member
Technical Expert
Joined
Nov 29, 2017
Messages
364
Likes
881
Thread Starter #11
You know... as I packed the 5128 to send it back I got sad. Maybe it was because the thing almost looks human. :) In reality I think it was because as flawed as the measurements are headphones are, it gave me a personal insight I did not have about them. Nor had I gotten that from reading a bunch of graphs elsewhere.

When I measure things, I try to tease out a story. What is the device about? Why is it acting the way it is? What does it net out to? As much as there are headphone sites out there with extensive measurements, I miss this in them.

OP gives me hope that there is more we can measure to get that story. Even if we don't want to buy something, it is good to be informed about what is out there.
Man, you can go pretty deep down that rabbit hole if you want to, as well - one of the nice things about headphones is that you can do a lot of "playing with acoustics" type dives with them; that was always one of my favourite features of Innerfidelity under Tyll's tenure, actually - Bob Katz too at times when they were hanging out. His review of the Bose Soundware is a good example, but really just about any time an "eccentric" headphone came across his desk - or he just felt like it - you'd get some sort of interesting probing into how things worked and why. He wasn't always right, and he wasn't perfect, but damn I loved the heart of IF under him.

FWIW, I do think that at least some of the sadness is anthropomorphization though - I know I always feel weird putting a HATS in its shipment box, I feel like an undertaker!
 

JohnYang1997

Major Contributor
Technical Expert
Manufacturer
Joined
Dec 28, 2018
Messages
5,053
Likes
8,599
Location
China
#12
Reading measurements is the simplest thing to pick out garbage to avoid. When the measurements are good enough then it's worth it to give it a try. Then decide to buy it or not.

In terms of measurement itself, I need someone/something I trust to produce the measurements. Otherwise it's simply meaningless to me.

More in the details, if one gets used to and gains experience with measurements produced by certain setups, you can find details that indicates some of the qualities that aren't easily interpreted. Like in the high frequencies.
 

solderdude

Major Contributor
Joined
Jul 21, 2018
Messages
7,123
Likes
13,926
Location
The Neverlands
#14
When I measure things, I try to tease out a story. What is the device about? Why is it acting the way it is? What does it net out to? As much as there are headphone sites out there with extensive measurements, I miss this in them.
I have been curious about the questions below, the moment you mentioned you plan to measure headphones as well.
While at the same moment you are already struggling to keep up with the huge amount of measurements and then have to bite yourself into something 'new'. I fully understand the last part though, which probably is the most appealing aspect.


What exactly is it that you are missing in the measurements already out there ?

What is the plan you set out to do, what info would you add that is not available on other websites ?

How extensive would the reporting be and what plots/info would you put in there that isn't present on other sites ?
IMO when it isn't complete (so you don't have to peruse other websites to gather relevant info) what would be the addition ASR could bring that is worth the investment to the headphone enjoyment community (which is a rather small one)
 
Last edited:

trl

Major Contributor
King of Mods
Joined
Feb 28, 2018
Messages
1,326
Likes
1,344
Location
Iasi, RO
#15
[...] what do you want from headphone measurements? What do you go in looking for? What's the focus you're looking for from headphone analysis? Perhaps this might be helpful to Amir in deciding both what he's looking for from test equipment, and whether this headphone measurement malarky is worth his time at all! :D
-1) Freq. response - it will tell me if it's bright or bass-head, also spikes around 5-6KHz will make me mad alot.
-2) Square and impulse response for bass (anything between 50...100Hz should do) - will give me a good idea how well it will reproduce the low-end freqs. (usually round corners means diaphragm will not be perfectly accurate for low-bass), also if bass is "fast" or "slow" (phase delay is high or not).
-3) Phase linearity and impedance across audible range - will tell me if it will pair well with "difficult" headamps (high impedance amps usually, but tubes amps too or badly designed amps that might oscillate if impedance is not constant).
-4) Fit and comfort - this is very important and without a good fit all other measurements will not matter, especially on IEMs.
 
OP
Mad_Economist

Mad_Economist

Senior Member
Technical Expert
Joined
Nov 29, 2017
Messages
364
Likes
881
Thread Starter #16
-2) Square and impulse response for bass (anything between 50...100Hz should do) - will give me a good idea how well it will reproduce the low-end freqs. (usually round corners means diaphragm will not be perfectly accurate for low-bass), also if bass is "fast" or "slow" (phase delay is high or not).
Somewhat confused by what you mean regarding square waves here.

In terms of phase, why not just directly derive group delay from the linear response of the headphone? Mind you, I don't think you'll find much there - I surely haven't...
 

JohnYang1997

Major Contributor
Technical Expert
Manufacturer
Joined
Dec 28, 2018
Messages
5,053
Likes
8,599
Location
China
#17
Somewhat confused by what you mean regarding square waves here.

In terms of phase, why not just directly derive group delay from the linear response of the headphone? Mind you, I don't think you'll find much there - I surely haven't...
Most people don't know how to interpret square wave and impulse response. The dependency between frequency response and time based responses tells us that ideal impulse response and square waves aren't what we thought. It needs to go up and down three times like er2se/er4s.
 
OP
Mad_Economist

Mad_Economist

Senior Member
Technical Expert
Joined
Nov 29, 2017
Messages
364
Likes
881
Thread Starter #18
Most people don't know how to interpret square wave and impulse response. The dependency between frequency response and time based responses tells us that ideal impulse response and square waves aren't what we thought. It needs to go up and down three times like er2se/er4s.
Well, I suppose one could use an input filter that's equivalent to the intended target response, and then the "ideal" response would be equivalent to a (band-limited) square-wave, but I was unclear on what else he intends to derive from them than frequency response and the corresponding time domain information.

Edit: Rather old paper doing the input filter thing with impulses, as an example.

Edit 2: Come to think of it, I believe that B&K and Head-Acoustics' portable recorder preamp/PSU units have inverse DF-HRTF EQs on their mic ins. Maybe they should have marketed them as "oscilloscope-friendly" :p
 

JohnYang1997

Major Contributor
Technical Expert
Manufacturer
Joined
Dec 28, 2018
Messages
5,053
Likes
8,599
Location
China
#19
Well, I suppose one could use an input filter that's equivalent to the intended target response, and then the "ideal" response would be equivalent to a (band-limited) square-wave, but I was unclear on what else he intends to derive from them than frequency response and the corresponding time domain information.

Edit: Rather old paper doing the input filter thing with impulses, as an example.
One can use an inverse of target (minimum phase) for either the headphone output or the sound card input. I prefer the latter. This also applies to CSD measurements.
 
Last edited:
OP
Mad_Economist

Mad_Economist

Senior Member
Technical Expert
Joined
Nov 29, 2017
Messages
364
Likes
881
Thread Starter #20
One can use an inverse of target (minimum phase) for the headphone either input or the sound card input. I prefer the latter. This also applies to CSD measurements.
Naturally so. I generally rarely see much utility from that sort of measurement for headphones, though - we can pull the phase and delay data directly from an impulse response, yaknow?
 
Top Bottom