Master Complaint Thread About Headphone Measurements

[amirm]

Put me aside for a minute: did you just make up stupid stuff just in order to ban people who criticize you in order to silence them?

No, we can use a class clown and seems like you volunteered. Go after me but don't you dare go after our large membership from other parts of the world. We have zero tolerance for racism of any kind.

 

[solderdude]

I thought this was a master complaint thread about headphone measurements not a master complaint thread about Amir.

Just another Amir hater fueled by talk on other forums/media.

 

[pkane]

Just in case someone is interested in the non-linear shiftings between different rigs and headphone designs, our colleague @Mad_Economist has linked a very comprehensive roundup of this topic in the 5128 HATS thread:
https://forum.headphones.com/t/measurements-charts-graphs-software-methods/2333/38

You may also check this analysis examining the delta between a few established measurement databases:
https://www.audiosciencereview.com/...omparing-headphone-measurement-systems.12634/
Even if you compensate for the average unit-to-unit variation of the devices under test, the demonstrated fundamental issues of this field are hard to omit.

Regards
Dreyfus

The concept of measuring headphones away from the ear is not necessarily doomed to failure. For example, measuring distortions inherent in the transducer is easy enough without getting the artificial or real ears involved. Measuring linearity can also be done with a flat plate or a similar device. Frequency response, resonances, reflections, etc., are going to be near impossible to do with a flat plate, and are best done with the real ear attached to the transducer, unless one can measure precisely the difference between flat plate and real ear transfer functions.

Here are some measurements I dug up from a while back, testing HD650 with a microphone attached to a flat plate, the headphone cup pressed against the plate with a little pressure. The plate was a 2-inch thick manufactured wood, microphone was a calibrated in-ear mic that I also use to measure headphone response from inside the ear.

Red is flat plate, green is the in-ear mic inside my left ear (with the ear canal blocked by the mic):

Obvious large differences below about 2.5k, but seem to almost agree between 2.5k and 8k, measured at 90dBSPL. Maybe I should use some ballistic testing gel instead of a flat plate, to better simulate head consistency?

Same, but aligned at the lower frequencies:

Here's the measurement of THD, also at 90dBSPL mounted on the same flat plate:

 

[DualTriode]

Hello All,

I am inclined to believe that the red and green plots do more alignment in the bass and sub-bass frequencies and to some limited extent go their separate ways in the high frequencies.

To hear the story told you would think that there would be completely divergent red and green stories.

Without the ear in the headphone cup there is the potential of a much clearer view of the performance of the headphone hardware. Things like; frequency response, driver stability, phase and distortion. All this is likely disturbed by putting a rubber or real ear in there. The same kind of testing you would do with a plane wave tube bolted on a compression drive.



Thanks DT

 

[pkane]

Hello All,

I am inclined to believe that the red and green plots do more alignment in the bass and sub-bass frequencies and to some limited extent go their separate ways in the high frequencies.

To hear the story told you would think that there would be completely divergent red and green stories.

Without the ear in the headphone cup there is the potential of a much clearer view of the performance of the headphone hardware. Things like; frequency response, driver stability, phase and distortion. All this is likely disturbed by putting a rubber or real ear in there. The same kind of testing you would do with a plane wave tube bolted on a compression drive.



Thanks DT

For fun and entertainment, I measured HD650 on a flat plate, repositioning the headphone cup by about 1mm horizontally after each measurement. Here's the result:

Showing the average of all the positions:

This points to the frequencies being mostly consistent up to, and reduced just after 1kHz, seemingly proportional to the displacement from the center. Whether this is caused by the microphone or the headphones will require additional testing, but obviously very important to center the cup over the microphone when measuring for consistency.

Don't know what was going on around 70-85Hz with a couple of the measurements. Maybe some noise interfered, such as hard disk spinning up/down?

Here's the same plot but over a linear frequency scale:

 

[DualTriode]

For fun and entertainment, I measured HD650 on a flat plate, repositioning the headphone cup by about 1mm horizontally after each measurement. Here's the result:

View attachment 103549

Showing the average of all the positions:

View attachment 103558

This points to the frequencies being mostly consistent up to, and reduced just after 1kHz, seemingly proportional to the displacement from the center. Whether this is caused by the microphone or the headphones will require additional testing, but obviously very important to center the cup over the microphone when measuring for consistency.

Don't know what was going on around 70-85Hz with a couple of the measurements. Maybe some noise interfered, such as hard disk spinning up/down?

Here's the same plot but over a linear frequency scale:

View attachment 103553

pkane,

I like your flat plane HD650 frequency plots.

I hope that there are more to come with impedance, distortion and the like.

I do have a set of similar HD600's to measure with flat plane and 45CA. I am shopping for a microphone to use with the flat plate.

I do plan to measure the AKG K712' in various conditions and amplifiers.

Thanks DT

 

[3125b]

Great, complaining is my favourite
But seriously, thank you for the great and educational reviews.

However, and that might have come up before, I'm not happy with the levels you measure distortion at, 124dBSPL seems very unrealistic.
Most measure at 90dB, wich makes sense.
I like the fact that you measure three different levels, however I'd suggest 75, 90 and 105dBSPL to correspond with normal, loud and "ow my ears" real life listening situations.

 

[pkane]

pkane,

I like your flat plane HD650 frequency plots.

I hope that there are more to come with impedance, distortion and the like.

I do have a set of similar HD600's to measure with flat plane and 45CA. I am shopping for a microphone to use with the flat plate.

I do plan to measure the AKG K712' in various conditions and amplifiers.

Thanks DT

I've not found the amplifier to be an issue, at least with the headphones I'm using and at levels I'm testing. My Apogee interface has a pretty underpowered headphone output jack, and I also have a THX AAA 789 headamp. Both produce very similar results, but I'm not stress-testing (don't want to damage the headphones.)

 

[solderdude]

Great, complaining is my favourite
But seriously, thank you for the great and educational reviews.

However, and that might have come up before, I'm not happy with the levels you measure distortion at, 124dBSPL seems very unrealistic.
Most measure at 90dB, wich makes sense.
I like the fact that you measure three different levels, however I'd suggest 75, 90 and 105dBSPL to correspond with normal, loud and "ow my ears" real life listening situations.

It's a typo and is 114dB.

 

[solderdude]

I've not found the amplifier to be an issue, at least with the headphones I'm using and at levels I'm testing. My Apogee interface has a pretty underpowered headphone output jack, and I also have a THX AAA 789 headamp. Both produce very similar results, but I'm not stress-testing (don't want to damage the headphones.)

Aside from output resistance I also have not found an amplifier to give any different results.
Of course assuming distortion is low enough and FR is well extended in both directions and there is enough power to drive it to the required maximum levels.

 

[Dreyfus]

@pkane:
Thank you for sharing your measurements!

The graphs of the in ear vs flat plate rig do line up quite nicely in the bass and mid frequencies, hence prove a good seal for the HD650 on your head. May I ask what capsule and fixing method you are using?

The "crossover" where the ear gain kicks in appears to be around 1 kHz for your setup. That matches what I would expect looking at the ear resonance model by Edgar Shaw (The External Ear, 1974). The gain however is rather reserved with just around 5 dB. That might be because to the placement of the capsule and your specific mic calibration (0° match to the Behringer ECM8000 I guess?).

The flat plate rig does offer decent repeatability without the "highly reactive" ear in place, indeed. That makes it very suitable for things like quality ensurance in production lines or isolated studies of driver-related resonances.
Also THD measurements might be possible in case you have a very good (high SNR, low THD) microphone.
One thing we have to keep in mind though is that there is still much less energy in the treble and that the specific resonances will all start to interact once we put the thing on an actual human head. I would not draw conclusions about a headphone solely based on flat plate measurements. But who am I ... someone who does not even trust in ear simulators.

@3125b:
Most tests are performed at such high levels because of the SNR in the measuring environment. If we wanted to capture THD + driver noise down to 0.1% we would need at least 60 db of ambient noise attenuation.
If you record at night and turn off all unneccessary electronic devices in your environment (also make sure there is no PSU hum or fan noise induced by the equipment) the 90-100 dB mark should be quite doable. But at some point you will have no other choice than building an isolation chamber.

 

[Dreyfus]

Right. I'm just concerned that variation due to position has more to do with the actual function of the ear than with any deficiencies in reproduction. I assume that the ear physiology creates these peaks/valleys including variations in frequency response to help with detecting the direction of sound. If we spend too much time trying to "correct" these variations to some specific, static target, we may just be destroying or distorting information that the ear expects to receive to perceive the sound as being more natural. The area where the measurements change drastically with tiny position changes are perhaps better not corrected at all.

Well, we got the Headphone Harman Curve to rely on as a Target Curve....which is kinda all we got when it comes to measurments & targets. That doesn't show a peak at 7kHz, therefore if you're EQ'ing to the target curve then you'd remove that peak if it was consistent across all your measurements

There is no satisfactory answer for to that topic. Less averaging and limitation to a small circle of seating position provides better accuracy. Still, the real life performance with varying head and ear shapes + seating positions can tell a fundamentally different story. That is where averaging comes into play. Averaging however can introduce its own kind of distortion because it simplifies the data set and only looks for mean values, smearing or even cutting away specific driver- or ear-related features. That gets especially tricky one we have multiple distinct heatzones in the Y axis.
Let's assume a set of data gathering a large amount of seating positions shows a clear tendency to resonate either with 98 dB or 92 dB around 8 kHz with just a few points between those extremes. What would be the reasonable mean value for this instance? 95 dB? Probably not.

IMO the healthies approach is to make as many measurements as possible and then generate a heatmap that shows the exact patterns than occur once we put a headphone on a human head or ear simulator and start to move the thing all around the reference point. Single point measurements as much as simplified averages are both imperfect in certain ways.

As for the Harman target rating and equalization though we should only use averaged results that smooth the ear resonance the same way as the target does.

 

[pkane]

@pkane:
Thank you for sharing your measurements!

The graphs of the in ear vs flat plate rig do line up quite nicely in the bass and mid frequencies, hence prove a good seal for the HD650 on your head. May I ask what capsule and fixing method you are using?

The mic is SP-TFB-2 which I also use for binaural recordings.

For flat plate pressure, I used a springy metal clip (thick) wire wrapped around the sides of the plate to push the headphone towards the plate at the connector. On my head, I don't use anything beyond what the headphone provides, but do position the headphone the best I can and push it against the ear to ensure a better seal.

 

[Feelas]

One thing we have to keep in mind though is that there is still much less energy in the treble and that the specific resonances will all start to interact once we put the thing on an actual human head. I would not draw conclusions about a headphone solely based on flat plate measurements. But who am I ... someone who does not even trust in ear simulators.

@Dreyfus, since you're very skeptical about averaged measurements & HRTF discrepancy of testing gears heads in general: what do you think about this? It's interesting to notice, that people with molds placed to disrupt the known HRTF can recover pretty quickly into proper localization; I think that's a good way to argue that since we're highly adaptable, the HRTF dilemma isn't that problematic, unless gear masks & throws frequencies away at whole, since then the information is gone, not malformed.

 

[DualTriode]

The mic is SP-TFB-2 which I also use for binaural recordings.

For flat plate pressure, I used a springy metal clip (thick) wire wrapped around the sides of the plate to push the headphone towards the plate at the connector. On my head, I don't use anything beyond what the headphone provides, but do position the headphone the best I can and push it against the ear to ensure a better seal.

@pkane while you are at it, please put those microphones in your ears and put the HD650's on your head and run a couple more test plots with your real ears in the HD650 cups.

Thanks DT

 

[Rock Rabbit]

Lower distortion at lower frequency (?) There must be something wrong!
For dynamic driver below resonance the max diaphragm displacement Xmax stays the same (same voltage at almost DC coil R then same B*l*I force). For sinusoidal X, velocity is proportional to X*frequency then velocity drops 6 dB/octave at lower frequency, taking acoustic resistance (proportional to f^2) we have the famous drop of 12 dB/octave for semi open hp (or closed box LS). Then at almost constant Xmax the source of distortion (B*l motor and compliance) works at same amplitude, but the fundamental tone in SPL is always decreasing at lower frequency...so THD must always growth at lower frequency (base freq on denominator of THD).
But some THD graphics shows a decreasing value at the lowest frequency even for very high SPL output (HD 650, Focal Clear...) with lower thd at 20 Hz.

For planars (or AMT) it seems suspicious too, to get flat at lower frequency X must grow 4x per lower octave. Flat to 10 or 20 Hz with flat very low THD seems unreal when mechanical compliance can't stay linear with the big membrane amplitude movement.

Maybe the microphone pickups rumble (headband, cups, pads vibrations) and add this to fundamental frequency, or some leaks at low frequency that spoil harmonics measurements (?)

 

[DualTriode]

Lower distortion at lower frequency (?) There must be something wrong!
For dynamic driver below resonance the max diaphragm displacement Xmax stays the same (same voltage at almost DC coil R then same B*l*I force). For sinusoidal X velocity is proportional to X*frequency then velocity drops 6 dB/octave at lower frequency, taking acoustic resistance (proportional to f^2) we have the famous drop of 12 dB/octave for semi open hp (or closed box LS). Then at almost constant Xmax the source of distortion (B*l motor and compliance) works at same amplitude, but the fundamental tone in SPL is always decreasing at lower frequency...so THD must always growth at lower frequency (base freq on denominator of THD).
But some THD graphics shows a decreasing value at the lowest frequency even for very high SPL output (HD 650, Focal Clear...) with lower thd at 20 Hz.

For planars (or AMT) it seems suspicious too, to get flat at lower frequency X must grow 4x per lower octave. Flat to 10 or 20 Hz with flat very low THD seems unreal when mechanical compliance can't stay linear with the big membrane amplitude movement.

Maybe the microphone pickups rumble (headband, cups, pads vibrations) and add this to fundamental frequency, or some leaks at low frequency that spoil harmonics measurements (?)

Hello All,

First the the low frequency distortion levels we are talking about my may be low enough as not to matter? Maybe they do?

I do not believe that the microphones, electronics and mathematics are able to differentiate between distortion products and noise. The old school THD distortion method used notch filters to sum the measured levels at the expected harmonic frequencies. Todays methods use FFT's to isolate and sum very narrow frequency bins at the expected harmonic distortion frequencies. The measured stuff between the expected harmonic frequencies is excluded allowing the measured stuff at the expected harmonic frequencies to be totaled up and labeled as THD. At low levels of distortion our tools can not differentiate between distortion and noise. At low levels it all counts.

There is one more thing to complicate things, it is periodic noise. Normally we think of noise as being random and will "fall" with averaging. Sometimes noise is periodic and not random. Fan noise can and does have resonant peaks. Mechanical and acoustic systems sometimes resonate and have frequency peaks.

Our tools will often fail to differentiate between distortion and noise.

Thanks DT

 

[zenki]

Measurement Index for headphones review please...

 

[Dreyfus]

The mic is SP-TFB-2 which I also use for binaural recordings.

For flat plate pressure, I used a springy metal clip (thick) wire wrapped around the sides of the plate to push the headphone towards the plate at the connector. On my head, I don't use anything beyond what the headphone provides, but do position the headphone the best I can and push it against the ear to ensure a better seal.

Thanks.

That's what I use:



Works just like an IEM, so it won't need the huge plastic holder in the upper and lower concha. With its small size it can be placed right inside the entrance of the ear canal.

Here is some HE4XX data:

3 x center position. Then 8 points rotating around the center in 45° steps, 8-10 mm offset from the center.

As expected, the flat plate provides better consistency. It also shows more energy in the sub bass (w/o additional pressure applied) because of the flat mounting surface.

Now both averages with 1/12 smoothing:

Hard to decide where to match the plots since my rather thin head is very susceptible to sealing issues. Hence the roll-off below 100 Hz.

Otherwise as expected. The flat plate drops between 600 Hz and 7 kHz because of the missing pinna flange, meatus and concha resonance.

Technical note:
In Ear Microphone: Primo EM258
Flat Plate Microphone: Beyerdynamic MM1

Both are matched within +/- 1,5 dB between 40 Hz and 14 kHz.

@Dreyfus, since you're very skeptical about averaged measurements & HRTF discrepancy of testing gears heads in general: what do you think about this? It's interesting to notice, that people with molds placed to disrupt the known HRTF can recover pretty quickly into proper localization; I think that's a good way to argue that since we're highly adaptable, the HRTF dilemma isn't that problematic, unless gear masks & throws frequencies away at whole, since then the information is gone, not malformed.

Your brain can adapt to modifications of the pinna, no doubt.
Headphones however do not work like natural sound sources. They excite only a fration of your HRTF and offer no sensory support other than your hearing. Adpation to that is very difficult because of the input (non-natural sound source) being distorted. Do also remember that this distortion changes every time you reposition the headphone on your head.

 

[Robbo99999]

There is no satisfactory answer for to that topic. Less averaging and limitation to a small circle of seating position provides better accuracy. Still, the real life performance with varying head and ear shapes + seating positions can tell a fundamentally different story. That is where averaging comes into play. Averaging however can introduce its own kind of distortion because it simplifies the data set and only looks for mean values, smearing or even cutting away specific driver- or ear-related features. That gets especially tricky one we have multiple distinct heatzones in the Y axis.
Let's assume a set of data gathering a large amount of seating positions shows a clear tendency to resonate either with 98 dB or 92 dB around 8 kHz with just a few points between those extremes. What would be the reasonable mean value for this instance? 95 dB? Probably not.

IMO the healthies approach is to make as many measurements as possible and then generate a heatmap that shows the exact patterns than occur once we put a headphone on a human head or ear simulator and start to move the thing all around the reference point. Single point measurements as much as simplified averages are both imperfect in certain ways.

As for the Harman target rating and equalization though we should only use averaged results that smooth the ear resonance the same way as the target does.

I think the tested headphone positions should reflect the perhaps small differences in mounting that we might see when we put our own headphones on our heads, because you can kind of feel how they feel on your head (you can even take a quick look in the mirror to see if the earcups are too high or too low, but most of the time doing it by feel provides the same position). I think an average of those small differences in positions has got to be the most reflective. We can't do anything about the variation of our own ears to that of the mannequin head.