• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Master Complaint Thread About Headphone Measurements

Mordecai

Member
Joined
Feb 13, 2024
Messages
20
Likes
6
Those folks should do their own studies demonstrating which headphone allows them to make better mix decisions.

Until then, listeners prefer the extra bass because that is what "real" sounds to them. By mixing against that consumer target, then we get standardization of music creation and playback. Using an entirely different response for mixing/mastering music means that we don't hear what they heard. This, by definition is wrong.
Hi, I’m new to the site and I‘m not pro at all on mixing/mastering however I do that bit on a regular basis. I agree that the mix should be to a target. But the tool to get there isn’t exactly the one that gives you that response. for example, a HP with a rolled off bass or flat is much better for mixing in my opinion. That way, you‘d naturally add a bass boost in the mix. Checking that with reference HP that follows the Harman curve would then give a fuller picture. This is how I’ve done it…am I wrong or have I missed something? In my experience, a mixing headphone with a neutral bass and mids with slightly elevated treble however fatiguing that may be is ideal. Using such tool intuitively results to a mix that’s rich in bass and a slightly muted treble. But frankly, any headphone with good resolution, channel balance, noise performance, and above all comfort, would be fine regardless of its frequency response. The key is to calibrate your ears with whatever headphones you prefer. Once you know the quirks of your tool, producing music to a target response can be done.
 

OnLyTNT

Active Member
Joined
Jan 10, 2023
Messages
115
Likes
132
Hi, I’m new to the site and I‘m not pro at all on mixing/mastering however I do that bit on a regular basis. I agree that the mix should be to a target. But the tool to get there isn’t exactly the one that gives you that response. for example, a HP with a rolled off bass or flat is much better for mixing in my opinion. That way, you‘d naturally add a bass boost in the mix. Checking that with reference HP that follows the Harman curve would then give a fuller picture. This is how I’ve done it…am I wrong or have I missed something? In my experience, a mixing headphone with a neutral bass and mids with slightly elevated treble however fatiguing that may be is ideal. Using such tool intuitively results to a mix that’s rich in bass and a slightly muted treble. But frankly, any headphone with good resolution, channel balance, noise performance, and above all comfort, would be fine regardless of its frequency response. The key is to calibrate your ears with whatever headphones you prefer. Once you know the quirks of your tool, producing music to a target response can be done.
I totally agree with you, especially with the last sentences. By the way, not every music is "bass" oriented, that also should be kept in mind, but the main stream music is. One should also check the mix with a headphone which is Harman targeted at the end of the day anyway. So, may be mixing with the target at the first place would be a good idea :).
 

Mordecai

Member
Joined
Feb 13, 2024
Messages
20
Likes
6
I totally agree with you, especially with the last sentences. By the way, not every music is "bass" oriented, that also should be kept in mind, but the main stream music is. One should also check the mix with a headphone which is Harman targeted at the end of the day anyway. So, may be mixing with the target at the first place would be a good idea :).
Yes, I use reference head phones post-mix from the cheap ones to relatively expensive ones. I’d say I never tried those that cost like a small car for referencing bec frankly I don’t see the point to do so. I also try it on mostly consumer-oriented sound systems just to see how it translates for a broader audience.
 

LeShog

Member
Joined
Feb 19, 2024
Messages
79
Likes
52
Going through the headphone reviews rn is a bit of a pain. A headphone ranking or a recommended list would be most appreciated.
 

IAtaman

Major Contributor
Forum Donor
Joined
Mar 29, 2021
Messages
2,409
Likes
4,165
If you search for yes, choose show all and sort by price, you get the following. Latest reviews are not there, but most of them are in.


Headphone Type
Brand
Model
Sensitivity
Price Each USD
Ascending
Recommended
Review Date
Review Link
Cable (IEM)SuperluxHD38730$15.90Yes4/21/22Review Link
On-EarKossPorta Pro102$50.00Yes12/17/21Review Link
In-EarTRUTHEARZero74$50.00Yes9/15/22Review Link
In-EarTRUTHEARCrinacle Zero:RED66$55.00Yes5/20/23Review Link
In-EarKZZSX27$60.00Yes11/9/23Review Link
On-EarSkullcandyGrind Wireless152$89.00Yes2/20/23Review Link
Over Ear1MoreSonoflow NCN/A$99.00Yes3/11/23Review Link
Over-EarAKGK271164$118.00Yes7/5/21Review Link
Over-EarHiFiMANHE400SE328$149.00Yes12/5/21Review Link
Over-EarAKGK37165$150.00Yes1/22/21Review Link
On-EarYamahaRH-5Ma96$158.00Yes7/6/21Review Link
GamingDROP & SENNHEISERPC38X149$169.00Yes6/23/21Review Link
Over EarSennheiserMASSDROP Sennheiser HD58X JUBILEE166$170.00Yes8/3/21Review Link
Over EarHiFiMANHE400i 2020 RevisionN/A$170.00Yes8/16/21Review Link
Over-EarSennheiserHD560S246$170.00Yes1/3/22Review Link
Over-Ear (ANC)StatusFlagship ANC72$179.00Yes7/23/21Review Link
Over-EarPSBM4U 159$230.00Yes3/16/21Review Link
Over-EarSennheiserHD 598 SE97$249.00Yes6/28/21Review Link
Over EarAune AudioAR500059$299.00Yes12/20/23Review Link
Over-Ear (ANC)BoseQuietComfort 35 II135$300.00Yes2/21/21Review Link
Cable (IEM)MOONDROPBlessing : II60$320.00Yes4/6/22Review Link
Over-Ear (ANC)BoseQuietComfort 45N/A$329.00Yes7/18/23Review Link
Over-EarHiFiMANSundara318$349.00Yes4/15/21Review Link
Over-EarVerum AudioVerum 154$349.00Yes3/6/22Review Link
Over EarAudezeLCD-1230$399.00Yes7/29/21Review Link
Over EarOppo DigitalPM3161$399.00Yes7/30/21Review Link
Over-EarSennheiserHD650288$400.00Yes12/25/20Review Link
Over EarSennheiserHD660S176$400.00Yes6/7/22Review Link
Over-Ear (ANC)AppleAirPods Max385$479.00Yes8/8/21Review Link
Over-EarE-MUTeak97$500.00Yes7/8/21Review Link
Over-EarDan Clark Audio (MrSpeakers)AEON RT677$500.00Yes1/14/21Review Link
Over-EarFocalElegia82$543.00Yes11/9/21Review Link
Over-EarDrop & FocalElex180$700.00Yes4/9/21Review Link
Over-EarN/ANan-6950$750.00Yes5/31/22Review Link
Over EarDan Clark Audio (MrSpeakers)AEON 2 Noire353$850.00Yes7/1/23Review Link
Over EarAurorus AudioBorealis87$899.00Yes2/7/23Review Link
Over-EarDrop & MrSpeakersEther CX182$900.00Yes7/2/21Review Link
Over EarAudezeLCD-X Review (2021 Edition Headphone)152$1,199.00Yes7/26/21Review Link
Over-EarHiFiMANArya214$1,229.00Yes11/4/21Review Link
Over-EarHiFiMANHE-61250$1,299.00Yes3/22/21Review Link
Over-EarSennheiserHD800 SN/A$1,700.00Yes12/14/20Review Link
Over-EarAudezeLCD-XC131$1,799.00Yes2/21/22Review Link
Over EarDan Clark Audio (MrSpeakers)Ether Flow 1.1164$1,799.99Yes6/22/23Review Link
Over-EarHiFiMANHE6se1370$1,800.00Yes5/21/21Review Link
Cable (IEM)64 AudioU12t28$1,999.00Yes4/2/22Review Link
Over-EarFocalStellia96$2,990.00Yes11/16/21Review Link
Over-EarKennertonRognir143$3,770.00Yes10/17/21Review Link
Over-EarDan Clark AudioStealth549$3,999.00Yes8/19/21Review Link
Over-EarFocalUtopia149$4,000.00Yes4/3/21Review Link
Over EarDan Clark AudioExpanse665$4,000.00Yes9/12/22Review Link
Over EarStaxSR-009SN/A$4,545.00Yes8/23/23Review Link
 

LeShog

Member
Joined
Feb 19, 2024
Messages
79
Likes
52
If you search for yes, choose show all and sort by price, you get the following. Latest reviews are not there, but most of them are in.


Headphone TypeBrandModelSensitivityPrice Each USD
Ascending
RecommendedReview DateReview Link
Cable (IEM)SuperluxHD38730$15.90Yes4/21/22Review Link
On-EarKossPorta Pro102$50.00Yes12/17/21Review Link
In-EarTRUTHEARZero74$50.00Yes9/15/22Review Link
In-EarTRUTHEARCrinacle Zero:RED66$55.00Yes5/20/23Review Link
In-EarKZZSX27$60.00Yes11/9/23Review Link
On-EarSkullcandyGrind Wireless152$89.00Yes2/20/23Review Link
Over Ear1MoreSonoflow NCN/A$99.00Yes3/11/23Review Link
Over-EarAKGK271164$118.00Yes7/5/21Review Link
Over-EarHiFiMANHE400SE328$149.00Yes12/5/21Review Link
Over-EarAKGK37165$150.00Yes1/22/21Review Link
On-EarYamahaRH-5Ma96$158.00Yes7/6/21Review Link
GamingDROP & SENNHEISERPC38X149$169.00Yes6/23/21Review Link
Over EarSennheiserMASSDROP Sennheiser HD58X JUBILEE166$170.00Yes8/3/21Review Link
Over EarHiFiMANHE400i 2020 RevisionN/A$170.00Yes8/16/21Review Link
Over-EarSennheiserHD560S246$170.00Yes1/3/22Review Link
Over-Ear (ANC)StatusFlagship ANC72$179.00Yes7/23/21Review Link
Over-EarPSBM4U 159$230.00Yes3/16/21Review Link
Over-EarSennheiserHD 598 SE97$249.00Yes6/28/21Review Link
Over EarAune AudioAR500059$299.00Yes12/20/23Review Link
Over-Ear (ANC)BoseQuietComfort 35 II135$300.00Yes2/21/21Review Link
Cable (IEM)MOONDROPBlessing : II60$320.00Yes4/6/22Review Link
Over-Ear (ANC)BoseQuietComfort 45N/A$329.00Yes7/18/23Review Link
Over-EarHiFiMANSundara318$349.00Yes4/15/21Review Link
Over-EarVerum AudioVerum 154$349.00Yes3/6/22Review Link
Over EarAudezeLCD-1230$399.00Yes7/29/21Review Link
Over EarOppo DigitalPM3161$399.00Yes7/30/21Review Link
Over-EarSennheiserHD650288$400.00Yes12/25/20Review Link
Over EarSennheiserHD660S176$400.00Yes6/7/22Review Link
Over-Ear (ANC)AppleAirPods Max385$479.00Yes8/8/21Review Link
Over-EarE-MUTeak97$500.00Yes7/8/21Review Link
Over-EarDan Clark Audio (MrSpeakers)AEON RT677$500.00Yes1/14/21Review Link
Over-EarFocalElegia82$543.00Yes11/9/21Review Link
Over-EarDrop & FocalElex180$700.00Yes4/9/21Review Link
Over-EarN/ANan-6950$750.00Yes5/31/22Review Link
Over EarDan Clark Audio (MrSpeakers)AEON 2 Noire353$850.00Yes7/1/23Review Link
Over EarAurorus AudioBorealis87$899.00Yes2/7/23Review Link
Over-EarDrop & MrSpeakersEther CX182$900.00Yes7/2/21Review Link
Over EarAudezeLCD-X Review (2021 Edition Headphone)152$1,199.00Yes7/26/21Review Link
Over-EarHiFiMANArya214$1,229.00Yes11/4/21Review Link
Over-EarHiFiMANHE-61250$1,299.00Yes3/22/21Review Link
Over-EarSennheiserHD800 SN/A$1,700.00Yes12/14/20Review Link
Over-EarAudezeLCD-XC131$1,799.00Yes2/21/22Review Link
Over EarDan Clark Audio (MrSpeakers)Ether Flow 1.1164$1,799.99Yes6/22/23Review Link
Over-EarHiFiMANHE6se1370$1,800.00Yes5/21/21Review Link
Cable (IEM)64 AudioU12t28$1,999.00Yes4/2/22Review Link
Over-EarFocalStellia96$2,990.00Yes11/16/21Review Link
Over-EarKennertonRognir143$3,770.00Yes10/17/21Review Link
Over-EarDan Clark AudioStealth549$3,999.00Yes8/19/21Review Link
Over-EarFocalUtopia149$4,000.00Yes4/3/21Review Link
Over EarDan Clark AudioExpanse665$4,000.00Yes9/12/22Review Link
Over EarStaxSR-009SN/A$4,545.00Yes8/23/23Review Link
thank you, most appreciated!
 

El_Arte

Member
Joined
Sep 24, 2021
Messages
50
Likes
31
This thread was lively.

Sean Olive and Harman’s approach are founded in the scientific method, there is no doubt about that.

However, I accept their sample sizes could be inadequate to establish universal preferences.
10 to 100 subjects may well be insufficient to draw absolute conclusions about what millions of people prefer.

That being said, no one will ever fund such studies with 1 million subjects. There’s no money in it and it’s not practical.

Does it mean such research has no value?
No, it doesn’t.
Human physiology exists in a relatively narrow bell curve.
Such research and measurements can tell you if something has a chance of being liked by many people; it can also tell you if something is likely to be utterly broken “by design”.

Yet, it will never be hard to find a product with 100,000 5-star reviews on Amazon that doesn’t match Harman’s targets.

So, in conclusion, Harman’s targets may perfectly match your preferences, in which case the graphs Amir gives you for free are gold.
Or, Harman’s targets may sound “not quite right” to you, in which case Amir’s graphs will also be useful.
 
Last edited:

Mr. Haelscheir

Active Member
Joined
Mar 16, 2023
Messages
110
Likes
84
A case for multi-tone and higher-order testing of headphone distortion

This is the latest formulation of what I had already documented in https://www.audiosciencereview.com/...n-susvara-headphone-review.50705/post-1888972 (post #1,183; the old methodology dB calibration in those measurements is possibly high by a few dB compared to the latest calibration methodology).

tl;dr:
  1. Two headphones EQed to the same in-ear frequency response that have very low harmonic distortion in a given frequency band can greatly differ in their multi-tone distortion within that band, therefore it may be of interest to use multi-tone distortion measurements as a better measure of exceptionality.
  2. In a recent measurement of a demo unit, a headphone (name and distortion measurement not to be disclosed unless the other party were to provide their context) that measured reasonably and as expected up to the third harmonic was found to have significant harmonic distortion of the fifth order and higher where the other headphones for the same measurement conditions primarily exhibited second order distortion with the rest being close to or below the noise floor.
This post will show my measurement methodology, present the harmonic distortion measurements, and then the commensurate multi-tone distortion measurements demonstrating why this type of measurement may be of interest to ASR. This post excludes measurements of certain headphones that I cannot disclose without the other party's authorization, yet would have likely been of particular interest to ASR.

Methodology

In-ear microphones, audio interface, and SPL calibration:

https://www.head-fi.org/threads/con...les-mattering-for-audio.970430/#post-17816683 had covered my latest measurement equipment, namely in-ear microphones plugged into RØDE VXLR+ adapters plugged into a MOTU M2 whose DAC's line output is connected via a male dual TRS to male 4.4 mm TRRRS adapter to a FiiO K9 Pro ESS DAC/amp's balanced line input in preamp mode (having the DAC and ADC on the same device facilitates more accurate and consistent phase response measurements and likewise measurement averaging for reducing the effective noise floor), measurements being captured using Room EQ Wizard (REW). The main difference is that I had developed new mounts for my in-ear microphones and a new method of SPL calibration as seen below.

2024-04-11 - Figure 1.jpg


Figure 1:
  • (a): The 4 mm electret microphone capsules used are KEEG1538WB-100LB (https://canada.newark.com/kingstate/keeg1538wb-100lb/microphone-condenser-wire-w-proof/dp/48W5995), supplied by https://www.earfish.eu/ primarily for HRTF measurement. The dust covers were removed since they were bound to be rubbed off upon insertion into the new mounts. The same Radians Custom Molded Earplugs CEP001-R Red putty was used to form a plug to be inserted into the 1/2" to 1" adapter that came with my SPL calibrator.
  • (b): The SPL calibrator used is the Tekcoplus ND9B (SLTK-885B) SPL calibrator. The tolerances are likely not as advertised (no Class 1 certification or calibration report is provided), but should be sufficient for my purposes. I had also purchased the Latnex SM-130DB SPL meter with "certificate" of calibration from facilities in Taiwan only specifying supposed compliance with ISO 9001:2015(CNS12681), the manual specifying IEC 61672-1:2002 class 2, IEC651 Type2, and ANSI1.4 Type2 as references, its measuring within the advertised +-1.8 dB @ 1 kHz spec between 0.4 dB to 0.9 dB too high compared to the ND9B 94 dB reference. In this case, the microphone capsule is mounted to the plug, inserted into the adapter, then inserted into the SPL calibrator which is turned on, SPL calibration being conducted via the instructions in https://www.roomeqwizard.com/help/help_en-GB/html/inputcal.html for "Use an external signal" until both channels register 94 dBC.
  • (c): The in-ear microphone mounts were formed using the Radians Custom Molded Earplugs CEP001-R Red putty (try at your own risk; it is highly recommended to use an otoblock in addition to embedding a looped string for safely removing the plugs once formed; as a general rule of thumb, use no more than an 8 mm to 8.5 mm diameter ball of each reagent per plug and press the material in to be flush with the canal entrance and no deeper). A rotary tool was used to form a mounting channel as well as a small hole on the side through which the microphone capsule can be elastically inserted for measurements and removed for calibration. It was difficult to get a perfect match of shape and depth between the left and right plugs, but this sufficed considering the variable of the physiological differences between the left and right ears and sides of the head. The plugs provide around 15 to 20 dB of attenuation for the test tones, though not all impression attempts had yielded good attenuation or seal.
  • (d): The in-ear microphones are inserted like so and the downward tension on the microphone wires minimized, the headphones then being placed on the head with the intuitive feedback not offered by industry standard heads. The REW "Measure" window (see https://www.roomeqwizard.com/help/help_en-GB/html/makingmeasurements.html) is opened at the default -12 dBFS test signal level, the measurement length set and the measurement rate set from 2 Hz to 96 kHz (purely for interest) with the measurement chain set to a sample rate of 192 kHz (averaged measurements run faster this way), "Check levels" clicked and the MOTU M2 and FiiO K9 Pro ESS volume knobs adjusted until the reference level (e.g. 94 dBC) is displayed, and the sweep then played with a piece of Amazon LENRD-shaped acoustic foam held behind the driver to absorb the backwave and prevent room reflections from interfering with the group delay and CSD measurements. Compared to my previous in-ear microphone mounts in https://www.head-fi.org/threads/mez...eadphone-official-thread.959445/post-17743502 (post #5,152), these provide much better positional consistency of the microphones as seen in Figure 4. They are also substantially more comfortable, and I can remove the plugs in the middle of a measurement session without concerns for measurement consistency.

Magnitude response calibration:

It was deemed uneconomical to invest in an Earthworks M23R measurement microphone and a commensurate Class 2 SPL calibrator. I had originally intended on upgrading my 4 mm electret microphone capsules to the TOM-1537L-HD-LW100-B-R (https://www.digikey.ca/en/products/detail/pui-audio-inc/TOM-1537L-HD-LW100-B-R/12152294; see https://www.digikey.ca/en/products/detail/pui-audio-inc/TOM-1537L-HD-R/7898333 for the datasheet) which had a 68 dB SNR as opposed to the 58 dB SNR of the KEEG1538WB-100LB. Though I did not have a calibrated measurement microphone, I did already have a pair of Genelec 8341A studio monitors. An indoor Genelec GLM calibration was done with the GLM microphone 1 m away on-axis from the tweeter facing away from the wall to the left of my desktop, the microphones being mounted as below.

20240331_234400.jpg


Figure 2: TOM-1537L-HD-LW100-B-R 4 mm electret microphone mounted with lower edge 1 cm above the tip of the GLM calibration microphone 1 m away on-axis from the tweeter. Alligator clips were used before committing to solder more closely matched pairs to adapter cables with colour-coded shrink tubing.

2024-04-01 - Mic comparison - 500 ms window from with 1_12 octave smoothing full scale 1 cm 2.jpg


Figure 3: 1/12 octave smoothing overlay of microphone responses in the configuration seen in Figure 2. As can be seen, these four capsules meant for use with two pairs of mics are impressively linear in "free-field" (in this context meaning without any other housing to interact with). I chose Mics 2 and 4 as my references. Measurements taken with the capsules only 0.5 cm above the GLM microphone's tip showed small variations likely more so related to height within the sound field than to the proximity to said tip. Comparisons with 3.51 ms impulse windowing and the same 1/12 octave smoothing showed similar trends for differences within the treble and top octave.

Unfortunately, when driven by the RØDE VXLR+ adapters which should have been providing plug-in power within spec, the TOM-1537L-HD-LW100-B-R capsules as seen in Figure 4 exhibited high second-order harmonic distortion tracking the measured headphone frequency response in a manner suggesting that the distortion came from the capsules themselves. It did not seem like I could have made a wiring error. As a last resort, the KEEG1538WB-100LB capsules with which I had already in the past measured impressively low distortion levels with for the Meze Elite were removed from their original in-ear mounts and inserted into the same left ear custom mount. 1/3 octave smoothed measurements were taken using TOM-1537L-HD-LW100-B-R Mics 2 and 4 in that same left ear custom mount, these being exported into Excel where they were RMS averaged, 1/3 octave smoothed measurements for the KEEG1538WB-100LB then being taken and exported into Excel where I could make calibration files to be imported into REW. These worked well and exhibited reasonable headphone channel matching for the measurements taken on April 5, though by April 10, the right KEEG1538WB-100LB which was known to have bass and treble roll-off was found to be much less stable than the left capsule.

2024-04-11 - Meze Elite L9p M4 1.jpg


Figure 4: Here is the unaveraged 4M length counterpart to Figure 9 where the TOM-1537L-HD-LW100-B-R capsule was used with the same VXLR+ adapter. This is a markedly different result from my measurements that use my older KEEG1538WB-100LB capsules, there only being second-order harmonic distortion above the noise floor that tracks the magnitude response and is vertically stretched perhaps due to nonlinear increase in distortion with respect to level.

Finally, here is a showcase the magnitude response measurement consistency for the left driver of my Meze Elite with "V3.1 PEQ"; see Figure 13 in https://www.head-fi.org/threads/rec...-virtualization.890719/page-121#post-18027627 (post #1,812) for how this EQ profile compares to my threshold of hearing EQ compensated left ear 30-degree speaker HRTF.

2024-04-11 - April 5 and 10 Meze Elite measurement consistency.jpg


Figure 5: Red trace: 4M length measurement taken on April 5 immediately after creating the calibration file. Cyan trace: 4M length measurement taken on April 10. Blue trace: 512k length measurement with 1/48 octave smoothing taken later on April 10 amid quicker measurements comparing FR variations with pad position, this measurement being representative of returning the cups to my preferred centered seating used for the previous two measurements.

Observations

Harmonic distortion:

In the below "4M8R" means that the measurement length in REW was "4M" and it was averaged across 8 repetitions to minimize the effective noise floor.

For all of these measurements, resting a hand on a grounded chassis like the MOTU M2's or FiiO K9 Pro ESS's attenuated a 60 Hz mains tone and its harmonics. Per https://audiosciencereview.com/foru...hones-with-motu-m2-and-rew.49384/post-1783062 (post #6), use of the RØDE VXLR+ incurs upwards of 10 dB more noise centered around 900 Hz than the RØDE VXLR Pro which uses an internal transformer to convert the unbalanced plug-in power signal to a balanced at the expense of greater third-order harmonics below 1.6 kHz. As such, the second-order harmonic distortion between 160 Hz and 600 Hz is more likely to be noise limited than the frequencies above it.

The demo unit measurements were taken at the audio shop in the dedicated listening room with sliding glass door, my SPL meter registering an ambient noise of around 45 dBA. These measurements were taken with the MOTU M2 USB connection plugged into my laptop with charger connected and the FiiO K9 Pro ESS plugged directly into the mains socket. All these demo units were connected to the DAC/amp with their stock unbalanced 1/4" TRS cable due to balanced cables having been unavailable.

2024-04-11 - Meze Elite L 4 4M8R - distortion - demo unit.jpg


Figure 6: Meze Elite demo unit with "V3.1 PEQ" left driver harmonic distortion at 94 dBC, 4M8R. An exceptional result comparable to my at-home result seen in Figure 7 and within Meze's spec, but a bit more noise limited in the bass up to 400 Hz.

The measurements below were taken at home with my own headphones, my SPL meter having registered an ambient noise of around 31 dBA. These measurements were taken with the MOTU M2 USB connection plugged into my desktop and the FiiO K9 Pro ESS plugged directly into the mains socket, its USB connection having still been connected to my desktop for use in regular listening.

2024-04-11 - Meze Elite L 7 4M8R - distortion.jpg


Figure 7: Meze Elite Tungsten with "V3.1 PEQ" left driver harmonic distortion at 94 dBC, 4M8R. Mainly cleaner or less noise limited below 400 Hz, likewise for the 100 dBC measurements.

2024-04-11 - ATH-M50xBT L 2 4M8R - distortion.jpg


Figure 8: Audio-Technica ATH-M50xBT with "V3.1 PEQ" (effectively matching the magnitude response of Figure 6) left driver harmonic distortion at 94 dBC, 4M8R. With EQ, the distortion performance above 140 Hz competes with the EQed Meze Elite.

2024-04-11 - HE1000se NS L 2 4M8R - distortion.jpg


Figure 9: HiFiMan HE1000se left driver harmonic distortion at 94 dBC, 4M8R. Only competitive in the upper bass and midrange. This is the third out of three HE1000se units I had encountered, the second due to a mistake on my part and the third due to the second unit having had clear Q/C issues with its distortion per https://www.head-fi.org/threads/totl-disappointments.925164/page-63#post-17949397 (post #936), whereby the first unit may have measured a bit better in the treble.

Multi-tone distortion:

Most interesting are the multi-tone distortion measurements which reveal cases where headphones with comparable harmonic distortion levels above 200 Hz differ in multi-tone distortion performance, in this case, the EQed Meze Elite coming out on top. Here, multi-tone distortion test signals provide an approximation of the busiest sections of music which can in some cases indeed be found through FFTs to comprise a superposition of tones following an approximately pink spectrum like below:

2024-03-11_00-03-53 - Mahler 5 first big tutti.png

Figure 10: FFT spectrum of the first big tutti of the opening of Boulez' recording of Mahler Symphony No. 5. I've found most orchestral music to technically rather follow a brown/red spectrum envelope unless substantial brass was involved.

These measurements are taken with REW's RTA Window (https://www.roomeqwizard.com/help/help_en-GB/html/spectrum.html), signal Signal Generator (https://www.roomeqwizard.com/help/help_en-GB/html/siggen.html), and SPL meter (https://www.roomeqwizard.com/help/help_en-GB/html/splmeter.html). The RTA window is set to a 4M FFT length with a rectangular window and "Exponential 0.50" averaging, my typically managing to show the result after only one or two averaging iterations.

Multi-tone distortion (of various causes) would show up as spuriae above the noise floor at the base of or between the main tones of the signal.

Here are TRS to XLR loop-back measurements from my MOTU M2's balanced line out to its left XLR input (I admit that an RCA to TRS adapter for testing single-ended ADC performance may have been a better test if I had such an adapter) with the preamp gain set to 12 o'clock as in the original measurements. The SPL scale was as calibrated with my SPL calibrator. The below results were only possible for the 4M FFT length and rectangular window.

2024-04-10 - MOTU M2 XLR loopback regular preamp matched volume for -10 dBFS - noise floor shown.jpg


Figure 11: -10 dBFS 1/10 octave pink spectrum multitone volume-matched to the levels seen in my headphone measurements; i.e. the ADC electronics see tones at the same scale as when receiving those signals from my microphones picking up the headphone signals. Spuriae are around 95 dB down at 1 kHz.

2024-04-10 - MOTU M2 XLR loopback nominal preamp max volume -10 dBFS multitone.jpg


Figure 12: -10 dBFS 1/10 octave pink spectrum multitone with max volume. Spuriae are around 90 dBFS down at 1 kHz, so still well below the distortions measured out of my headphones.

2024-04-10 - Meze Elite L RTA noise floor.jpg


Figure 13: At-home RTA noise floor for the left in-ear microphone with the Meze Elite worn; at least in this measurement, the 60 Hz mains tone was hard to suppress. The noise floor with the HE1000se was cleaner in the bass. I cannot share the at-shop noise floor measurement insofar as it shows the frequency response of one of the headphones that cannot be disclosed. Otherwise, the noise floor can be inferred from the graphs.

At-shop demo unit measurement:

2024-04-05 - Meze Elite L 1_10 decade pink spectrum multitone 94 dBA.jpg


Figure 14: Meze Elite demo unit with "V3.1 PEQ" left driver 94 dBA 1/10 decade pink spectrum multi-tone. Quite low, especially in the bass, the midrange distortion lobes being upwards of 56 dB down, or 0.16%, probably well below audible. I also have 99.8 dBA (approximating some of the loudest orchestral tutties like with the opening of Mahler Symphony No. 5) 1/24 octave pink spectrum multi-tone measurements which mainly incurs a denser distortion floor that is likewise raised by around 6 dB relative to the tones. The lower midrange multi-tone distortion happened to be worse on this unit than on my own unit at home as seen in Figure 20.

At-home personal unit measurements; my laptop happened to have exported the images smaller than on my desktop, my having not remembered the exact size number while at the shop:

2024-04-10 - Meze Elite L 1_10 decade pink spectrum multitone 94 dBA 2.jpg


Figure 15: Meze Elite Tungsten with "V3.1 PEQ" left driver 94 dBA 1/10 decade pink spectrum multi-tone. Cleaner in the lower midrange.

2024-04-10 - ATH-M50xBT EQ L 1_10 decade pink spectrum multitone 94 dBA.jpg


Figure 16: Audio-Technica ATH-M50xBT with "V3.1 PEQ" left driver 94 dBA 1/10 decade pink spectrum multi-tone.

2024-04-10 - HE1000se L 1_10 decade pink spectrum multitone 94 dBA.jpg


Figure 17: HiFiMan HE1000se left driver 94 dBA 1/10 decade pink spectrum multi-tone. Better than the ATH-M50xBT in some places, the worst in the treble. The bass noise floor is lower thanks to the pads contacting the skin further from my ears, incurring less heartbeat noise.

To check on the possible influence of higher bass distortion on the increased full-range multi-tone distortion measurements, here are measurements with the tones below 100 Hz truncated and the rest left at the same level, inherently reducing the test signal's amplitude:

2024-04-10 - Meze Elite L 1_10 decade pink spectrum multitone 94 dBA - start at 100 Hz.jpg


Figure 18: Meze Elite Tungsten with "V3.1 PEQ" left driver 94 dBA 1/10 decade pink spectrum multi-tone, tones below 100 Hz truncated. Quite clean. The multi-tone distortion is yet further improved, but the amplitude of the signal was likely also reduced. Such a spectral profile is probably also quite unlikely to be seen within music.

2024-04-10 - ATH-M50xBT EQ L 1_10 decade pink spectrum multitone 94 dBA - start at 100 Hz.jpg


Figure 19: Audio-Technica ATH-M50xBT with "V3.1 PEQ" left driver 94 dBA 1/10 decade pink spectrum multi-tone, tones below 100 Hz truncated. Upper bass and lower midrange harmonic distortion is still rather high despite the same single-tone sweep harmonic distortion in that range having been more competitive.

2024-04-10 - HE1000se L 1_10 decade pink spectrum multitone 94 dBA - start at 100 Hz.jpg


Figure 20: HiFiMan HE1000se left driver 94 dBA 1/10 decade pink spectrum multi-tone, tones below 100 Hz truncated. Here, the treble distortion was still salient. Though the single-tone sweep harmonic distortion between 100 Hz to 500 Hz was comparable to the EQed ATH-M50xBT, the HE1000se has notably better multi-tone distortion in that range.

Conclusion

Though different headphones may show comparable single-tone sweep harmonic distortion performance within a given frequency band, they can still differ significantly in their multi-tone distortion performance for those bands. As such, multi-tone distortion measurements may provide another meaningful and more practical (insofar as most music comprises a substantial superposition of tones) means for assessing the comparative performance between headphones. Sure, maybe even my HE1000se's multi-tone distortion may not be audible in practical listening, but multi-tone spuriae close to the noise floor would be quite a badge of exceptionality giving consumers full confidence in the inaudibility of distortion artifacts through the given headphone. Likewise, where manufacturers may be content with achievement under the harmonic distortion metric, others might be found to still be quite competitive in the multi-tone distortion metric, encouraging further technological competition and progress.
  • Again, this post excluded measurements of certain headphones that would have provided a more compelling case, but these cannot be shared unless the other party were to share their context or match the pink spectrum multi-tone and the levels thereof and convincingly demonstrate a discrepancy from my measurements.
Likewise, under the case that the measurement system is capable of showing higher-order harmonic distortion, these had ought to be checked at least up to the fifth harmonic to reveal any surprises or evince concerning issues with a new design choice.

For consideration

Should such multi-tone distortion measurements be adopted, it would be up to debate what parameters should be used. As seen here, I had chosen a pink spectrum as a practical average for music. A rectangular window should have a sufficiently high dynamic range for transducer measurements, and should supply the highest frequency resolution and floor particularly in the bass, especially when using up to a 4M FFT length provided that DAC to ADC sample synchronization is ensured. 1/24 octave measurements as in https://www.audiosciencereview.com/...n-susvara-headphone-review.50705/post-1888972 (post #1,183) are a reasonable extreme, but obscure resolution in the bass which is better seen with 1/20 or 1/10 decade multi-tone. Another multi-tone signal may be decided upon. For volume normalization, I chose 94 dBA for 1/10 decade pink spectrum multi-tone and 100 dBA for 1/24 octave pink spectrum multi-tone as approximating the perceived loudness of some of the loudest live orchestral tutties; this or whether to instead use dBC or a single tone may also be deliberated. Consideration may also be given to normalizing headphones to a reference EQ profile for a fairer comparison of driver distortion capability in yielding the same tonality.
 

Attachments

  • 2024-04-10 - MOTU M2 XLR loopback regular preamp full volume -40 dBFS.jpg
    2024-04-10 - MOTU M2 XLR loopback regular preamp full volume -40 dBFS.jpg
    412.5 KB · Views: 5
  • 2024-04-10 - ATH-M50xBT EQ L 1_10 decade pink spectrum multitone 94 dBA.jpg
    2024-04-10 - ATH-M50xBT EQ L 1_10 decade pink spectrum multitone 94 dBA.jpg
    438.8 KB · Views: 6
Last edited:
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,678
Likes
241,088
Location
Seattle Area
Thanks. There is information overload there. :)

Having done multitone tests, the issue I find with it, which I also see in your measurements, is that it is impacted by the frequency response of the headphone. The level of sidebands is directly proportional to that. So unless the headphones being compared have identical frequency response (which never happens), you can't rely on sideband levels to assess level of multitone distortion. With electronics, they have ruler flat frequency response so that is not an issue. But with speakers and headphones, it becomes a problem.

Same issue exists for absolute THD measurements but the analyzer is able to show a ratio which eliminates that effect. Unless you can do that same for Multitone, you won't have this option to get normalized results you can compare.
 

Mr. Haelscheir

Active Member
Joined
Mar 16, 2023
Messages
110
Likes
84
Thanks. There is information overload there. :)

Having done multitone tests, the issue I find with it, which I also see in your measurements, is that it is impacted by the frequency response of the headphone. The level of sidebands is directly proportional to that. So unless the headphones being compared have identical frequency response (which never happens), you can't rely on sideband levels to assess level of multitone distortion. With electronics, they have ruler flat frequency response so that is not an issue. But with speakers and headphones, it becomes a problem.

Same issue exists for absolute THD measurements but the analyzer is able to show a ratio which eliminates that effect. Unless you can do that same for Multitone, you won't have this option to get normalized results you can compare.
Figures 7:8, 15:16, and 18:19 show such a magnitude response normalization being done to the best of my ability. Otherwise, it is understandable if this would be too time consuming to do for every headphone measurement, and one would have to choose what to EQ toward. Indeed, that same proportionality to magnitude response also afflicts harmonic distortion sweeps, but I suppose the point there is to show the distortion performance amid the stock listening conditions. I would also suppose that the levels of the bass relative to the rest would have the largest influence on the overall multi-tone signal's amplitude and hence the multi-tone distortion of the other frequencies. Maybe it would suffice to systematically find a most likely perceptual volume match estimating how most users would be likely to volume-normalize the given headphones when A/Bing, thus comparing the multi-tone distortion under those conditions.
 
Last edited:

Robbo99999

Master Contributor
Forum Donor
Joined
Jan 23, 2020
Messages
6,996
Likes
6,866
Location
UK
Thanks. There is information overload there. :)

Having done multitone tests, the issue I find with it, which I also see in your measurements, is that it is impacted by the frequency response of the headphone. The level of sidebands is directly proportional to that. So unless the headphones being compared have identical frequency response (which never happens), you can't rely on sideband levels to assess level of multitone distortion. With electronics, they have ruler flat frequency response so that is not an issue. But with speakers and headphones, it becomes a problem.

Same issue exists for absolute THD measurements but the analyzer is able to show a ratio which eliminates that effect. Unless you can do that same for Multitone, you won't have this option to get normalized results you can compare.
How about using detailed EQ to Harman Curve to serve as a basis for this test to ensure that headphones have identical frequency response? I'm aware though that some headphones have sharp troughs or sharp peaks that can't be fully rectified, but you can get them closer to the target curve with EQ, and in testing would it be enough to be able to flexibly move the tested frequencies slightly so that it didn't correspond with an unEQ'able dip or peak? It would be a lot more work in a review though, because accurate computerised EQ (not eyeballed) would ideally require multiple measurements and then an average to base the EQ on......unless thinking about it you could do it based on one measurement and if you were to leave the headphone on the rig without moving it both before & after EQ then you could guarantee the EQ'd frequency response was spot on for your IMD test as you're completely removing the element of placement variation (because you haven't moved the headphone & it's still on the rig in the same position both through initial measurement and also through subsequent EQ and IMD measurement).

Additional thought: Sure the EQ'ing to Harman Curve will penalise headphones that are a long way off Harman as big boosts could be required which could worsen IMD vs stock, but a lot of us EQ to Harman anyway, so it's a real world test in a way, and of course the Harman Curve is the standard we measure against currently.....so I think there's relevance there.
 
Last edited:

Mr. Haelscheir

Active Member
Joined
Mar 16, 2023
Messages
110
Likes
84
How about using detailed EQ to Harman Curve to serve as a basis for this test to ensure that headphones have identical frequency response? I'm aware though that some headphones have sharp troughs or sharp peaks that can't be fully rectified, but you can get them closer to the target curve with EQ, and in testing would it be enough to be able to flexibly move the tested frequencies slightly so that it didn't correspond with an unEQ'able dip or peak? It would be a lot more work in a review though, because accurate computerised EQ (not eyeballed) would ideally require multiple measurements and then an average to base the EQ on......unless thinking about it you could do it based on one measurement and if you were to leave the headphone on the rig without moving it both before & after EQ then you could guarantee the EQ'd frequency response was spot on for your IMD test as you're completely removing the element of placement variation (because you haven't moved the headphone & it's still on the rig in the same position both through initial measurement and also through subsequent EQ and IMD measurement).

Additional thought: Sure the EQ'ing to Harman Curve will penalise headphones that are a long way off Harman as big boosts could be required which could worsen IMD vs stock, but a lot of us EQ to Harman anyway, so it's a real world test in a way, and of course the Harman Curve is the standard we measure against currently.....so I think there's relevance there.
Amusingly, the Meze Elite I EQed and measured technically demonstrated the complete opposite of being "penalized" by its heavy diversion from Harman, whereby I suppose with its relaxed ear gain having already been so low distortion, the elevated bass and lower midrange which already exhibited very low distortion only did much better for the rest of the transducer when relaxed relative to the ear gain, possibly reducing its excursion and incurring those exceptional multi-tone results which have surprisingly been the best I have measured even relative to the undisclosed headphones.

Here it is without EQ (at-shop demo unit measurement; I usually measure this headphone at home with my EQ on):

2024-04-23 - Meze Elite L - magnitude and phase - demo unit.jpg


I will say I have indeed had a chance to see what Harman or diffuse-field tuning looks like through my in-ear mics, which would basically be like my V3.1 PEQ measurements (Figure 7 in my original post), but with an around +5 dB bass shelf below 130 Hz and an around +5 dB elevation between 3 kHz and 8 kHz, not to the level of the HE1000se.

I otherwise suspect that unless the headphone already has high upper midrange or treble distortion, it is more likely for headphones requiring a boost in the bass to be "penalized" in being driven to greater excursions than they were nominally designed for.

As for EQ automation, I've found that Room EQ Wizard struggles with large bass or midrange shelves which are best done manually, plus you would have to go through the trouble of exporting to and importing from spreadsheets to obtain a compensation file that would need to be EQed to full-range flat. At least the filters can be copied into Equalizer APO with ease, but if including fine-tuning across multiple measurements, this approach could take between 30 minutes to an hour per channel and is thus of course not ideal. But that was also subject to the movements of my head, so maybe a quicker one-shot autoEQ is feasible. But that is specifically when wanting to use multiple parametric filters, whereby I suppose generating a detailed graphic EQ would be a lot quicker.
 
Last edited:

Robbo99999

Master Contributor
Forum Donor
Joined
Jan 23, 2020
Messages
6,996
Likes
6,866
Location
UK
Amusingly, the Meze Elite I EQed and measured technically demonstrated the complete opposite of being "penalized" by its heavy diversion from Harman, whereby I suppose with its relaxed ear gain having already been so low distortion, the elevated bass and lower midrange which already exhibited very low distortion only did much better for the rest of the transducer when relaxed relative to the ear gain, possibly reducing its excursion and incurring those exceptional multi-tone results which have surprisingly been the best I have measured even relative to the undisclosed headphones.

Here it is without EQ (at-shop demo unit measurement; I usually measure this headphone at home with my EQ on):

View attachment 365441

I will say I have indeed had a chance to see what Harman or diffuse-field tuning looks like through my in-ear mics, which would basically be like my V3.1 PEQ measurements (Figure 7 in my original post), but with an around +5 dB bass shelf below 130 Hz and an around +5 dB elevation between 3 kHz and 8 kHz, not to the level of the HE1000se.

I otherwise suspect that unless the headphone already has high upper midrange or treble distortion, it is more likely for headphones requiring a boost in the bass to be "penalized" in being driven to greater excursions than they were nominally designed for.

As for EQ automation, I've found that Room EQ Wizard struggles with large bass or midrange shelves which are best done manually, plus you would have to go through the trouble of exporting to and importing from spreadsheets to obtain a compensation file that would need to be EQed to full-range flat. At least the filters can be copied into Equalizer APO with ease, but if including fine-tuning across multiple measurements, this approach could take between 30 minutes to an hour per channel and is thus of course not ideal. But that was also subject to the movements of my head, so maybe a quicker one-shot autoEQ is feasible. But that is specifically when wanting to use multiple parametric filters, whereby I suppose generating a detailed graphic EQ would be a lot quicker.
I too would expect large bass boosts to be the main driving cause of IMD.

Just on your last paragraph where you mention multiple measurements, I did outline a way you could do the EQ and IMD testing all from just one initial headphone measurement without the need to replace the headphone and do multiple measurements - ie a way to do it that doesn't require averaging multiple measurements before the EQ stage (so is less work). It's probably better that way because you're taking the element of placement variation out of the equation. Amir would measure the headphone as he does currently until he gets what he thinks is a representative and sensible measurement (ie the one he displays in his reviews currently), he would then leave the headphone on the rig in that same position, create the EQ to Harman & apply it to the headphone, measure IMD, job done (the headphone would not be moved from it's initial measurement position during that entire process).
 
Top Bottom