Mad_Economist Canjam Presentation - These are the Dark Ages

[Jeromeof]

Here is Blaine's i.e. the @Mad_Economist presentation on the problems with current measurements from Canjam. Very interesting as ever:

 

[IngMishael]

Yeah I was at the presentation... Really thought provoking stuff and hope this video reaches a lot of people

 

[Mad_Economist]

Thanks for posting this! My seminar slides could...use work, to say the least, but I'm glad y'all enjoyed it.

I'm happy to field questions/comments/angry mobs here if folks have thoughts on this!

 

[Curupira]

Thanks for posting this! My seminar slides could...use work, to say the least, but I'm glad y'all enjoyed it.

I'm happy to field questions/comments/angry mobs here if folks have thoughts on this!

Prepare for the zealots, you heathen.

 

[Somafunk]

Thanks for posting this! My seminar slides could...use work, to say the least, but I'm glad y'all enjoyed it.

I'm happy to field questions/comments/angry mobs here if folks have thoughts on this!

Must be daunting to present such a seminar with an actual “oratory” in the audience throwing quips at you , well done.

That headphone with the multiple seating variations that you comment on and show the slides for, is this due to pad structure, position of the driver in relation to the pinna or just pot luck in getting a good seal?

 

[Mad_Economist]

Must be daunting to present such a seminar with an actual “oratory” in the audience throwing quips at you , well done.

That headphone with the multiple seating variations that you comment on and show the slides for, is this due to pad structure, position of the driver in relation to the pinna or just pot luck in getting a good seal?

Honestly, that's my daily life when Konstantin and I are in the same room. I'll get him back at his next presentation.

The two headphones (HP2 and HP6, the DCA Stealth and DCA Aeon Closed) which had high positional variation at both low and high frequencies in my test set did not massively vary in the band between 500 and 1500hz. I believe that this implies there are two separate mechanisms in play: leakage sensitivity/high acoustic source Z at low frequency, but for high-frequency I'm not sure, I did not disassemble the headphones or even remove their pads, so I cannot be certain of causation.

I will say that their variation at high frequencies was high relative to other planar designs with meaningfully larger earpads, where I was able to (and did) move the headphones more during placement. This may reflect a non-uniformity of distribution of sound pressure in the earpad, but I didn't test that. I am highly confident that it isn't just seal, however.

Edit: Since I'm not going to get yelled at in person now, I can identify all the headphones from my slides:
HP1-6 were from my own test, and do not relate to numbering in anyone else's work.

• HP1 = Sennheiser HD600
• HP2 = DCA Stealth
• HP3 = Hifiman HE560 (though it's a V1)
• HP4 = Hifiman Arya
• HP5 = Focal Elear
• HP6 = DCA Aeon

For the three data series taken from "A Statistical Model That Predicts Listeners' Preferences for Around-Ear and On-Ear Headphones", HP14 = HD600, HP20 = HD800, HP26 = DT990

 

[DEF]

Honestly, that's my daily life when Konstantin and I are in the same room. I'll get him back at his next presentation.

The two headphones (HP2 and HP6, the DCA Stealth and DCA Aeon Closed) which had high positional variation at both low and high frequencies in my test set did not massively vary in the band between 500 and 1500hz. I believe that this implies there are two separate mechanisms in play: leakage sensitivity/high acoustic source Z at low frequency, but for high-frequency I'm not sure, I did not disassemble the headphones or even remove their pads, so I cannot be certain of causation.

I will say that their variation at high frequencies was high relative to other planar designs with meaningfully larger earpads, where I was able to (and did) move the headphones more during placement. This may reflect a non-uniformity of distribution of sound pressure in the earpad, but I didn't test that. I am highly confident that it isn't just seal, however.

Edit: Since I'm not going to get yelled at in person now, I can identify all the headphones from my slides:
HP1-6 were from my own test, and do not relate to numbering in anyone else's work.

• HP1 = Sennheiser HD600
• HP2 = DCA Stealth
• HP3 = Hifiman HE560 (though it's a V1)
• HP4 = Hifiman Arya
• HP5 = Focal Elear
• HP6 = DCA Aeon

For the three data series taken from "A Statistical Model That Predicts Listeners' Preferences for Around-Ear and On-Ear Headphones", HP14 = HD600, HP20 = HD800, HP26 = DT990

I use DT 770 Pro and used DT 990 Pro. When I tested the highly praised Sennheiser HD800 S, it was the absolute worst for my ears.
I ended up with an EQ people would find absolutely wild, too. But this confirms the HRTF variance being such a huge factor that, even if it had measured great, it would be really abysmal on some heads. Especially considering that volume makes a big difference, too.

Currently, the best I owned was the Focal Utopia, but I didn't keep it because at the time I needed the money and because it slipped from my head way too easily.

 

[Somafunk]

I will say that their variation at high frequencies was high relative to other planar designs with meaningfully larger earpads, where I was able to (and did) move the headphones more during placement. This may reflect a non-uniformity of distribution of sound pressure in the earpad, but I didn't test that. I am highly confident that it isn't just seal, however.

Edit: Since I'm not going to get yelled at in person now, I can identify all the headphones from my slides:
HP1-6 were from my own test, and do not relate to numbering in anyone else's work.

Now you’ve named the headphones I’m going to rewatch and pay more attention

The only eggy shaped headphone I’ve owned was the Edition XS, well two of them actually as the first one arrived with a very noticeable imbalance so boxed up and sent back within the hour and the replacement pair developed a crinkly but also raspy sound after a 30hrs approx of use at home so sent back and I didn’t bother trying for a third time lucky.

When they were working and I was sat extremely still they were fine until the headband pressure got too much, but the slightest movement of my medium-large (according to bike helmet sizing) shaped shaved head and the bass would drop right off, the cups seemed to come too far down past my ears despite having the headband adjusted correctly and even taking a sip of coffee would bugger up the sound as I swallowed or moved my jaw.

So I’ve kinda gave up on eggy headphones after my conclusive and exhaustive study , I bought HD 490’s earlier this year on an impulse and whilst not going to set the world on fire are excellent for comfort being so light and accommodating on my head, The fit is the same each time and the sound is perfectly satisfactory for my liking.

I did buy Grell OAE1 on another impulse/curiosity purchase, they were listed and sold on headfi after 24hrs.

I really should’ve listened to the reports/paid attention to the facial contortions as they were demo’d on video.

 

[Mad_Economist]

I use DT 770 Pro and used DT 990 Pro. When I tested the highly praised Sennheiser HD800 S, it was the absolute worst for my ears.
I ended up with an EQ people would find absolutely wild, too. But this confirms the HRTF variance being such a huge factor that, even if it had measured great, it would be really abysmal on some heads. Especially considering that volume makes a big difference, too.

Currently, the best I owned was the Focal Utopia, but I didn't keep it because at the time I needed the money and because it slipped from my head way too easily.

Well, maybe. The thing there is that this isn't a blind test, this is the definition of sighted, and so there are potentially sighted biases here. Just because there's a mechanism (or multiple mechanisms) which could potentially explain varying perception, that doesn't mean that they are the source of any given variation in perception, and we have very strong evidence for the effects of psychology on these things.

 

[Mad_Economist]

Now you’ve named the headphones I’m going to rewatch and pay more attention

The only eggy shaped headphone I’ve owned was the Edition XS, well two of them actually as the first one arrived with a very noticeable imbalance so boxed up and sent back within the hour and the replacement pair developed a crinkly but also raspy sound after a 30hrs approx of use at home so sent back and I didn’t bother trying for a third time lucky.

When they were working and I was sat extremely still they were fine until the headband pressure got too much, but the slightest movement of my medium-large (according to bike helmet sizing) shaped shaved head and the bass would drop right off, the cups seemed to come too far down past my ears despite having the headband adjusted correctly and even taking a sip of coffee would bugger up the sound as I swallowed or moved my jaw.

So I’ve kinda gave up on eggy headphones after my conclusive and exhaustive study , I bought HD 490’s earlier this year on an impulse and whilst not going to set the world on fire are excellent for comfort being so light and accommodating on my head, The fit is the same each time and the sound is perfectly satisfactory for my liking.

I did buy Grell OAE1 on another impulse/curiosity purchase, they were listed and sold on headfi after 24hrs.

I really should’ve listened to the reports/paid attention to the facial contortions as they were demo’d on video.

Having weird sounds + having the bass drop without seal on an eggyman tends to imply that the drivers are in some respect damaged. This is, sadly, not at all uncommon. If Hifiman's QC and general construction were better, they'd make some of the best stuff on the market. Like, it's an eggyman that I literally held the pad half off the head and got this plot (pink) whoops, my memory is failing me, I can do some measurements of the impact of seal variation on HP4 once DMS sends my pinnae back though.

But I've also had many a hifiman whose diaphragm was fenestrated or glue to its own magnets, and those, unsurprisingly, did not behave the same here.

HD490 is on my to-try list, I love the pitch of a not-insanely-priced, ergonomic, decent-sounding headphone.

 

[DEF]

Well, maybe. The thing there is that this isn't a blind test, this is the definition of sighted, and so there are potentially sighted biases here. Just because there's a mechanism (or multiple mechanisms) which could potentially explain varying perception, that doesn't mean that they are the source of any given variation in perception, and we have very strong evidence for the effects of psychology on these things.

How do you characterize yourself to match with a certain model?
Head Shape?
Pinna Shape?
Ear canal diameter?
Even your Bones make a difference.

 

[amirm]

I don't know who this is "dark ages" for. It certainly doesn't represent my view of testing I do. We have a nicely lit road as far as I am concerned.

Sure, folks are making a mess out there with creating this and that target curve, on this and that fixture. If you are one looking for consensus between measurements, you won't get it for that reason.

This problem ultimately has no absolute solution because the music production has no standards. I have test clips which I am confident would invalidate some of the research. My sub-bass tracks for example, easily prove that distortions are audible and can override importance of frequency response. Have a headphone click and fall apart and listeners will absolutely knock it down regardless of frequency response. As a general rule, such tracks are under-represented even for frequency response tests.

With speakers, we have some of the above issues but I see no one complaining about dark ages there. Measurements help us get 2/3 of way there which is massively more than nothing. Some of the rest can be had with disciplined, subjective evaluation with proper protocol. Ultimately every playback system needs to have EQ and an enthusiasts needs to learn to use it to create tonality they want. Again, just like we have with speakers.

With speakers, we are there as far as desirability of flat anechoic response. We need the same objective graph and measurement setup for headphones and let the rest be solved per above. Having more than one answer to the same question is going to create the wild west where everyone does what they want.

 

[Old_School_Brad]

We’re not in the "dark ages", but I think we’re still in the early stages with headphones, and that’s where improvement is needed. More refined, objective measurement standards could help guide better subjective assessments and reduce the "wild west" approach.
I don't think the measurements are misunderstood, but rather that, due to the lack of a clear reference or standard, people end up creating their own interpretations.

 

[amirm]

We’re not in the "dark ages", but I think we’re still in the early stages with headphones, and that’s where improvement is needed.

Harman has moved on for the most part. As they did with speakers. I don't expect much more to come.

 

[Mad_Economist]

I don't know who this is "dark ages" for. It certainly doesn't represent my view of testing I do. We have a nicely lit road as far as I am concerned.

Sure, folks are making a mess out there with creating this and that target curve, on this and that fixture. If you are one looking for consensus between measurements, you won't get it for that reason.

This problem ultimately has no absolute solution because the music production has no standards. I have test clips which I am confident would invalidate some of the research. My sub-bass tracks for example, easily prove that distortions are audible and can override importance of frequency response. Have a headphone click and fall apart and listeners will absolutely knock it down regardless of frequency response. As a general rule, such tracks are under-represented even for frequency response tests.

With speakers, we have some of the above issues but I see no one complaining about dark ages there. Measurements help us get 2/3 of way there which is massively more than nothing. Some of the rest can be had with disciplined, subjective evaluation with proper protocol. Ultimately every playback system needs to have EQ and an enthusiasts needs to learn to use it to create tonality they want. Again, just like we have with speakers.

With speakers, we are there as far as desirability of flat anechoic response. We need the same objective graph and measurement setup for headphones and let the rest be solved per above. Having more than one answer to the same question is going to create the wild west where everyone does what they want.

I take it you haven't watched the linked presentation - if you do, I'd be interested in your thoughts on the issues raised (primarily after 40 minutes).

Flat anechoic response is, of course, less than half of what we need in speakers - this was a design goal for many decades of quite bad speakers before @Floyd Toole and @Sean Olive's work conclusively indicated the importance of directivity. Sadly, at present, we lack something analogous to DI in predicting the consistency of perceived headphone response, much as an axially flat loudspeaker with uneven DI may be quite inconsistent in the impressions of listeners depending on their rooms.

 

[Mad_Economist]

Harman has moved on for the most part. As they did with speakers. I don't expect much more to come.

Something perhaps of some interest to note is that while Harman as an organization has moved on, Sean remains interested in exploring some of these questions. I'm very appreciative of his (slightly under the table) ongoing research.

 

[amirm]

I take it you haven't watched the linked presentation - if you do, I'd be interested in your thoughts on the issues raised (primarily after 40 minutes).

I watched all of it.

 

[amirm]

Flat anechoic response is, of course, less than half of what we need in speakers

That is not correct (it being half). Due to precedence effect, and to some extent, listener preference, on-axis response by far determines preference for speakers. Once that is in place, then we like to see similar off-axis performance. That aspect is not nailed down nearly as much as on-axis is.

 

[Mad_Economist]

That is not correct (it being half). Due to precedence effect, and to some extent, listener preference, on-axis response by far determines preference for speakers. Once that is in place, then we like to see similar off-axis performance. That aspect is not nailed down nearly as much as on-axis is.

Please see Sean's multiple-regression model for the impact of direct sound, I wasn't pulling that number out of a hat.

I watched all of it.

I suppose that's that, then. If you do have any thoughts on the three categories of issues raised - I'm particularly interested in your views regarding the emergent interaction of headphones and ears, and the visualization of positional variation in headphone response - I'd be interested in them.

 

[SalutaryGraveness]

"Two different listeners may hear a given headphone meaningfully differently"

Yeah, that's a bit like saying that two people may see different colors in the same perfectly calibrated computer monitor. ( and they might ). In the science of colorimetry they solved most of these problems with solutions such a "standard observer", which is nothing but a *reasonable* model of a what an actual observer might see. This "standard observer" would tell you the "truth" which is not an absolute nor "universal" truth, but rather an objective truth. Which is what science actually cares about. So many people are confused with this.

And just like that, you don't care anymore if someone thinks the white color doesn't look white. Because you got objective standards that eliminate many variables and subjectivities that are not really very meaningful nor useful. That's how, I think, we got out of many dark ages.