Should we (I) get into speaker testing & measurement

[GrimSurfer]

@oivavoi raised some excellent points, Amir.

My advice would be to keep it simple. Focusing on electronics has a decent shot at influencing the industry for the better.

Adding speakers to your testing, while interesting to everyone on the forum (myself included), may water-down the significance of your broader work to the point where the industry can simply ignore it.

And that would be a great shame.

 

[DDF]

Thanks for the references. Unfortunately all of these schemes require work, manual intervention to get reasonable results

My pleasure. The klippel system looks amazing and I hope it works out but if the expense proves insurmountable, it may be worth a look at Sound Easy which has the Benjamin type technique built in (the music and design link posted earlier) removing much of the manual aspect. Most systems now also have a turntable interface to run polars though mic height must be changed for vertical slice.

IME the number of points can be greatly reduced from the Harman suggestions while still providing 98% of the value

 

[amirm]

Yes, but this requires a extreme level of precision matching of the channels and excellent signal-to-noise ratios to work properly, like any other subtractive technique.

To clarify, this is not simple subtraction. This is part of the math for separation of the fields:

Since the Klippel system can move the microphone, the dual measurements are performed using the the same mic and electronics so that part doesn't vary.

 

[Shadrach]

I think it's a great idea, but I can't see how to fund and implement realistically so the site can deal with what imo needs a good look at and that's so called Hi End speakers. There are already decent reviews from other sites on the studio monitor type speakers up to the price range of say $2000.
What there isn't is a a major measurement and tear down of speakers say above $10000. It's this range and above that imo really hurt audio.

 

[amirm]

Can of worms, really.

Headphone testing is indeed that. There is a reason that despite availability of many measurements, people are still bewildered about them.

It is so bad that I gave up on the project until I chatted with Sean Olive at CanJam. My thought is to precisely replicate not only the gear but the procedures Harman uses for their measurements. This will be done with in person review of what Harman is doing and exchanging measurement data back and forth until we get full alignment. Once there, then the work they have done to correlate their measurements to listening tests will be applicable.

This is not a 100% solution but I think it can get us close.

Assuming so, would people want me to go after headphone testing first???

 

[edechamps]

Assuming so, would people want me to go after headphone testing first???

My 2 cents: you would have way more impact on the world by publishing loudspeaker measurements. Other websites, most notably Rtings, already do plenty of headphone measurements (and, in the case of Rtings, they do it quite well). In contrast, finding loudspeaker measurements feels a lot like winning the lottery. Data on loudspeakers is sorely lacking. I would not hesitate to donate to any effort aimed at changing that state of affairs.

 

[mi-fu]

I would love to see measurements of speakers.

But considering the costs of the equipment and the troubles of sending heavy speakers back and forth, I would say it is a huge investment. It all depends on the willingness of people to contribute. Say, if we have 300-400 people who are willing to donate $100 or so. Then we have 30,000-40,000. Amir probably can have a sense whether having such 300-400 people is an realistic estimation or not, based on the current donation / subscription.

For myself, I'm not entirely against sponsorship, as long as the ads are done by third parties (like Google ads). Then it may help to maintain the site's neutrality.

Assuming so, would people want me to go after headphone testing first???

I think it is a much manageable direction. It also aligns with the current focus here. Headphone measurements will also drive more visits.

With more visits, it will also help to fund the speaker measurement project.

 

[617]

Headphone testing is indeed that. There is a reason that despite availability of many measurements, people are still bewildered about them.

It is so bad that I gave up on the project until I chatted with Sean Olive at CanJam. My thought is to precisely replicate not only the gear but the procedures Harman uses for their measurements. This will be done with in person review of what Harman is doing and exchanging measurement data back and forth until we get full alignment. Once there, then the work they have done to correlate their measurements to listening tests will be applicable.

This is not a 100% solution but I think it can get us close.

Assuming so, would people want me to go after headphone testing first???

Amir, I am a speaker guy. I would love to see more speaker measurements, but at least with speakers there are meaningful measurements out there. However, the problems with headphone measurement are far more challenging and interesting, and it would be wonderful if we could develop a protocol which draws on all the expertise here to meaningfully compare headphone performance. It would also make community participation easier - many members may one have one or two pairs of speakers, which are heavy, but with headphones you could get hundreds of models compared and sent around at low cost.

The challenges facing headphone measurement have been summarized here, and it seems impossible, but that is why it is seems like a more interesting task to tackle.

As I see it, the key to headphone measurement might be wide participation with lots of ears and lots of comparisons. Yes, we can't define what ideal headphone performance is in an acoustical context, but that doesn't mean we can do what we've done for speakers, which is define listener preferences and compare, if not measure them.

 

[zermak]

I've voted yes and considering your objectivity (as much as you can be) it would be great to read some nice reviews and measurements on loudspeakers.
It is quite challenging tho but I can tell that you have the skills and the passion to do it in a proper way.
Plus I hope you will focus on the mainstream loudspeaker market so modest people like me can relate and maybe even buy it ahah

Out of context I would also like to read some more AVR reviews on the mch amplifier part to verify how good the non-stereo channels amps are.

 

[Krunok]

My 2 cents: you would have way more impact on the world by publishing loudspeaker measurements. Other websites, most notably Rtings, already do plenty of headphone measurements (and, in the case of Rtings, they do it quite well). In contrast, finding loudspeaker measurements feels a lot like winning the lottery. Data on loudspeakers is sorely lacking. I would not hesitate to donate to any effort aimed at changing that state of affairs.

I fully agree.

 

[Thomas savage]

I guess I could finally give in to demand and sell my used underpants to Russia oligarchs, this would cover costs here.

 

[Krunok]

I think it is a much manageable direction. It also aligns with the current focus here. Headphone measurements will also drive more visits.

I'm not sure either is true, but maybe there should be another voting with option for headphones included..

 

[GrimSurfer]

My thought is to precisely replicate not only the gear but the procedures Harman uses for their measurements. This will be done with in person review of what Harman is doing and exchanging measurement data back and forth until we get full alignment. Once there, then the work they have done to correlate their measurements to listening tests will be applicable.

This has moral danger written all over it. You wouldn't be the first person to fall under the spell of big business once you share a working relationship.

You have considerable experience in tech, Amir, but companies have a way of exerting incremental influence under the guise of R&D until people find themselves humming the corporate anthem in their sleep.

 

[mi-fu]

I'm not sure either is true, but maybe there should be another voting with option for headphones included..

Yes. having a voting option for headphone measurement would be good.

 

[amirm]

Yes. having a voting option for headphone measurement would be good.

Here you go: https://www.audiosciencereview.com/...get-into-testing-headphones-or-speakers.7910/

 

[oivavoi]

This has moral danger written all over it. You wouldn't be the first person to fall under the spell of big business once you share a working relationship.

You have considerable experience in tech, Amir, but companies have a way of exerting incremental influence under the guise of R&D until people find themselves humming the corporate anthem in their sleep.

One of the cool things about the Klippel box, though, is that it can measure lots of distortion types which Harman haven't seemed to pay much attention to, publicly at least. For those who have nothing better to do with their life, here are 69 pages where Wolfgang Klippel discusses "Loudspeaker Nonlinearities – Causes, Parameters, Symptoms"

Klippel has also proposed a rather elaborate method for combining subjective and objective loudspeaker assessments. I remember that I tried reading it some time ago, but gave up, became annoyed and had a beer instead.

With all of that said, I agree with Amir's comment that there at least are some low-hanging fruit here. There are by now many small budget speakers. Some of these must surely measure much better than others along the criteria that everybody agrees is important - smooth frequency response, no annoying distortions, etc.

 

[Travis]

Oh, we can measure those just as well. Here is Harman's measurements of the ML:

Amir,

Can you post a link to the source of this Harman data please?

 

[GaryS]

So my question is do we have enough people on this site to fund this? How much do I need to donate? It sounds like Amir needs $70K.
Speakers=yes

 

[Mad_Economist]

Some commentary on headphone measurement, since this is an area of immense personal significance to me:

Headphones are (properly) measured on anthropomorphic fixtures - either Head And Torso Simulators (HATS) per IEC60318-7 and ITU-T P58, or cheaper "ear and cheek" or "hearing protection test fixtures" which feature an anthropomorphic ear on a flat mounting plate such as GRAS' 43AG or 45CA. In either case, an anthropomorphic human pinna based on a population average (examples and requirements are given in ITU-T P57, IEC60318-7, and IEC60268-7, although to my knowledge only GRAS produces a pinna from the last) with a short "canal extension" tube leading from its ear canal entrance is mated with an IEC60318-4/IEC711 "ear simulator", which emulates the impedance of the human ear at the drum. These systems are used by most professionals measuring headphones.

Some have suggested in this thread that it is necessary to know the HRTF of a specific user to characterize headphone response. There may certainly be some truth here - the results of Smyth's excellent Realizer certainly could indicate in this direction - but I feel that it appears to be taken too far as currently interpreted. When the HRTFs of HATS systems have been measured, they fell relatively close to population averages (as they were intended to), and measurements done using HATS systems and analogous ear simulators have produced robust predictions of subjective frequency response and preference - as an example, the work of Sean Olive, which was done primarily on a GRAS 43AG and 45CA, fairly reliably predicts user preference for headphones based on frequency response.

Consequently I feel it's quite hard to argue that the measurement of headphones is a truly uncertain space. There are certainly nuances - variations in both placement and individual anatomy may influence results to some degree - and I would say that it is a less filled in space than the world of speakers, but it's an area where we have fairly strong tools and a reasonable understanding of how to use them. It's a developing area, without question, and one that was long neglected in favour of speakers, but we have a reasonable body of literature to draw upon at the moment (Olive's headphone paper collection bundles many of the major ones), and can come to fairly robust conclusions.

@SIY you mention that changing systems significantly changed the response of the headphones you were measuring - may I ask what systems you were comparing? While in principle I would expect some variation based on the parameters of the specific pinnae used - something on the order of the differences between the major brands' HATS perhaps, which I would expect to be largely (although perhaps not completely) accounted for by using a DF-HRTF appropriate to the system in question as compensation - but I wouldn't expect it to be very large. Although if you're mostly measuring speakers and I'm mostly measuring headphones, perhaps we have different definitions of large frequency response variation

 

[Mad_Economist]

I listen to a pair of Sennheiser hd380 pro sealed headphones. They have a pretty smooth FR to my ear, with a good balance between treble and bass, yet the measurements looks like this:
https://www.innerfidelity.com/images/SennheiserHD380Pro.pdf

15db swing from bass to treble.

Part of me just can't believe headphones can't be characterized better. They don't even need to be measured at different points in space!

One thing of note here is that - particularly for headphones with a sealed front volume, which I believe the HD380 has - low frequency response will vary significantly based on the presence of leaks induced by ex. poor coupling of the pads to the head, hair, glasses, etc. As a result, it's quite possible that you get a different level of bass at your eardrum to what Tyll Hertsens measured. Additionally, the work of Sean Olive indicates that people prefer bass in excess of a flat line relative to the HRTF of the HATS, and may perceive as "neutral", "balanced", or "natural" boosts of up to 6-8dB at 20hz relative to 200hz (usually implemented in Olive's work as a 2nd order shelf filter at 105hz).