• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Should we (I) get into speaker testing & measurement

Should we get into proper speaker measurements?

  • Yes

    Votes: 247 76.5%
  • Yes, but do it later.

    Votes: 30 9.3%
  • No. Stay with Electronics.

    Votes: 46 14.2%

  • Total voters
    323

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,388
Location
Seattle Area
EDIT: I have now purchased this system and measuring one speaker every other day!

Hello everyone.

I am doing some planning on where we are going from here. We have tested a ton of audio electronics and while there is still endless supply of such gear waiting to be tested, I hear complaints from time to time that we are not addressing the most audible components in the chain, namely speakers and headphones. I will leave headphones for another time. For now, wanted to discuss speakers.

As some of you know, the gold standard in objective audio measurements for speakers is the work Dr. Toole/Olive have done at NRC and Harman. Basically, a set of weighted measurements incorporating direct and indirect sounds can predict listener preference for a speaker to high degree. We are talking 80% or so predictive power.

The challenge for anyone to perform these measurements is the requirement for anechoic chamber to measure the speaker and not the room. And needing to have many measurements to cover all axis around the speaker. Magazines and others perform the test using far fewer axis and gating (stopping measurement before first reflections). The latter limits the low frequency response though which is an important metric (as much as 30% of preference for a speaker is from bass).

Enter the company Klippel. They have built what they call a near-field scanner. Using dual measurements and some very fancy math, they are able to a) eliminate the effect of reflections and b) extrapolate far field data (what we want to have) from near field measurements (close in to speakers which is much simpler to perform). This is a combination of software and hardware. The hardware is a robotic measurement arm that scans the speaker in 3-D space. You can see it in this short video:


The system actually works better than anechoic chamber because even the largest of such rooms, still have very low frequency room modes. You can see this in this example measurement from Klippel comparing Harman anechoic chamber measurements to theirs:

1561499888684.png

If you look to the left, the Klippel system in black has a more correct curve than the one in red (anechoic).


Once the measurements are performed, and signal processing performed, they are then able to produce Harman style measurements (which are now standardized in CEA 2034). Here is sample data for a Harman speaker (F228):

Spin%2B-%2BRevel%2BPerforma3Be%2BF228Be.png

Here, we would look for flat on-axis response (black) and smooth curves for the rest. Or alternatively the directivity index (lines below) not having kinks in them.

Anyway, bottom line is that unlike our electronic measurements, such speaker measurements are highly correlated to what we hear. Currently no one is performing such measurements for the public. Even Harman which has such data, doesn't publish it routinely for their own speakers.

We have the chance to completely transform the speaker industry, bringing much needed objective and useful data to the table.

That is all the good news. Here is the really, really bad news. The basic hardware alone costs $39,000 Euros. With necessary software this cost balloons to over $70,000! This puts my nearly $30K investment in Audio Precision to shame. On top of this, we also have the logistic cost of sending speakers back and forth.

My question to you all is whether we should go in this direction now, later or not bother. Poll options are above. If we do go forward, funding of this will be a serious thing to figure out. Options exist with respect to potential sponsorship, serious donations from you all and my pocketbook.

Any and all thoughts are welcome including doing this for headphones where the costs are much lower (but the work less certain).
 
Last edited:

pwjazz

Addicted to Fun and Learning
Forum Donor
Joined
Sep 21, 2018
Messages
507
Likes
746
Who uses Klippel's system at the moment? Manufacturers for internal testing?
 

digitalfrost

Major Contributor
Joined
Jul 22, 2018
Messages
1,521
Likes
3,086
Location
Palatinate, Germany
The quality and science of this website had tremendous impact on me. I spent a lot of money because of you, because I trust your results. That is more than I can say of most forums or review pages. I would absolutely love it to have the same care and quality applied to speaker reviews.

Given what you propose, it seems you could do even better than bascially anybody else at the moment? I'm all for it. From a business perspective however, I can hardly see how you could make that worthwhile. If it's really that good, why is Stereophile not doing it? And what if they are?
 

Ron Texas

Master Contributor
Joined
Jun 10, 2018
Messages
6,078
Likes
8,914
It's an impressive machine, but I suppose a lot of folks just take the speaker outside.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,388
Location
Seattle Area
Who uses Klippel's system at the moment? Manufacturers for internal testing?
Due to high expense, some manufacturers are going to their labs and making measurements there.

THX also had one and was using it to rate speakers but seems like their website has vanished.
 

SIY

Grand Contributor
Technical Expert
Joined
Apr 6, 2018
Messages
10,383
Likes
24,749
Location
Alfred, NY
It's an impressive machine, but I suppose a lot of folks just take the speaker outside.

Or do what I do: gated quasi-anechoic, then stitch that to a Keele-style nearfield measurement at low frequencies.

The key is to have the right measurement mikes for these methods and properly interface them to the AP.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,388
Location
Seattle Area
The quality and science of this website had tremendous impact on me. I spent a lot of money because of you, because I trust your results. That is more than I can say of most forums or review pages. I would absolutely love it to have the same care and quality applied to speaker reviews.

Given what you propose, it seems you could do even better than bascially anybody else at the moment? I'm all for it. From a business perspective however, I can hardly see how you could make that worthwhile. If it's really that good, why is Stereophile not doing it? And what if they are?
Thanks for the kind words. :) On stereophile doing it, it really is not an impact on what I do here as the frequency of their measurements is nowhere close to what I do. My time is free, theirs is not. :)
 

Daverz

Major Contributor
Joined
Mar 17, 2019
Messages
1,294
Likes
1,451
I'd like to see more measurements of inexpensive (loudspeaker) power amps. So far well-measuring power amps under $1k have been elusive.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,388
Location
Seattle Area
Of course my answer is yes :) But only if sponsorship (and/or other forms of advertising) can be avoided.
That is my preference too. But do we have the wherewithal as a community to come together and fund such a thing? How many people like me are willing to write significant checks toward this? I can do the heaviest lifting but still need major contributions from members.

A while back Steve Guttenberg challenged the community to come up with such money.


Wonder if we can or should rise up to the occasion....
 

digitalfrost

Major Contributor
Joined
Jul 22, 2018
Messages
1,521
Likes
3,086
Location
Palatinate, Germany
I'd like to see more measurements of inexpensive (loudspeaker) power amps. So far well-measuring power amps under $1k have been elusive.
I absolutely agree. We have plenty of good measuring line level devices, but amplifiers remain a mystery unless you can spent the money on NC400 or AHB2.

That said, I don't think this is an either-or question. Why not both :D
 

andreasmaaan

Master Contributor
Forum Donor
Joined
Jun 19, 2018
Messages
6,652
Likes
9,399
Perhaps @amirm you might try something along the lines of SIY's suggestion first to see what response you get on the forum? You would not get a fully accurate low frequency response with the near-field technique, but if you have some open space nearby, it is possible to get pretty accurate results using the ground-plane technique.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,388
Location
Seattle Area
Perhaps @amirm you might try something along the lines of SIY's suggestion first to see what response you get on the forum? You would not get a fully accurate low frequency response with the near-field technique, but if you have some open space nearby, it is possible to get pretty accurate results using the ground-plane technique.
It is not just the low frequency response but the rest of the measurements in order to arrive at the right data set to then predict listener preference.

That can be done using semi-manual system with an automated turntable and manual or automated microphone placement. Harman measures 70 points for example. But even this setup goes well above $10,000 and still not optimal.

To get the high throughput that I have now on electronics, an automated system that performs all of these measurements quickly and far more accurately is needed. Otherwise the scattering of speaker measurements is already out there.

In a nutshell, we are doing first-class evaluation of electronics. Do we want to invest in first-class evaluation of speakers? One where the prediction results for listener preference is almost a given?
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,388
Location
Seattle Area
That said, I don't think this is an either-or question. Why not both :D
That's the plan. Electronic measurements will still go on and full pace.
 

Biblob

Addicted to Fun and Learning
Joined
Sep 13, 2018
Messages
633
Likes
600
I would love to see it.
It would mean that ASR would grow even larger and reach more people. My guess is ASR would grow a lot. Because speakers are so important and interesting.

But yeah, how can this investment be made possible? For this I have no answer, unfornutately.
 

suttondesign

Addicted to Fun and Learning
Forum Donor
Joined
Feb 4, 2019
Messages
732
Likes
1,310
Location
Bellingham, WA
The best speakers I have ever heard are ones you can't really buy or that otherwise can't be measured easily, like Linkwitz and Martin-Logan. The gazillions of box speakers present an impossible burden for any site or person to take in, measure, and give much insight into. Plus, part of the fun of audio is experiencing different air movers. With electronics, I just want instrument-grade stuff that I know isn't changing the signal, because I know the air-mover and the room will.
 

SIY

Grand Contributor
Technical Expert
Joined
Apr 6, 2018
Messages
10,383
Likes
24,749
Location
Alfred, NY
Automated turntables are ideal, but only work on a horizontal axis; Old Colony used to sell one, but if there's any left, the staff of the successor company don't know where they'd be. Unfortunately, I have to manually move the mike and speaker around (lazy susan turntable with angle markings, and a set of marks on my mike stand for vertical angles), and indeed that's time consuming. But I can get a full set of vertical and horizontal measurements in an afternoon, so the throughput isn't horrible. And the LF stuff isn't very angle sensitive, so only has to be done once.

It is important, though, to have separate lab-grade mikes for the quasi-anechoic and nearfield because of spl vs noise tradeoffs. Fortunately, I have some excellent calibrated condenser mikes from PCB.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,388
Location
Seattle Area
The best speakers I have ever heard are ones you can't really buy or that otherwise can't be measured easily, like Linkwitz and Martin-Logan.
Oh, we can measure those just as well. Here is Harman's measurements of the ML:

Harman%2520Listening%2520Tests.jpg
 

mikewxyz

Member
Joined
Jun 19, 2019
Messages
37
Likes
39
Hi Amir: First, let me thank you for the time and work you put into this site. I find it interesting.

To your question of whether to do speaker measurements, I think you need to have a clearer statement of purpose.

I see your DAC measurements and ranking as a Consumer Report like buyer’s guide. The data supports the larger view of this site that real differences in audio can be objectively measured even if, in some cases, they are below the audible threshold.

I think everyone would agree that speakers sound different both as a function of design and room environment. It’s not practical to evaluate a long list of speakers. Your costs would be crazy high compared to established audio e-magazines who get product on loan.

How would these measurements differ from Stereophile? Would you test a sample of speakers from representative driver technology or anything that comes your way? What would people do with the data? Reasonable people can disagree about the preferred shapes of FR graphs or what “sound” sounds best to them.

Sounds like a lot of work.
 
Top Bottom