• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Should we (I) get into speaker testing & measurement

Should we get into proper speaker measurements?

  • Yes

    Votes: 247 76.5%
  • Yes, but do it later.

    Votes: 30 9.3%
  • No. Stay with Electronics.

    Votes: 46 14.2%

  • Total voters
    323

Nango

Major Contributor
Joined
Aug 6, 2018
Messages
1,463
Likes
985
Location
D:\EU\GER\Rheinhessen
That is my preference too. But do we have a wherewithal as a community to come together and fund such a thing? How many people like me are willing to write significant checks toward this? I can do the heaviest lifting but still need major contributions from members.

A while back Steve Guttenberg challenged the community to come up with such money.


Wonder if we can or should rise up to the occasion....


Why not approaching the community around philantropics like Gates & Buffet for sourcing funds, at the end, this is science what you are doing, and education as well, as already mentioned before.
 

andreasmaaan

Master Contributor
Forum Donor
Joined
Jun 19, 2018
Messages
6,652
Likes
9,399
Oh, we can measure those just as well. Here is Harman's measurements of the ML:

Harman%2520Listening%2520Tests.jpg

I'm disappointed Harman used one of the worst-measuring panel speakers in their main published test.

Given the fact this speaker fails on the basic test of axial frequency response, I don't think we can draw any conclusions about panel speakers, or even other speakers in the ML line, from this result.

(No particular fan of panel speakers here btw, and have never even listened to or measured MLs! Just pointing out the perils of drawing a general conclusion from a very particular instance.)
 

tomissoawesome

Member
Forum Donor
Joined
Aug 17, 2018
Messages
17
Likes
46
Quickly measuring electronics has been a pretty big differentiator for ASR. Other measurements pop up here and there, but not as consistently or comprehensively. Still, for DACs and headphone amps, at least, it seems to me that ASR has identified:
  • the state of the art
  • the best values
  • and the overall quality of many of the popular models.
At this point, there's diminishing returns on desktop electronics measurements. The measurements are still interesting, just a touch less necessary. (I'm not a headphone guy, for what that's worth.) Obviously there's still work to be done for AVRs, integrated amps, power amps, and phono stages.

The speaker measurement field is pretty crowded, but I think ASR could offer a different perspective. The differentiator there would be that presumably the text and the measurements would compliment each other rather than appear uncorrelated.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,382
Location
Seattle Area
I'm disappointed Harman used one of the worst-measuring panel speakers in their main published test.

Given the fact this speaker fails on the basic test of axial frequency response, I don't think we can draw any conclusions about panel speakers, or even other speakers in the ML line, from this result.

(No particular fan of panel speakers here btw, and have never even listened to or measured MLs! Just pointing out the perils of drawing a general conclusion from a very particular instance.)
I did not intend to have a discussion about the specifics there (we have had that elsewhere). The point was that there is nothing special about these speakers with respect to measuring their response. The system that I am talking about will make a spherical measurement and give us the data to then analyze.
 

restorer-john

Grand Contributor
Joined
Mar 1, 2018
Messages
12,579
Likes
38,274
Location
Gold Coast, Queensland, Australia
Just as ASR reviews a seemingly endless number of tiny little Chinese D/A converters, practicality would mean speakers would be small two ways, powered mini-monitors and the odd DSP powered medium standmounter. Once you get into 20-50kg+ towers and above, where are you going to put them, pack them, test and ship them back? Damage in transit will be a serious issue as is always has been with speakers.

You won't be having SOTA large and heavy speakers landing on your doorstep every other day being loaned to you, so the representation is likely to be at the 'value' end of the equation isn't it?

To me, more comprehensive testing of the product categories you already cover would have greater value.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,382
Location
Seattle Area
I think everyone would agree that speakers sound different both as a function of design and room environment. It’s not practical to evaluate a long list of speakers. Your costs would be crazy high compared to established audio e-magazines who get product on loan.

How would these measurements differ from Stereophile? Would you test a sample of speakers from representative driver technology or anything that comes your way? What would people do with the data? Reasonable people can disagree about the preferred shapes of FR graphs or what “sound” sounds best to them.

Sounds like a lot of work.
Answering the last bit, yes, it is a lot more work including physical aspects of handling speakers. My current thought is to mostly limit the testing to small to medium with some exceptions here and there. In the larger scope, I am more worried about the cost of the instrumentation than the rest. I can go on a better diet and learn to lift 100 pound speakers. :) Can't wish for $70K to fall on my lap.

The measurements will be very different than stereophile and much more understandable. The system will measure a complete sphere around the speaker, not a few degrees left and right. This allows to one plot a curve like this:
1561510067538.png


A weighted sum of the three lines above allow one to predict listener preference to high degree. It takes 70 measurements for Harman to produce the above graph in an anechoic chamber. This method of correlation is based on measurements of countless speakers and hundreds of listeners across many trials.

In contrast, the stereophile measurements are rather crude:

508Ushfig5.jpg


Notice how their measurements stops at 300 Hz on the low side.

There is also the question of the future of stereophile measurements. I am not confident that JA will continue in this role to make measurements, speakers or otherwise.

On availability of speakers to test, that is an open question to membership. I can source fair number myself without purchasing. There are also a ton of budget speakers/monitors which would be good to review from Elac to Yamaha. I think if we just profiled every small speaker, we would be in a far better place than we are now.

We also have a number of manufacturers that sell on measured performance such as 8C, Kii, Dutch & Dutch, Genelec, etc. who over time may be willing to send samples in for testing. Indeed some may not have access to such measurement system and see it as a value to them just to have the data, much like some companies do with respect to Audio Precision APx555 which I have now for electronics.

But yes, this is a complicated thing and hence the reason I am asking for feedback. :)
 

restorer-john

Grand Contributor
Joined
Mar 1, 2018
Messages
12,579
Likes
38,274
Location
Gold Coast, Queensland, Australia
In contrast, the stereophile measurements are rather crude:

508Ushfig5.jpg

How is that plot remotely crude? And it's hardly just
a few degrees left and right.
The measurements are 180 degrees (+/-90 degrees, appears to be close to 3 degree points based on the plot lines) and the progression of FR change is clearly visible, something the other plot shows nothing of.

If the HK one takes 70 measurements in a 360 degree plot, it's front 180 degrees are at much coarser increments than the MLSSA plots aren't they?

One is a emulsified, easily digestible plot, the other contains way more information IMO.
 

StevenEleven

Addicted to Fun and Learning
Forum Donor
Joined
Dec 1, 2018
Messages
581
Likes
1,188
Answering the last bit, yes, it is a lot more work including physical aspects of handling speakers. My current thought is to mostly limit the testing to small to medium with some exceptions here and there. In the larger scope, I am more worried about the cost of the instrumentation than the rest. I can go on a better diet and learn to lift 100 pound speakers. :) Can't wish for $70K to fall on my lap.

The measurements will be very different than stereophile and much more understandable. The system will measure a complete sphere around the speaker, not a few degrees left and right. This allows to one plot a curve like this:
View attachment 28327

A weighted sum of the three lines above allow one to predict listener preference to high degree. It takes 70 measurements for Harman to produce the above graph in an anechoic chamber. This method of correlation is based on measurements of countless speakers and hundreds of listeners across many trials.

In contrast, the stereophile measurements are rather crude:

508Ushfig5.jpg


Notice how their measurements stops at 300 Hz on the low side.

There is also the question of the future of stereophile measurements. I am not confident that JA will continue in this role to make measurements, speakers or otherwise.

On availability of speakers to test, that is an open question to membership. I can source fair number myself without purchasing. There are also a ton of budget speakers/monitors which would be good to review from Elac to Yamaha. I think if we just profiled every small speaker, we would be in a far better place than we are now.

We also have a number of manufacturers that sell on measured performance such as 8C, Kii, Dutch & Dutch, Genelec, etc. who over time may be willing to send samples in for testing. Indeed some may not have access to such measurement system and see it as a value to them just to have the data, much like some companies do with respect to Audio Precision APx555 which I have now for electronics.

But yes, this is a complicated thing and hence the reason I am asking for feedback. :)

I think all of these tests of amplifiers and DACs and headphone amps and whatnot are irrelevant to over 99 percent of the population at large, as interesting and admirable as they may be, and to the extent they put the heat on the snake-oil and incompetent sectors of high-end audio, yes, those folks have had it coming for a long time, for decades.

However, in my view sonic differences in speakers swamp sonic differences in competently designed receivers, amps, DACs, etc.

So I’d like to see measurements of speakers and I’d be much more interested in what makes for a good speaker than how far beyond or just short of the thresholds of audibility the other electronics at each price point perform.

Also, many, many speakers don’t make it cleanly down to 40 hz without some substantial roll-off. If you want full range sound of a symphony or a jazz quartet, for example, you want to hear the bottom end down to 40 hz (approximately the low E on an electric or acoustic double bass). Or even 27 hz for the bottom end of a piano. And modern pop and electronic music can go really low. So I think the capability that powered subwoofers have added to reproduce the lower frequency range of music with relatively affordable gear is among the biggest advances in fidelity we have seen in recent years. It’s something you can definitely hear and feel to make music more realistic and something that commercial digital recordings can provide that LPs, for example, had a lot of trouble with. It’s not that last nth degree of s/n + thd or whether the treble is rolled off by 1db at 20 kHz. Low bass is real and a genuine part of the music and makes a substantial and palpable difference when it is reproduced accurately. So I would want to see powered subwoofers tested too.

Just my two cents!
 
Last edited:
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,382
Location
Seattle Area
How is that plot remotely crude? And it's hardly just
It is 7 data points versus 70. It is way undersampling the field, causing spatial aliasing in addition to severe lack of resolution.

This is what high-resolution data looks like:

1561514892775.png


The data is also hard to interpret compared to directivity information in the Harman plot.

Most importantly, I know how to predict preference using CEA 2034/Harman data since they two have been correlated. There is no such thing for stereophile data.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,382
Location
Seattle Area
However, in my view sonic differences in speakers swamp sonic differences in competently designed receivers, amps, DACs, etc.
Most definitely. I often talk about what the latest science says about the best speakers. But then people ask me how to get that kind of data, who makes speakers that comply with it, etc. and I can't answer. No one publishes the right data. Most don't even measure it.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,382
Location
Seattle Area
Where are you getting 7? It’s 37 by my count.
Ah, you are right. But that is still very coarse compared to what we can measure. I just post an example with 500 points.
 

DDF

Addicted to Fun and Learning
Joined
Dec 31, 2018
Messages
617
Likes
1,355
Or do what I do: gated quasi-anechoic, then stitch that to a Keele-style nearfield measurement at low frequencies.

The key is to have the right measurement mikes for these methods and properly interface them to the AP.

This works well, if overlaying a model of the cabinet diffraction on top of the near field before splicing. Stereophile doesn't do this, making their results below the gate frequency inaccurate. It can make a dB or so difference over a very wide bandwidth, well worth the effort. It is a pain though as the diffraction response is observation distance and angle dependent.

The gating itself also greatly reduces frequency resolution and can lead to missed or misidentified resonances or issues.

Benjamin has a technique where pre-emphasis on the MLS is used followed by de-emphasis in processing, and it effectively suppresses the impact of reflections and allows longer windows/higher frequency resolution. I world strongly recommend @amirm investigate that option before dropping significant coin on the Klippel.
 

garbulky

Major Contributor
Joined
Feb 14, 2018
Messages
1,510
Likes
827
Hi @amirm I don't know anything about that new system. It seems like it trying to simulate an anechoic chamber out of in room measurements sounds rather hokey. Reminds me about all the promises of room correction systems. Then I sit down and listen to them and well I can't wait to turn them off. Also that measurement scanner machine looks bulky. Having said that, I clearly know next to nothing about it.
But it feels like you are trying to get SOTA. It looks liek this will cost $$$$ and also take time.

But somebody like me would simply be happy with some basic measurements. As long as the environment/conditions stay the same over time. There is not a whole lot of websites that do speaker measurements of popular speakers and even less that are as popular or easy to access like ASR.
Things I would be interested in seeing is both near field response and then an actual in room response. I would want to know things like does the bass extend anywhere close to the claims because most speakers have very little flat response in the deep frequencies and that's something that's bugged me with a lot of speakers. Especially monitors claim all kinds of things like 50 hz bass when in reality you are having roll off at 120 hz down. Something as small as that would be invaluable for a lot of people.

The other would be some sort of listening test. I get that BLIND listening tests would be very challenging. But there is a lack of A/B tests. Though I'm not a fan of level matching, that may also be quite interesting to me. So simply listening tests with comparisons done in a repeatable manner would be quite interesting to me.

Summary: I would like to say bass frequency to see if the speakers can actually cut it - including in room. I would like to see some sort of comparison listening tests between speakers including level matched. The precision doesn't matter too much to me. So a bit of time and a lot of $$$$ sacrificed to get SOME information would be what I would like to see.

Also the same goes for headphones. Here I am not interested in precise frequency response tests. But I am interested in things like how do different amps power a headphones in terms of bass response. Also I would be very interested in blind (ior perhaps level matched) listening tests on headphones.
 

Attachments

  • 1561515821817.png
    1561515821817.png
    1.6 KB · Views: 132

SIY

Grand Contributor
Technical Expert
Joined
Apr 6, 2018
Messages
10,383
Likes
24,749
Location
Alfred, NY
Can you give an example of something significant you can see with 500 datapoints that you miss with 37? Serious question, since my measurements tend more toward the latter (noting that in my case, and I think JA's, each measurement is the average of three to five measurements with the mike slightly repositioned around nominal). And so far, my measurements have correlated very well with how the speakers sounded.

Other thing is that polar data below 300 Hz isn't terribly meaningful given the wavelengths involved. The Keele data tell you what you need to know about the speaker's bass output- from there, it's up to the room.
 

MZKM

Major Contributor
Forum Donor
Joined
Dec 1, 2018
Messages
4,240
Likes
11,462
Location
Land O’ Lakes, FL
Ah, you are right. But that is still very coarse compared to what we can measure. I just post an example with 500 points.
I’m actually not a fan of polar plots, they are less forgiving than FR graphs (a speaker can jump 3dB at 2kHz a bit off-axis, yet the color stays the same).

I do dislike how low-resolution (pixel-wise) Stereophile’s images are, even the photos of the products are low res.

What the Spinorama doesn’t show is distortion and compression, which the NRC’s measurements (on SoundStageNetwork’s site) do show, which is also very helpful, as usual speaker measurements are done simply at 2.83V. The Spinorama can be amazing, but if it’s an 85dB sensitive speaker that distorts when fed 10W and up, then it’s only suited for desks and bedrooms.
 
Last edited:

SIY

Grand Contributor
Technical Expert
Joined
Apr 6, 2018
Messages
10,383
Likes
24,749
Location
Alfred, NY
Hi @amirm I don't know anything about that new system. It seems like it trying to simulate an anechoic chamber out of in room measurements sounds rather hokey.

Nope, above the gating frequency (I go to double that to be sure), the quasi-anechoic data are anything but hokey, and correspond very well to anechoic data. During the time I had access to an anechoic chamber, I did some comparative measurements and saw no significant differences.
 

DonH56

Master Contributor
Technical Expert
Forum Donor
Joined
Mar 15, 2016
Messages
7,834
Likes
16,496
Location
Monument, CO
Intriguing but I share the concerns of initial investment and imposition of trying to ship big honkin' speakers around. How on earth would I get my Salon2's to you to measure? Oh, wait, nevermind, I'm good. or rather you are... :)

Is there an intermediate approach, say building a small anechoic chamber'ish and gating the (near-field) measurements to provide an approximate (but decent) rendition of full-range performance?

I think it would be neat to develop a simple stand-alone data collection system members could you to collect data and either process it themselves or send to you. E.g. a little box to plug in-line with speakers and gather info about impedance, power levels, and all that jazz. I've though about such a thing for years but never had the time and resources to do it... Now, with Raspberry Pi and Python it may be attainable. But then again I have put in 27 hours this week and it's Tuesday, spare time is not one of my luxuries...
 

SIY

Grand Contributor
Technical Expert
Joined
Apr 6, 2018
Messages
10,383
Likes
24,749
Location
Alfred, NY
The gating itself also greatly reduces frequency resolution and can lead to missed or misidentified resonances or issues.

With a million point FFT, I'm not too worried about that loss of resolution! The other thing to do if you're not trying to measure ultrasonics is to reduce the sampling frequency.

Can you give me a reference to the Benjamin paper?
 

MZKM

Major Contributor
Forum Donor
Joined
Dec 1, 2018
Messages
4,240
Likes
11,462
Location
Land O’ Lakes, FL
Is there an intermediate approach, say building a small anechoic chamber'ish and gating the (near-field) measurements to provide an approximate (but decent) rendition of full-range performance?

If Amir’s home isn’t close to neighbors, bookshelf speakers can be measured outside, pic:
image


Tower speakers, especially those weighing a ton, would be a challenge though.

One issue in measurements is knowing what the reference axis is, many speakers have it so tweeter height is the intended listening axis, but others can have it lower, and sometimes it isn’t specified in the manual. The latest Wilson Audio speaker Stereophile measured is like this, and J.A. even notes this when describing the step response, yet for some reason he measured on the tweeter axis, so the performance looks worse than it seems.
 
Last edited:
Top Bottom