Loudspeaker Frequency Response: Stereophile vs true Anechoic

[staticV3]

Though Stereophile provide measurement data for their speaker reviews, it is well known that their method of graphing anechoic response (splicing gated nearfield data), results in some inaccuracies, especially at low frequencies.

To get a feel for how large these inaccuracies are, I looked through their list of speaker reviews, checked for which speakers I could find true anechoic response data, then plotted the difference between them.

Since Stereophile's graphs use a 30° horizontal AVG, I used Listening Window data as reference (not a perfect match, but better than on-axis).

Here are all 45 response errors with the AVG error in red:


Three distinct groups are visible:

• Bass shelf:
  
• Shelf+midrange scoop:
  
  
• Accurate:
  

In hopes of finding trends, each measurement title contains the year in which the speaker was tested by Stereophile, as well as the type (B=Bookshelf, T=Tower).

Unfortunately I did not find any trends related to review date or speaker type.

Though in general, the response starts to deviate below 300Hz.

Attached below is the .MDAT with all individual error curves, as well as a folder containing the raw response curves in .csv, traced from JPGs/PNGs or directly from Amir's Zips.

Enjoy!

Spoiler: Sources

Floor Loudspeaker Reviews | Stereophile.com

stereophile.com

Stand Loudspeaker Reviews | Stereophile.com

stereophile.com

A collection of loudspeakers measurements

www.spinorama.org

 

[kemmler3D]

Great post. Looks like this tells us something we already knew (their low frequency data is not accurate) and maybe something we didn't - those deviations at 10khz and 20khz weren't on my radar.

Also, the typical deviation from anechoic seems pretty large. Like at least a few dB for any given measurement... some of that is probably due to differences in LW vs. 30deg averages, but hmm...? Did you compute any stats on average / median deviation from anechoic?

 

[DSJR]

In my crude and now highly amateurish way, I simply take 3dB or so off their ~120Hz region to get a basic impression of a speakers' possible bass performance. The extreme top however, often tallies fairly well with other more thorough measurements I think, especially those of speakers with hf that takes off on axis...

It's the HFN plots that look truly suspect to me...

 

[staticV3]

Did you compute any stats on average / median deviation from anechoic?

Not yet. Though if anyone wants to analyze the data more thoroughly, that'd be much appreciated.

 

[mlsstl]

Interesting, but I find the variability of bass in recordings often renders the accuracy of a speaker setup's bass in a room somewhat meaningless. Lots of recordings where the bass is way overdone while pretty much nonexistent on other recordings. Sometimes the results are funny. I recall listening to an old Peter, Paul & Mary recording from the early 60s that was quite enjoyable until I added a subwoofer to my system a few years back. The sub was setup with room equalization so hadn't been overdone. However, on this recording, it brought out all the thumping present on the recording, apparently from a creaky old wooden stage. A good recording turned into an annoying one!

That's why I don't get overly obsessed with chasing the last nth degree of performance from my system -- the variability of recordings for the wide range of music I enjoy often exceeds any deficiencies of my system.

 

[kemmler3D]

Interesting, but I find the variability of bass in recordings often renders the accuracy of a speaker setup's bass in a room somewhat meaningless. Lots of recordings where the bass is way overdone while pretty much nonexistent on other recordings. Sometimes the results are funny. I recall listening to an old Peter, Paul & Mary recording from the early 60s that was quite enjoyable until I added a subwoofer to my system a few years back. The sub was setup with room equalization so hadn't been overdone. However, on this recording, it brought out all the thumping present on the recording, apparently from a creaky old wooden stage. A good recording turned into an annoying one!

That's why I don't get overly obsessed with chasing the last nth degree of performance from my system -- the variability of recordings for the wide range of music I enjoy often exceeds any deficiencies of my system.

Just because the view along the drive isn't always good, doesn't mean you shouldn't keep your windshield clean...

 

[kemmler3D]

Not yet. Though if anyone wants to analyze the data more thoroughly, that'd be much appreciated.

Totally fair, I felt lazy even asking, but should be a good exercise for someone with a spreadsheet and some spare time...

 

[mlsstl]

Just because the view along the drive isn't always good, doesn't mean you shouldn't keep your windshield clean...

Agreed. I think my point was more to note that it is easy to become obsessive about looking for perfection in the system. Besides the recording, there is also the room itself. One can have a speaker that tests perfectly but throws much of that away the minute it is placed in a listener's room. Amazing how many expensive speakers are placed in a room with no RC or acoustic treatment.

 

[Juhazi]

Stereophile has changed it's measurement protocol

Steve Temme of Listen Inc. is now doing loudspeaker measurement for S, with very different and more modern presentation

SVS Ultra Evolution Bookshelf loudspeaker & SB-3000 R|Evolution subwoofer Measurements | Stereophile.com

Sidebar 3: Measurements

www.stereophile.com

 

[fpitas]

In my crude and now highly amateurish way, I simply take 3dB or so off their ~120Hz region to get a basic impression of a speakers' possible bass performance. The extreme top however, often tallies fairly well with other more thorough measurements I think, especially those of speakers with hf that takes off on axis...

It's the HFN plots that look truly suspect to me...

I seem to recall JA saying his measurements had a lump down there of about 3dB.

 

[fpitas]

EQ is no longer something only recording studios can afford. So, unless the polar plots are completely off-the-wall, I'd think people who care would measure and EQ. Certainly taking down room-induced bass peaks is worth a little time.

That's not to say I would buy a speaker with a crazy on-axis FR. That to me indicates either incompetence or snake oil.

 

[Mort]

Though Stereophile provide measurement data for their speaker reviews, it is well known that their method of graphing anechoic response (splicing gated nearfield data), results in some inaccuracies, especially at low frequencies.

To get a feel for how large these inaccuracies are, I looked through their list of speaker reviews, checked for which speakers I could find true anechoic response data, then plotted the difference between them.

Since Stereophile's graphs use a 30° horizontal AVG, I used Listening Window data as reference (not a perfect match, but better than on-axis).

Here are all 45 response errors with the AVG error in red:
View attachment 469059

Three distinct groups are visible:

• Bass shelf:
  View attachment 469058
• Shelf+midrange scoop:
  View attachment 469057
  
• Accurate:
  View attachment 469056

In hopes of finding trends, each measurement title contains the year in which the speaker was tested by Stereophile, as well as the type (B=Bookshelf, T=Tower).

Unfortunately I did not find any trends related to review date or speaker type.

Though in general, the response starts to deviate below 300Hz.

Attached below is the .MDAT with all individual error curves, as well as a folder containing the raw response curves in .csv, traced from JPGs/PNGs or directly from Amir's Zips.

Enjoy!

Spoiler: Sources

Floor Loudspeaker Reviews | Stereophile.com

stereophile.com

Stand Loudspeaker Reviews | Stereophile.com

stereophile.com

A collection of loudspeakers measurements

www.spinorama.org

Kinda useless and completely fascinating. Thank you!

 

[Beave]

While I appreciate that Stereophile is still providing measurements after JA's retirement, I believe that the new measurements need considerable improvement in how they are presented.

--The impedance graph is poorly scaled for the phase. There is no need to have the chart go from +180 degrees to -180 degrees. It should be +90 to -90. As it is, it's hard to discern anything useful from the phase plot.

--The frequency response at 1m is ranged from 20Hz to 40kHz. I see no need to go so high. 30kHz is plenty high enough, and it could be argued that even 20kHz is sufficient (the only reason to go above that being to see a tweeter's ultrasonic resonance). Having this scale also blurs information in the more relevant/audible frequency ranges.

--I like that they still include individual driver responses, but the scale going to 40kHz is still unnecessary. The woofer nearfield response doesn't show the dip that occurs due to the port. Why not? And where is the port output? It almost looks like it's actually not a nearfield response of just the woofer but instead a response of the woofer and port together at farfield.

--The impulse response graph is improperly scaled and impossible to interpret as provided. It goes from -1ms to +10ms, but in the text above they state that the first reflection off of the floor occurs at 5ms. So why show the plot all the way to 10mS? And if the first reflection occurs at 5ms, where is it in the plot? A better range for this plot would be -1ms to 5ms, or even to 3ms.

--The waterfall/CSD plot is shown at a weird angle.

--The horizontal polar response should be provided taller, and the color mapping used needs improvement. Also, it apparently goes all the way to 45kHz, which is entirely too high and unnecessary, thus reducing amount of info that can be gleaned from the more important audible range.



"Thank you for your attention to this matter."

 

[SigbergAudio]

While I appreciate that Stereophile is still providing measurements after JA's retirement, I believe that the new measurements need considerable improvement in how they are presented.

--The impedance graph is poorly scaled for the phase. There is no need to have the chart go from +180 degrees to -180 degrees. It should be +90 to -90. As it is, it's hard to discern anything useful from the phase plot.

--The frequency response at 1m is ranged from 20Hz to 40kHz. I see no need to go so high. 30kHz is plenty high enough, and it could be argued that even 20kHz is sufficient (the only reason to go above that being to see a tweeter's ultrasonic resonance). Having this scale also blurs information in the more relevant/audible frequency ranges.

--I like that they still include individual driver responses, but the scale going to 40kHz is still unnecessary. The woofer nearfield response doesn't show the dip that occurs due to the port. Why not? And where is the port output? It almost looks like it's actually not a nearfield response of just the woofer but instead a response of the woofer and port together at farfield.

--The impulse response graph is improperly scaled and impossible to interpret as provided. It goes from -1ms to +10ms, but in the text above they state that the first reflection off of the floor occurs at 5ms. So why show the plot all the way to 10mS? And if the first reflection occurs at 5ms, where is it in the plot? A better range for this plot would be -1ms to 5ms, or even to 3ms.

--The waterfall/CSD plot is shown at a weird angle.

--The horizontal polar response should be provided taller, and the color mapping used needs improvement. Also, it apparently goes all the way to 45kHz, which is entirely too high and unnecessary, thus reducing amount of info that can be gleaned from the more important audible range.



"Thank you for your attention to this matter."

You should send them an email.

 

[LiveTek]

While I appreciate that Stereophile is still providing measurements after JA's retirement, I believe that the new measurements need considerable improvement in how they are presented.

--The impedance graph is poorly scaled for the phase. There is no need to have the chart go from +180 degrees to -180 degrees. It should be +90 to -90. As it is, it's hard to discern anything useful from the phase plot.

--The frequency response at 1m is ranged from 20Hz to 40kHz. I see no need to go so high. 30kHz is plenty high enough, and it could be argued that even 20kHz is sufficient (the only reason to go above that being to see a tweeter's ultrasonic resonance). Having this scale also blurs information in the more relevant/audible frequency ranges.

--I like that they still include individual driver responses, but the scale going to 40kHz is still unnecessary. The woofer nearfield response doesn't show the dip that occurs due to the port. Why not? And where is the port output? It almost looks like it's actually not a nearfield response of just the woofer but instead a response of the woofer and port together at farfield.

--The impulse response graph is improperly scaled and impossible to interpret as provided. It goes from -1ms to +10ms, but in the text above they state that the first reflection off of the floor occurs at 5ms. So why show the plot all the way to 10mS? And if the first reflection occurs at 5ms, where is it in the plot? A better range for this plot would be -1ms to 5ms, or even to 3ms.

--The waterfall/CSD plot is shown at a weird angle.

--The horizontal polar response should be provided taller, and the color mapping used needs improvement. Also, it apparently goes all the way to 45kHz, which is entirely too high and unnecessary, thus reducing amount of info that can be gleaned from the more important audible range.



"Thank you for your attention to this matter."

Im glad someone pointed this all out.

Gotta be honest, dont really at all care about data from stereophile. So many outlets dont know how to measure a speaker right and present the data correctly, I just dont have the energy to entertain those who havent caught up yet. Are my expectations too high?

 

[SigbergAudio]

Im glad someone pointed this all out.

Gotta be honest, dont really at all care about data from stereophile. So many outlets dont know how to measure a speaker right and present the data correctly, I just dont have the energy to entertain those who havent caught up yet. Are my expectations too high?

Stereophile is one of the relatively few review outlets that actually provide measurements. If we care about that, shouldn't we applaud it - and if needed suggest adjustments/improvements rather than go "I can't be bothered." ?

 

[LiveTek]

Stereophile is one of the relatively few review outlets that actually provide measurements. If we care about that, shouldn't we applaud it - and if needed suggest adjustments/improvements rather than go "I can't be bothered." ?

Id totally agree, if they were just some hobbyist blog.

I applaud when folks present data appropriately, which isnt hard to do. I get very annoyed at commercial operations that cant even match what your average DIY'r is doing.

I stopped caring about anything stereophile put out after reading their comments on the grimm ls1 previews, and then seeing the data that completely contradicts what they said. IMO they are amatuers. Nothing of value was lost.

 

[john61ct]

So they are gone?

I haven't looked at anything from them since before the Eternal September of the last millenium's internet, my Dad (born in the millenium before that!) was a subscriber...

 

[norman bates]

Check out stereophiles's review on thiel cs3.

I came across these monsters in a used store in Columbus Ohio for a surprisingly reasonable price.
I've owned thiels before (model 4 and cs2).
Flipping through different speakers, I could hear something was wrong in the upper voice.
I then hopped on the internet, found stereophile's review/measurements...
There is a 10db hole near 700hz....
"It was never audible" they said.....

Thiel CS3.6 loudspeaker Measurements part 2 | Stereophile.com

The cumulative spectral-decay, or "waterfall," plot reveals how a loudspeaker behaves over time after being excited by an impulse. The MLSSA system takes FFTs at discrete slices of time as the impulse decays. Each horizontal line is a separate FFT. When viewed in three dimensions (amplitude...

www.stereophile.com

 

[SigbergAudio]

Check out stereophiles's review on thiel cs3.

I came across these monsters in a used store in Columbus Ohio for a surprisingly reasonable price.
I've owned thiels before (model 4 and cs2).
Flipping through different speakers, I could hear something was wrong in the upper voice.
I then hopped on the internet, found stereophile's review/measurements...
There is a 10db hole near 700hz....
"It was never audible" they said.....

Thiel CS3.6 loudspeaker Measurements part 2 | Stereophile.com

The cumulative spectral-decay, or "waterfall," plot reveals how a loudspeaker behaves over time after being excited by an impulse. The MLSSA system takes FFTs at discrete slices of time as the impulse decays. Each horizontal line is a separate FFT. When viewed in three dimensions (amplitude...

www.stereophile.com

Interesting commentary:

Although this dip does exist, it probably does so only at the 45" measurement distance. Getting farther away from the loudspeaker will likely result in better integration of the individual drivers' outputs, making the suckout disappear.

Why didn't they move further away and measured again to verify this theory?