Dubious about SPL used by objective testers

[Justin Ayers]

Our loudspeaker measurements are performed by the prestigious National Research Council of Canada. The NRC’s facilities include a modern anechoic chamber and precision measuring devices, along with staff with decades of experience conducting these tests. All measurements are performed separate from the subjective evaluation -- the body of the review. In all, we perform a total of eight tests displayed on five charts to give perspective into the measured performance of the loudspeakers under evaluation.

link

Please note: an SPL level of 90dB measured anechoically is very loud and considered far beyond normal listening levels, particularly for small loudspeakers. To give more information for real-world listening levels, if it appears that the speaker is being strained beyond its output abilities at this level we will provide a second measurement at at lower SPL (the SPL level will be printed with the chart).

Purpose: Measures THD+N output at discrete frequency intervals for above-normal listening levels. Please note that 90dB output at a 2-meter distance is equivalent to an SPL level of 96dB at a 1-meter distance.

What it tells you: Audibility of distortion varies as to type of distortion and also the frequency at which it is occurring. Distortion measurements for loudspeakers are usually many times that of electronics (i.e., amplifiers, receivers, etc.). Furthermore, certain types of distortions are more audible than others and the audibility of that also depends on the frequency. Our distortion measurements give a general indication of how much distortion is occurring for a given output level at above normal listening levels. Distortion levels will be less (sometimes much less if the speaker is being stressed beyond capabilities at 90dB) at lower SPLs

I am dubious about at least one aspect of this methodology. For consistency, all speakers should have the lower-decibel chart in their review. Leaving it up to humans to decide, subjectively, if "it appears that the speaker is being strained beyond its output abilities at this level" seems to be an avoidable variable where error can creep in.

I also am unclear about the value of providing a measurement of something "very loud and considered far beyond normal listening levels".

Our distortion measurements give a general indication of how much distortion is occurring for a given output level at above normal listening levels.

Why? The only value I can see here is for when a speaker is being used like PA equipment. If PA performance is important why aren't PA speakers in the test results, like anything from Peavy?

Distortion levels will be less (sometimes much less if the speaker is being stressed beyond capabilities at 90dB) at lower SPLs.

Making your test potentially irrelevant and potentially misleading (by placing value in something that has little to no value as a consumer comparative guide indicator of speaker quality).

I recall earplugs being mentioned in a review on this site, to compensate for an uncomfortably loud testing level. I simply don't understand where the value is in testing speakers by stressing them beyond the workload they will encounter. If they're PA speakers that's a different matter, since things that are important to most audiophiles (such as safe and comfortable listening levels) aren't always important at all in that arena. There is also the edge case of home hifi stuff being used for loud house parties but that's stretching things too far in my view. If a person needs PA speakers there are plenty of affordable options out there.

I have heard the justification that the measuring equipment needs a high volume to get useful data. But, is the data actually useful?

edit: corrected quote attribution

 

[Sancus]

liI simply don't understand where the value is in testing speakers by stressing them beyond the workload they will encounter.

Well, they aren't. I don't know where this idea that 96dB@1m is "so loud" has come from, exactly, other than perhaps the fact that most cheap, small 2-ways cannot handle it. Music has dynamic range. Film/TV has even more, beyond most music(other than some types of classical). If you listen at an "average" level of 75dB, it's entirely possible to encounter material that will briefly push the speaker to 105dB. It's not extremely common, but it does exist. And that's at 1m. If you go out to 3m(a common household listening distance) you're already looking at -6 to -9dB of loss depending on how you do that math/account for reflections.

As far as I understand, these measurements are typically a sweep. A single sweep's pretty short. If anything, that's fairly generous to the speakers and only tells you about brief peaking behaviour. The kind of multi-tone, high spl compression testing that is done in @hardisj's reviews seems like what is necessary to really discern speaker limitations.

 

[Justin Ayers]

Well, they aren't. I don't know where this idea that 96dB@1m is "so loud" has come from, exactly, other than perhaps the fact that most cheap, small 2-ways cannot handle it. Music has dynamic range. Film/TV has even more, beyond most music(other than some types of classical). If you listen at an "average" level of 75dB, it's entirely possible to encounter material that will briefly push the speaker to 105dB. It's not extremely common, but it does exist. And that's at 1m. If you go out to 3m(a common household listening distance) you're already looking at -6 to -9dB of loss depending on how you do that math/account for reflections.

As far as I understand, these measurements are typically a sweep. A single sweep's pretty short. If anything, that's fairly generous to the speakers and only tells you about brief peaking behaviour. The kind of multi-tone, high spl compression testing that is done in @hardisj's reviews seems like what is necessary to really discern speaker limitations.

These are the words of NRC:

very loud and considered far beyond normal listening levels

So, it appears the problem is that NRC's description is strongly misleading. Should it be "very loud and considered far beyond normal listening levels from the testing distance but not from a common human listening distance"?

Thanks for the clarification. I didn't see how wearing earplugs during testing and testing speakers at levels "very loud and far beyond normal listening" would be useful. But, the drop in level at distance seems to explain the situation.

edit: Corrected apparent dyslexia

 

[pozz]

Just be to clear. It's not "NRC" aka the National Research Council of Canada that's saying those things, but the writers working for SoundStage!, an audio magazine. They make measurements in the NRC's facilities.

To my eye, the parts of the article you highlighted read sort of disclaimerish, in the sense that they are attempting to justify the SPL they use for comparisons, and why it might be unfair for some speakers but well below the capabilities of others. I wouldn't take it too seriously. SoundStage! measurements are good, accurate, but not comprehensive.

 

[Sancus]

Thanks for the clarification. I didn't see how wearing earplugs during testing and testing speakers at levels "very loud and far beyond normal listening" would be useful. But, the drop in level at distance seems to explain the situation.

Another thing to consider is that these are flat sweeps, typically, which is also not like regular music. Music has a very downsloping loudness level, here's a graph from a study that checked a wide range of music:

It's one of the reasons that tweeter loudness/power is basically irrelevant, as by 3khz you are looking at -20db down if not even more than that for most material.

Our ears are a lot less sensitive to low frequencies than high frequencies. 96dB at 100hz is easily tolerable, but I would not be excited to hear a 96dB sweep at 3-5khz. If you play music with an SPL meter and see peaks at 90 or 100dB, that will almost always be in the range from 50-500hz. Never much beyond that.

So the other answer is that a 96dB sweep IS dangerously/uncommonly loud... but only at high frequencies.

 

[Justin Ayers]

Just be to clear. It's not "NRC" aka the National Research Council of Canada that's saying those things, but the writers working for SoundStage!, an audio magazine. They make measurements in the NRC's facilities.

To my eye, the parts of the article you highlighted read sort of disclaimerish, in the sense that they are attempting to justify the SPL they use for comparisons, and why it might be unfair for some speakers but well below the capabilities of others. I wouldn't take it too seriously. SoundStage! measurements are good, accurate, but not comprehensive.

I see. I still question the methodology of only posting the second SPL measurement "if it appears that". I would think it would be less error-prone to measure and post the data for all speakers rather than relying on subjectivity in that regard.

To give more information for real-world listening levels, if it appears that the speaker is being strained beyond its output abilities at this level we will provide a second measurement at at lower SPL (the SPL level will be printed with the chart).

This also implies that the normal SPL test is excessive, which led to my questioning of it.

in the sense that they are attempting to justify the SPL they use for comparisons, and why it might be unfair for some speakers but well below the capabilities of others

Doesn't that reinforce my questioning, though? How can it be unfair if it's a level that's quite a lot below what normal listening can subject speakers to?

Perhaps what it should be is "normal listening for small nearfield speakers versus normal listening distance for tower speakers"? If the speakers are meant to sit by one's computer they don't need to be capable of the SPL of those being listened to from many feet away. It seems that SoundStage could improve the description but I also would like to see the reduced SPL test for all speakers tested.

 

[pozz]

that the normal SPL test is excessive,

It is excessive for some speakers, like small ones, that cannot handle those kinds of levels.

Take a really good speaker like the JBL M2. It can handle outputting 120dB SPL in some part of the spectrum without problem.

Also, if you don't play loudly, you won't be able to excite distortions in some speakers to meaningful levels. So it's not straightforward.

Like I said, their measurements are good, but not comprehensive.

 

[Justin Ayers]

It is excessive for some speakers, like small ones, that cannot handle those kinds of levels.

Take a really good speaker like the JBL M2. It can handle outputting 120dB SPL in some part of the spectrum without problem.

Since distortion is reduced at the lower SPL level (90 decibels as I recall) then why not simply test every speaker with that level (in addition to the other higher-SPL test) and post the data? That seems to eliminate guesswork about whether or not a speaker is being overly stressed. It also gives people an idea in terms of to what degree distortion will be reduced in larger speakers at the lower SPL, a level people with hearing issues may spend a lot more time with.

Also, if you don't play loudly, you won't be able to excite distortions in some speakers to meaningful levels. So it's not straightforward.

Like I said, their measurements are good, but not comprehensive.

Just caught that addition to the post. Not getting distortion at the lower SPL would be useful information.

 

[pozz]

Since distortion is reduced at the lower SPL level (90 decibels as I recall) then why not simply test every speaker with that level (in addition to the other higher-SPL test) and post the data? That seems to eliminate guesswork about whether or not a speaker is being overly stressed. It also gives people an idea in terms of to what degree distortion will be reduced in larger speakers at the lower SPL, a level people with hearing issues may spend a lot more time with.

It's definitely subjective on their part.

But then they aren't heavily invested in pure method. Their magazine still publishes stuff about power conditioners and so forth.

 

[Justin Ayers]

It's definitely subjective on their part.

But then they aren't heavily invested in pure method. Their magazine still publishes stuff about power conditioners and so forth.

I thought the NRC is the one doing the testing. Are they passively doing only the testing requested by SoundStage? In that case, their reputation for being "prestigious" and having engineers doing the work who have "decades of experience" could be compromised. I assumed that the NRC experts were the ones deciding what measurements are important and how to conduct them.

 

[pozz]

I thought the NRC is the one doing the testing. Are they passively doing only the testing requested by SoundStage? In that case, their reputation for being "prestigious" and having engineers doing the work who have "decades of experience" could be compromised. I assumed that the NRC experts were the ones deciding what measurements are important and how to conduct them.

I haven't looked into the details. You could ask @Doug Schneider.

I'm sure there is some back-and-forth, but the editors has the final say about what's published and what's most meaningful. One of the most difficult things in publishing is satisfying multiple kinds of audiences simultaneously. You will have people writing you and demanding more graphs and others who ask to put that content to the side, sometimes very emotionally. It's not a scientific publication after all, but it's good that they take the time to make those measurements, despite whatever reservations a part of their readership or they themselves might have about their efficacity, and have also prepared lists online of the tests. This is better than the Stereophile approach, which does not keep a separate listing or database, and bury the measurements in reviews.

 

[Sancus]

If you test all speakers the same way, your tests are going to be unfair/dangerous/bad for some subset of those speakers. If you test them differently, then you introduce complexity and a judgement call about which ones to subject to the harder tests that some may deem "unfair".

There's no obvious/easy choice here to make, especially if you are trying to keep your tests to simple sweeps. If you are willing to get into more complex test batteries, however, you are also adding much more work and/or very expensive equipment(Klippel stuff).

 

[Justin Ayers]

If you test all speakers the same way, your tests are going to be unfair/dangerous/bad for some subset of those speakers. If you test them differently, then you introduce complexity and a judgement call about which ones to subject to the harder tests that some may deem "unfair".

What speakers would be disadvantaged by doing the lower-SPL test for all of the tested speakers?

The only ones I can think of are those that have higher distortion at the lower SPL. That's not unfair, though. That seems to be quite useful information. Speakers that distort more at the lower SPL level and which have all other characteristics identical to a competing model can be discarded from purchase consideration.

The only other possibility is that some designs don't perform as well at lower SPLs, which is also something people will need to know. I have read a number of claims about certain speakers not sounding good at lower SPLs — various planar speakers in particular. I don't know if that's true or not. Having consistent SPL-based distortion testing would help to eliminate distortion being a question mark in that regard. It wouldn't address levels lower than 90 decibels but it would be much more information in that regard than only having the higher SPL data.

I can't see any strong justification for choosing to test only certain speakers at a lower SPL, while testing all of them at the higher SPL. That's even without the added unnecessary problem of relying on subjectivity to decide which speakers to test with both SPL levels.

 

[Sancus]

What speakers would be disadvantaged by doing the lower-SPL test for all of the tested speakers?

The ones that can play louder but which don't do better on the other tests would be disadvantaged. And that's not always obvious, either. For example the JBL stage A130 can play surprisingly well at 96dB, whereas the larger woofer 308p does much worse. Distortion is also non-linear so you can't accurately estimate higher SPL results from lower SPL results except in a very rough fashion.

Personally, I find sub-90dB distortion tests to be largely useless. Most speakers that are otherwise well designed do fine at those levels. Some do not, but you don't really learn much about distortion in most cases until you push a speaker IMO.

If you legitimately never push your speakers(lets say, 60dB avg/80dB peak listening levels) then distortion testing is pretty irrelevant to you in general, I would say. But I don't really know how people who do that enjoy their music

E: And yes, if you don't test ability to play loud, people will complain. This is, for example, at the root of why people get upset when a large floorstander doesn't "score better" than a small bookshelf when they are equally smooth on/off axis.

 

[ernestcarl]

... people get upset when a large floorstander doesn't "score better" than a small bookshelf when they are equally smooth on/off axis.

Oh, yeah... It’s so human to overestimate the value of scores and “awards”. Quite typical of us as a species really. We see this everywhere in the internet: top ten this or that product, best of 2020, top student, top sales guy, etc. The list is never-ending. LOL

Reality, of course, is so much more complicated.

 

[Justin Ayers]

The ones that can play louder but which don't do better on the other tests would be disadvantaged.

No, they wouldn't.

1) All the speakers would be tested with the louder test. Speakers that "can't play louder" would be exposed by that test.

2) All the speakers would be tested with the lower SPL test as well. Speakers that distort less but also can "play louder" would be exposed by the two tests' data via comparison.

Personally, I find sub-90dB distortion tests to be largely useless.

Well, that's a different topic (changing goal posts). And, remember this exchange?

I said: What speakers would be disadvantaged by doing the lower-SPL test for all of the tested speakers?

You then said: The ones that can play louder but which don't do better on the other tests would be disadvantaged.

How is that possible, if the reduced-SPL test is useless?

Most speakers that are otherwise well designed do fine at those levels. Some do not, but you don't really learn much about distortion in most cases until you push a speaker IMO.

In most cases is where your argument is weak. Also, "largely useless".

 

[richard12511]

IMO, "90dB measured anechoically is very loud and considered far beyond normal listening levels " really depends on the listener. It certainly isn't "far beyond" my normal listening levels. If anything, it's not loud enough to show me the distortion levels I'll experience at the peaks of my normal listening levels.

I'm glad they show distortion at those levels, and I wish they would actually show distortion levels quite a bit louder. The loudest peaks I hear will be in the 105dB range at a listening distance of almost 4m, so their test is far too quiet for me, personally.

However, I do agree with you that they should show the quieter distortion measurements for all speakers, and not just the smaller ones.

 

[richard12511]

The reason they don't measure all speakers at the quieter level may be time/money related. I'm guessing they have to pay for time spent using NRC's anechoic chamber.

While I do think quieter distortion measurements would be useful data for all speakers, they're certainly more useful for speakers who can't really handle the louder test.

 

[Sancus]

No, they wouldn't.

1) All the speakers would be tested with the louder test. Speakers that "can't play louder" would be exposed by that test.

2) All the speakers would be tested with the lower SPL test as well. Speakers that distort less but also can "play louder" would be exposed by the two tests' data via comparison.

Sure, agreed. Maybe I misread. My point was only doing a distortion test below 90dB is largely useless to me.

In most cases is where your argument is weak. Also, "largely useless".

I don't think that's true. If you are trying to compare low-budget (like $200/speaker or less) two-way bookshelves only, then yeah, an 86dB distortion test has some value. But even then, there's obviously speakers that won't show much at that level. That's obviously an opinion, but I think if your test regime is only good for analyzing very low-end speakers then it's not a very good test. I would be fine with something like "we test speakers only up until they start to show serious strain." -- no need to run 105dB tests on a JBL 308p However, that complicates the test regime and requires a judgement call.

It is much better if you use some kind of automated system that cuts off at a certain threshold of compression/distortion, and that is IMO the gold standard and probably what S&R uses to do their "max SPL at 3%/10%" graph. Which is a very useful graph that adapts to the capability of the speaker.

I think the "one test at 86, one at 96" we are discussing here is just a very basic test that is simple to run on everything and will generally tell you SOMETHING -- but it's far from any sort of ideal.

In a perfect world, every speaker would be adaptively tested to the max SPL it can produce before hitting the threshold of audibility for both THD and IMD. However, since there isn't clear research on what those thresholds are, you end up picking a somewhat subjective standard(like IEC 60268-21) or an alternative of your own making.

 

[RayDunzl]

I have heard the justification that the measuring equipment needs a high volume to get useful data. But, is the data actually useful?

In my not very noisy room, with the main system, measured at the listening position, the distortion from the speakers just begins to rise out of the noise floor around 90dB SPL.

The measurement point is 10 feet from the speakers.

These THD measures - 46 to 92dB SPL

Lowest SPL yields the measure at the top - noise is interpreted as THD

Only at the highest SPL I measured, did the distortion measures begin to rise (see some bunching at the bottom)

I didn't go louder as I was satisfied with the measures at the levels tested.

(the problem around 48Hz is not distortion, but a hole in the frequency response - cancellation - at those frequencies)

Ambient noise particularly messes with the low-frequency distortion measures at lesser SPL.

SPL levels for each sweep