Complaint thread about speaker measurements

[JohnBooty]

I wouldn't buy a speaker or an amplifier based on the content of the ASR reviews.

Well, let's be clear.

Is that literally the goal here? To be able to buy a speaker solely on the basis of some data?

I would say "no" but maybe I'm not being ambitious enough.

These are factors that Amir can't/won't test in his garage system. Maybe not in his lounge-room either. So where does that leave people?

Currently?

I would certainly use ASR's tests to rule out a bad speaker. I feel confident that something with ragged or otherwise weird response would not sound good.

ASR's current speaker tests rule out bad speakers and point us towards a much smaller pool of speakers worth considering. In a more general sense they are very educational and help to hold manufacturers accountable.

If that is "all" ASR's speaker tests ever accomplish I would call that a very worthy contribution to the field/hobby. Of course it is always good to shoot for the stars and if it's possible to accomplish more than is currently accomplished, great.

 

[amirm]

No, that's the problem. You are either properly reviewing the speakers or you are pissing around babying them for fear of possibly damaging them. That makes for a completely useless review.

The #1 criteria in speaker preference is the tonality. Nothing about that remotely requires pushing the speaker to breaking point. We are not talking about PA speakers or sound reinforcement.

 

[direstraitsfan98]

My last attempt to get Amir to verify the accuracy of his measurements.

In the upper part of the following diagram you can see six speaker measurements of Amir. Between 5kHz and 10kHz the ripple still looks too much like a pattern to me.

In contrast, measurements from my developed DIY loudspeakers.

The red curve shows a metal tweeter in a DIY wave guide measured at 180cm distance. In green the measurement at a distance of 30cm (with very slightly changed crossover). Nice to see, the ripple decreases when the microphone is closer to the speaker.

The blue curve shows the same DIY wave guide, but with a ring-dome tweeter. Since the wave guide has a decisive influence on the frequency response, the ripple is very similar to the red curve.

The orange curve shows a different DIY speaker. The tweeter used has its wave guide already integrated. The ripple of the measurement shows no similarity at all to the blue and red curves.

View attachment 47808

A discourse with Amir about his measurements will be fruitless. He does not want to change or implement what he's comfortable with. Please note how he facetiously created this new thread as well as it being a form of censorship. I am honestly surprised that some of my prior posts calling Amir out for being so closed minded in regards to feedback on his measurement system have not been moderated already! I appreciate your effort into calling the discrepancy out, its a shame it seems to get buried. This place can be quite an echo chamber at times.

 

[stunta]

I don't understand why you would apply a filter and negate evaluating the speaker's roll-off. After all, the roll off is part of the speaker design.

As for listening tests, I find them very useful as a sanity check and to corroborate the measurements. My listening evaluations tend to agree with yours, Amir, so that helps.

Suggestion: Perhaps a poll is in order to see which measurement/graphs are really needed (I won't vote cos I have no clue) and Amir can prioritize and save some time.

 

[North_Sky]

About listener sound tonality personal preference? ...And that goes as well with the type of music he/she prefers to listen to.
Can we measure personal listener preference of sound, tonality?
Can we measure pleasurable emotional brain pulses from various type of music reproduced by various mechanical loudspeakers connected to various audio gear including the sources...both digital and analog?

 

[RayDunzl]

If it's not a 10 then I don't want it.

*does not apply to anything I already have in or near the rack

 

[amirm]

A discourse with Amir about his measurements will be fruitless. He does not want to change or implement what he's comfortable with. Please note how he facetiously created this new thread as well as it being a form of censorship. I am honestly surprised that some of my prior posts calling Amir out for being so closed minded in regards to feedback on his measurement system have not been moderated already! I appreciate your effort into calling the discrepancy out, its a shame it seems to get buried. This place can be quite an echo chamber at times.

 

[sweetchaos]

Suggestion:
Amir can write a quick review with his usual "Spinorama Speaker Measurements" and place a place-holder on the 2nd post for more advanced measurements at a later time? I think that would satisfy both sides, those who want minimal charts and those who want *all charts*.

 

[North_Sky]

What's the best bias?

 

[majingotan]

I'm already in audio nirvana with a mediocre 2.8 preference scoring Yamaha HS7 (should've similar performance to HS5). Have heard the much higher 4.5 scoring KEF LS50 and I still prefer the HS7 to my subjective preferences

 

[Sancus]

I personally don't care about listening tests at all.

As far as the graphs go, I feel strongly that contour maps are valuable, but if they had to go, I could live with just the attached data being posted(+/- 180 at 10 degree resolution). Spinoramas are less interesting to me simply because they're far more common... good contour maps are harder to find and high resolution(5 degree or less) ones basically don't exist from any other source I've seen.

I'm fine with the current review pace as long as you don't have so many speakers piling up that it's causing a storage issue or anything, we're in really no rush here. We're getting great data that will stand and be linked to for a long time, why not take our time and make it the best?

In general, I like jhaider's checklist.

 

[Trouble Maker]

Not a complaint, but a question.
I've noticed you comment a few times about the temperature in your garage.
Does the temperature effect the results? I come from a test background where it does, but very different fields so it's mostly a curiosity.

And on that note a suggestion for personal comfort.
You can insulate any external walls, drywall, either get an insulated garage door (kind of expensive) or add insulation to your current one. Then add a heater out there, you could also do A/C, but maybe not a big issue where you are. At least it will be a ton more comfortable for you to work out there!
There are also many flooring options that can make it a nicer place to work in too; etching, painting or even garage floor tiles. The floor tiles seem like a PITA since I think mostly you'll want to pull them up and clean occasionally, but other than that they do seem like the nicest option.

Thank you for what you are doing and keep up the good fight!

 

[Wombat]

FWIW: I have the same philosophy on this subject that I espouse under circumstances when one is provided with free food.

One should never complain about food when it is free! I like to say.
In terms of measuring loudspeakers, it is a safe bet that (you + Klippel) know more than I. Far more than I.

EDIT: Oh, I should acknowledge that I do realize it ain't actually free! But I hope you all catch my drift. This is a service being offered to a community; it's kind of hard to complain!

Don't confuse complaining with disagreement or difference of opinion.

No, that's the problem. You are either properly reviewing the speakers or you are pissing around babying them for fear of possibly damaging them. That makes for a completely useless review.

Both speakers and amplifiers need to tested to determine their rated performance limits. Presently, this is not done and the reviews are therefore incomplete. In my opinion, they do not give an remotely accurate representation of the actual real-world performance of the products, and as such, they have limited value. Certainly to me. I wouldn't buy a speaker or an amplifier based on the content of the ASR reviews. I would however buy a D/A converter based solely on the ASR reviews as they are comprehensive.

Speakers can be tested with tone-bursts if you're scared of blowing them up. Enough excuses about not wanting to damage something. It's BS Amir. I haven't blown up a decent amplifier in 35 years testing. Speakers need to be tested against their specifications and that includes a maximum power where THD remains at a reasonable level across the audible bandwidth. Compression or even a linearity test up to maximum power is needed. What happens to these little 6.5" 2 ways after 15 minutes of loud-ish music? I'll tell you, the dynamic compression/distortion goes through the roof. These are factors that Amir can't/won't test in his garage system. Maybe not in his lounge-room either. So where does that leave people?

With active speakers, you are testing the amplifier/s as well. You cannot make a recommendation on the product if you don't even test part of it. They should be run at 1/5-1/3 rated output (measure it at the driver terminals) for an hour prior to all the testing. Give them pink noise, put them in a padded box and go shopping if it's too loud. If they blow up, they are rubbish and should be called for what they are.

Who cares if a speaker is dead flat at 1W or 5W but completely falls apart to distort, pole the voice coil or lose a tweeter when the level is increased? How linear is the speaker at various input levels up to rated power? Does the speaker produce huge amounts of frequency doubling at LF or port noise at medium/high powers? Is the rated power the manufacturer quotes realistic or complete fantasy?

This is what we need to know.

Simple solution. If the lender is not comfortable with maximum full load testing, then don't provide the test item.

 

[restorer-john]

A discourse with Amir about his measurements will be fruitless. He does not want to change or implement what he's comfortable with. Please note how he facetiously created this new thread as well as it being a form of censorship. I am honestly surprised that some of my prior posts calling Amir out for being so closed minded in regards to feedback on his measurement system have not been moderated already! I appreciate your effort into calling the discrepancy out, its a shame it seems to get buried. This place can be quite an echo chamber at times.

Sadly, this is true.

I've been sitting back from the speaker reviews deliberately not participating because criticism or suggestions by members is usually just completely ignored or dismissed. Standard modus operandi is to ignore an entire, carefully thought out and presented post, or focus on one tiny part part and declare it unimportant with no justification whatsoever.

So an alternate thread on "criticisms" is started and even in that thread, the same thing happens...

The #1 criteria in speaker preference is the tonality. Nothing about that remotely requires pushing the speaker to breaking point. We are not talking about PA speakers or sound reinforcement.

The number one criteria for selecting a speaker is fitness for purpose. Tonality means absolutely nothing if the speaker cannot fulfill its primary duty of filling an area with sound at a level the buyer wants. That varies massively depending on the application. A 6.5" two way on a desk vs a speaker filling a vaulted ceiling warehouse apartment are completely different applications demanding different speakers.

Power handling ability across the spectrum is phenomenally important. You cannot dismiss it and expect credibility going forward. You need to test the SPL/THD capabilities and power handling abilities of your reviewed speakers. Go read some audio magazines from the 1980s and 1990s and see how they tested every single loudspeaker for power handling. Did they blow them up? Nope.

Buy some cheap sacrificial speakers from a thrift store and see what it takes to destroy woofers on single cycle tone-bursts. You'll be surprised. Your amplifier will clip long before the speaker expires.

So here are a few speakers (truly randomly selected) from early 90s reviews. One can absorb 1200 watts @100Hz single cycle and the other rattles at 55 watts due to a dodgy passive radiator. Both medium/large floor standers. Both were a similar price. Big difference in power handling ability and a huge differentiator wouldn't you say? How would your current testing regime uncover these differences?

 

[JohnBooty]

Can we measure pleasurable emotional brain pulses from various type of music reproduced by various mechanical loudspeakers connected to various audio gear including the sources...both digital and analog?

Pretty much. You play speakers for people and ask them how much they enjoyed them.

This is the basis of much, much research conducted by Harman and others.

I don't think you will find any kind of music that benefits from a weird, ragged response or dispersion. So ASR tests are good for ruling out poor speakers for any genre of music. They are useful for that.

Of course I'll be the first to admit that there are aspects of listening enjoyment that can't be measured in this way. If you have never heard a pair of properly set up Magnepan speakers driven with solid amplification, they are really incredible in ways that I'm certain could not be captured by these kinds of measurements. (I would be happy to be proven wrong) And the excellently-measuring JBL 305Ps, while very enjoyable in my book, would not be many people's idea of an endgame speaker to say the least.

 

[Blumlein 88]

Pretty much. You play speakers for people and ask them how much they enjoyed them.

This is the basis of much, much research conducted by Harman and others.

I don't think you will find any kind of music that benefits from a weird, ragged response or dispersion. So ASR tests are good for ruling out poor speakers for any genre of music. They are useful for that.

Of course I'll be the first to admit that there are aspects of listening enjoyment that can't be measured in this way. If you have never heard a pair of properly set up Magnepan speakers driven with solid amplification, they are really incredible in ways that I'm certain could not be captured by these kinds of measurements. (I would be happy to be proven wrong) And the excellently-measuring JBL 305Ps, while very enjoyable in my book, would not be many people's idea of an endgame speaker to say the least.

If you have never heard a pair of properly set up Magnepan speakers driven with solid amplification, they are really incredible in ways that I'm certain could not be captured by these kinds of measurements. (I would be happy to be proven wrong)

I very much doubt whatever you like about Maggies would be some artifact not captured by these measurements.

I agree with the comment that tonality is the key quality of speakers. How do you think these Maggies sound:

And this sort of thing is more or less common to the line. Try listening to some outside an open door of the room or in an adjacent room. You'll hear more clearly the response in that graph above.

Off axis isn't very helpful to the cause of a tonally neutral faithful transducer either.

Both of these are the 3.6 R for those wondering. It makes it easy to think in blind conditions this speaker vs a relatively smooth evenly toned speaker will stand out like a couple of sore thumbs. And likely not sound appealing via immediate comparison.

Now don't get me wrong, I've owned some Maggies and the bulk of my life I've owned and listened to ESL panel speakers. So while I love them, you can't tell me the uneven response in direct comparison is a good thing. Nor that some of the wow factor is not contained in these measurements.

For fairness here is a Quad 989 which is quite similar to the Quad 2805 and 2812. Again that uneven FR is going to be very obviously heard vs a smooth responding speaker. Off axis is maybe better than a Maggie.

If you want to do something interesting some time, if you have a smooth responding or corrected speaker, try EQing a few tracks to match the curve displayed for frequency response of a Quad or Maggie. Listen to it. It won't sound as bad as it looks actually. Some kinds of music even seem exciting and better with it. Being able to turn that response on and off it is obviously very different. Obvious to me that it is a coloration too.

 

[digicidal]

Although it might be meaningless in this context, I can't help but compare ASR to the other forums/review publications I frequently read... those dealing with equally "hot" topics... cars and aftermarket performance. While most, if not all, manufacturers will sacrifice dozens of samples in the interest of testing theoretical limits in reality... most reviewers are contractually prevented from this. In @amirm's case, it's not the manufacturer, but the equipment owner that's likely to either directly (or at least implicitly) express a desire to have a working device returned.

The limits in my example abound... but rarely in the published reviews - instead it's the build-logs and race-logs of amateurs who have (inadvertently in many cases) discovered the conclusive proof of engineering limitations of a particular part and/or vehicle. Usually this involves explosions and flying pieces of metal. In a few tragic cases, we get hazy details from their surviving friends and relatives... in the rest we get real usable data, unavailable from either the manufacturer or the press.

So while it might be "proper" or "comprehensive" to measure all data points up to and including unexpected failures in equipment "under extreme conditions" - I don't see ASR providing these unless it's making millions in revenue somehow. If @amirm was purchasing everything himself (in multiples) then it would be a different story, but then it would also likely require manufacturer sponsorship... and would become just like the aforementioned car reviews. Either that or we'd be lucky to get a single review per month - as opposed to nearly one per day.

I can definitely see value in the kinds of comprehensive stress testing and edge-use-cases often pointed out by several members, but at the end of the day there's already a dearth of objective review sites and even fewer that aren't entirely controlled by the manufacturer's interests (by declaring publishing restrictions if nothing else). I'll take less comprehensive (or even partially flawed) reviews over paid shilling or no information whatsoever.

 

[TimVG]

@amirm

Is it correct to assume that the klippel system, while measuring in the nearfield, simulates the data in the farfield, and that this is the reason the precise acoustical axis doesn't matter as much as it would when portraying the actual nearfield data?

 

[JohnBooty]

I agree with the comment that tonality is the key quality of speakers. :

I think tonality is certainly key. If you don't have that, you don't have anything. However... assuming we are listening to music and not test tones... correct tonality in some parts of the frequency spectrum matters much more than in others.

How do you think these Maggies sound?

I'd feel pretty confident that music sounds fundamentally tonally correct but details would be what I guess folks call "sparkly" which may or may not be pleasing.

It makes it easy to think in blind conditions this speaker vs a relatively smooth evenly toned speaker will stand out like a couple of sore thumbs. And likely not sound appealing via immediate comparison.

I don't think that's cut and dried. This is an answer where we benefit from thinking about music rather than engineering a little more. The highest note sung by a soprano has a fundamental frequency of ~880hz and obviously most vocals are below that. The highest note on a piano has a fundamental frequency of ~4000hz but obviously most musical content will be below that point.

Looking at the waterfall graphs of the Quads and the Maggies, they seem seem to have a pretty smooth frequency response in that key < 1000hz region where the majority of music is happening.

I'd be interested to know what the sum of the Magnepans' on- and off-axis output looks like, ala the "predicted in-room response" graphs from ASR's reviews, as that's more or less what we're actually hearing. Not saying it'd make a better case for them, just saying I'd be curious. Or do the waterfall graphs include that?

 

[restorer-john]

So while it might be "proper" or "comprehensive" to measure all data points up to and including unexpected failures in equipment "under extreme conditions" - I don't see ASR providing these unless it's making millions in revenue somehow.

Do not for one second assume that testing to specified performance limits, implies destruction. It does not. I do it all the time with amplifiers.

An amplifier that fails to achieve its rated specifications deserves to be lambasted in no uncertain terms. A speaker that fails to achieve its rated performance characteristics needs the same level of derision.

A product that exceeds its rated and advertised specifications also needs to be acknowledged.