DO SPECS REALLY MATTER in Audio? - Understanding Speaker Measurements!

[a-gainer]

The lack of standards is a problem.

Sure! That's the point where Harman, Olive/Toole did something. But what the standard is, was determined by statistics over subjective "preferences". This guy here gets - do You say that, beaten because he expresses the relevance of, You name it, preferences?

I'm totally with You all regarding the prime importance of objective data, because we deal with a technical product. But with the statistical background of the founding studies of Olive/Toole, there still is room for just accepting deviating choices, made just for reasons unknown.

 

[Koeitje]

A big problem is the recordings themselves. If we accept that the large majority enjoys a neutral sound and regression (tilt), any deviation from this in the monitoring setup/calibration will give the inverse effect on the recording itself. The lack of standards is a problem.

Unfortunately that is not something we as consumers can do something about. Our best bet is to just buy the best loudspeakers we can afford for the money we want to spend on it and hopefully recording/mastering standards follow. The alternative is to introduce yet another random factor in the chain.

 

[Feelas]

I'm totally with You all regarding the prime importance of objective data, because we deal with a technical product. But with the statistical background of the founding studies of Olive/Toole, there still is room for just accepting deviating choices, made just for reasons unknown.

Look, I think we all agree, but still there's one more problem: the manfactureres themselves are very relcutant to tell you anything real about what they're selling. In essense, the whole audio business - even the speakers! - is one big snake-oil in itself.

One can make deviating choices, but one should know what these choices are, so he can do them all over again the future and maybe deviate if the person in question finds them limiting. That's the whole point!

 

[Helicopter]

Great to see you here @Gene DellaSala ! Welcome!

 

[BDWoody]

There is too much in it, I cannot express without too much of back and forth background argumentation, but this was meant to be fun, right?

To anyone who may not know, creating an alter-ego or multiple registrations is actively discouraged here. If you don't want to post under your 'real' name, don't post.

This name banned, original name given a week off.

 

[GGroch]

Youtuber Joe N Tell just posted his response to Andrew, defending measurements. Very well thought out I think.

I have watch a lot of Andrew Robinson's videos and I think he may have been knowingly kicking a hornets nest here. If so, he was successful.

In my experience it is very often reviewers like Joe who come from auto-sound backgrounds who best understand the benefits of measurement. They understand that being pro audio measurements is no different than being pro physical measurement. You can't be a successful auto-sound installer/designer without both.

 

[hardisj]

I commend Amir and Erin's efforts and hope we can all work together going forward.

Me, you and Amir all do a live session to talk about the data and why it matters. Between Amir and I we can pull in examples from our own tests. I'm game.

 

[napilopez]

I think the great irony of all of this is that I've found measurements are more useful to understand and interpret for audio than for many if not most other consumer products. And in some respects, they're easier to do in a meaningful way.

The site I write for is a general tech site, and indeed, most of my reviews are about different consumer tech. Speakers are like 10 percent of what I do. Phones, laptops, tablets, smart watches, ebikes, camera's, etc.

I run benchmarks on phones and PCs, but I find they're little useful for the average consumer and describing the breadth experience of what a phone or PC can do and what it feels like to use one. I'm not aware of research out there about preferences for a smartphone relating to a specific combination of performance metrics, for instance. And if a laptop has an awful keyboard and trackpad, that's going to outweigh almost any marginal performance gains it might have over other products in it's category.

But for speakers, you can learn so much and be far more confident in your purchasing choices when extensive measurements are available. For the most part, they are not multifunction devices. They have acoustics and they have aesthetics. That's it.

To @MattHooper 's point earlier, understanding measurements doesn't only mean always preferring a specific target, you can also use them to know which deviations you like, even if you have different hearing deficiencies as Andrew goes into in his video.

The real problem is dispelling the notion that measurements are not adequately descriptive of sound.

FR, directivity, and output capability are like 90-95 percent of it. And even if available measurements somehow miss that last 5-10 percent(which is almost certainly contained in other measurements of distortion, time domain, etc) that's still SO much more useful than the typical words-only review. My goal is to get readers to eventually pay more attention to my measurements than to the purely subjective portion, because I think it's more valuable.

I've said before that the problem with many subjective reviews isn't actually the descriptions of tonality. I've often found that, if you learn to read through the fluffy language, subjective reviews actually often do agree with measurements, or at least can be explained by them. After all, that's how measurements are correlated to subjective impressions in the first place.

The greater problem is when it comes to value judgements for those aspects of sound quality and comparisons to other speakers. Like, is this $4000 speaker with the recessed midrange really better than that $400 speaker that measures more neutral?

I know I was once guilty of assuming that if one speaker was several times more expensive than another, it had to be better in some way, even if I couldn't quite detect it. That's when words like "rhythmic verve" and "luscious, sprightly microdynamics" start coming out, although I hope I was never quite that bad haha. What's more, once the price tag starts going up, what might've been described as "harsh, abrasive treble" seems to become "I heard details I never heard before!" because your brain has to justify the price difference somehow lol.

I've been working on my "how to understand speaker measurements" article on and off for like a year lol. I think it's about time I finish it.

 

[FeddyLost]

The greater problem is when it comes to "value judgements." Like, is this expensive speaker with the recessed midrange really better than that $400 speaker that sounds more neutral? I know I was once guilty of assuming that if one speaker was several times more expensive than another, it had to be better in some way, even if I couldn't quite detect it

I think if value judgements in hi-fi magazine review will be done "blind", there will be no more hi-fi magazines ...
I'd be totally satisfied if magazines provide full data like Amir (or at least Stereophile) does , and then start to lipstick summary with all audiophiliac terms that they want.
I can omit this marketing BS anyway, but usually there's only BS.
And without raw measurement data there's almost no chances to find out difference between cheap speakers lineup and top tier without critical listening "face to face".

 

[Ericglo]

Youtuber Joe N Tell just posted his response to Andrew, defending measurements. Very well thought out I think.

I have watch a lot of Andrew Robinson's videos and I think he may have been knowingly kicking a hornets nest here. If so, he was successful.

You think? If he didn't know what he was doing, then that says a lot about his knowledge of the subject. If so, then his subscribers should ask themselves why they tune in.

This was calculated to both pander to his audience and create some controversy to encourage clicks. On the latter, it obviously worked.

I haven't watched Joe's video yet, I was critical of him in a previous post, but that may have been premature. I know Erin thinks highly of him and that gives him more credibility than I originally gave him.

 

[Pharos]

Surely the origins of objective measurement were in a darker time when early on in audio engineering an attempt was being made to understand what we were hearing, to develop tools with which to analyse what we hear.

In the 70s I used to buy Hi-Fi Choice and look at the speaker graphs, to then try to predict what a write-up would say about them,
I was usually right, and this became a useful tool with which to 'calibrate' myself.

The earlier point about dismissiveness towards objectively based assessment extends well beyond Hi-Fi into many realms of society, and this egotism based proselytising of a personal view whilst dismissing the concepts of objectivity manifests everywhere. The worst thing about it is that those who do not understand the realties are most prone to accept these expressed views, and this must be surely a majority of the population, and the results are predictable.

From this scenario falsehoods are becoming widespread, and the population becoming dumbed down, which is the very last zeitgeist we need in the current world difficulties.

I have a friend who with 4 degrees, says that there is no such thing as objectivity, he says; "We see the world not as it is, but as we are".

I despair; objectivity is the last frontier to conquer, it can be very hard work, but without that we are lost.

 

[sweetchaos]

I've been working on my "how to understand speaker measurements" article on and off for like a year lol. I think it's about time I finish it.

Excellent news. Looking forward to it. ^^

 

[sweetchaos]

Here's @joentell video regarding this discussion:

Thanks for the link @GGroch, but I'd like to re-post it, so more people can see it.

 

[direstraitsfan98]

Well..that sounds a tad draconian.

Let's ban people in our hobby who we disagree with?

I said ban them from giving advice. That is, giving buying advice to people on YouTube. Normally I wouldn’t care but the fact is these people are peddling absolute nonsense. This is beyond having an opinion and it’s to the point they are doing harm to newcomers to the hobby.

 

[Robh3606]

I said ban them from giving advice. That is, giving buying advice to people on YouTube. Normally I wouldn’t care but the fact is these people are peddling absolute nonsense. This is beyond having an opinion and it’s to the point they are doing harm to newcomers to the hobby.

You do realize that these You Tubers are emulating Tone Audio, Tas, Stereophile. You want to stop them as well???

I still don't get the outrage about this. No different than picking up one of the magazines.

There are manufacturers of speakers that are on record for ignoring measurements because of they like the way the speakers sound. If you look at the measurements you would have a heart attack. People know this and still purchase the speakers.

This hobby is polarized, always has been. I am an amateur speaker builder and heavily into using measurements to make them. I am on your side of the issue and agree that snake oil is being peddled by some and that's not going to change.

Rob

 

[hardisj]

wow... so, @joentell pinned my comment which you'll see below. Not because of what I said about AR blocking me from his channel but because I thought his video (Joe's) was rational and I appreciated that.

Then AR's girlfriend jumps in and ... well... I'll let you guys see...

At least we know exactly what we are dealing with now. I just didn't think they would stoop to the level of actually lying. And I'm damn glad I took screenshots of my comments (which were posted here on Monday) because I would have no way to defend myself against such a baseless claim. "Harassment" isn't a term that should be thrown around so frivolously. Crazy people out there.

 

[981CS]

I've heard a few Harman speakers that I absolutely do not like and wouldn't own if given to me for free. Is the problem with me and my tastes/preferences/hearing or is it that the speakers just don't measure very well? (Note: I've also heard some that I like too). If they measure well and it's been "scientifically" proven that folks should like them, then why don't I like them?

So with that point, I think some of you are taking this far too personally and/or letting your ultimate desire for measurements get in the way of what he's perhaps trying to say. I don't know the guy or even follow him all that much, but from afar I think I can see the angle he's taking with this.

I'm just glad that both sides have a voice and maybe this will start more dialogue because clearly there's some kind of gap starting to form. And if you don't like what a youtuber is saying, then don't watch them. I've many that I'll never click on again.

 

[Rottmannash]

I don't normally watch these things, but Gene Dellasala @Audioholics sent me the link probably with intention to geting a reaction out of me.

I listened to 30 s, dismissed it.. Later I went back and listened to more, and heard him talking about Harman. He didn't say we don't listen but I thought he sort of mischaracterized or dismissed the effort we put into doing listening and doing it right.

At any rate, the main objections I have to this video are:

1) It dismisses the importance of measurements which are are highly predictive of listeners' loudspeaker preference ratings.
2) It dismisses tests done under laboratory conditions as being irrelevant to consumers because he says "variables like a) rooms, b) programs, c) hearing and d) personal tastes"

a) Rooms _ As people know here, the room is mostly dominant below the room transition frequency (~200 Hz) but above that the ANSI 2034 measurements are generally good predictors of sounds that will be heard in a room (e.g. the PIR or predicted-in-room response ). Above 2-3kHz, due to the directivity behavior of the speaker and room absorption, the listener hears mostly direct sound which is represented by the on-axis/listening window. Good off-axis response and smooth directivity ensure the reflected sounds are neutral.


b) Program -- Yes, loudspeaker-program interactions are real but you deal with them from a statistical standpoint. Andrew gives examples of bright programs being compatible with dull speakers. What is the chance that you only listen to just bright mixes or dull mixes? On average, programs likely converge on neutral, so you make the speaker neutral to be compatible with neutral recordings, and use tone controls for bright and dull recordings. The recording industry monitors are generally converging on flat so hopefully neutral recordings will be the trend (assuming the producer has no serious HF hearing loss).

c). Hearing - We generally screen for normal hearing although in larger studies we may include unscreened listeners and older listeners. Does that invalidate the results for Andrew's audience who may have significant hearing loss? So far, I have seen little evidence that people with slight-moderate hearing loss prefer speakers that are not neutral. I just saw a recent study where 3 groups of listeners. (normal, slight, moderate HL) preferred the same headphones. The difference was the more hearing loss, the noisier and less discriminating the ratings where. The HL groups used a smaller range of ratings but the rankings, at least for the top 50 percentile ranked headphones was consistent with the normal hearing listeners.

d) Taste- When listeners are asked to rate headphones or speakers based on preference there is remarkable agreement on which ones are most and least preferred. Recently I looked at segmentation of listeners based on headphone preferences and found there are three groups: 64% who prefer the Harman Target Curve, 21% who prefer the Target with less bass and 15% who like the Target with more bass. So again, there appears to be a majority of people who like what they consider neutral or accurate.

So how can 35+ years of research into perception and measurement of loudspeakers and headphones not jive with Andrew's reality of what he and listeners prefer?

The simple answer is: the lack of controlled, unbiased listening. When you do not control the variables (normal hearing, listener training, loudspeaker position, double- blind, loudness matching, randomized order, program, statistical analysis, hidden anchors and references ) you will tend to get random, noisy and unexpected results. A single stimulus demonstration doesn't even allow the opportunity for the listener to hear what is "neutral".

In the absence of measurements we are encouraged to let our ears choose what we like, but no guidance is given on how to do scientific listening so we avoid making rookie mistakes. Instead let all the acoustic, psychological and physiological nuisance variables and biases run amok, roll the dice, and choose whatever speaker they like or sounds fun. Andrew even gives us some speakers that measure bad but sound "fun" to further bias his audience towards making mistakes.

This is *exactly* the situation where good technical measurements are needed to help consumers avoid making poor decisions in sub-optimal listening conditions (or no listening which reflects increasingly more internet sales) yet, the video generally dismisses their usefulness.

I know the F208s are the best speakers I've yet heard in my system.

 

[JustJones]

My current speakers I bought strictly by how they measured, never heard them before. Best speakers I've had in my house. Genelec 8351B. Might not be the best way but nowadays you do what you can and hope for the best. I figured less risk knowing the measurements and how they should sound.

 

[Beave]

I've heard a few Harman speakers that I absolutely do not like and wouldn't own if given to me for free. Is the problem with me and my tastes/preferences/hearing or is it that the speakers just don't measure very well? (Note: I've also heard some that I like too). If they measure well and it's been "scientifically" proven that folks should like them, then why don't I like them?

We don't know, because you didn't tell us which speaker models you didnt like, nor did you tell us under what conditions you listened to them or what you compared them with. It's also doubtful you listened or compared blinded and level-matched, so who know what other factors came into play.