John Atkinson's of Stereophile Talks About Measurements

[Sgt. Ear Ache]

Personally, I think it's totally true that people buy equipment that they "like" the sound of. But I don't necessarily think that's the best way to do it. Our system of hearing - our ears combined with our brains - has evolved to be able to adapt to our environment. We adapt to the sound of the world around us. Our hearing is malleable. As such, it seems to me that there needs to be an external, objective baseline for good sound. For me, that is neutral. I want the sound that reaches my ears from my system (whether it's headphones or speakers) to be as close to un-colored as I can get it. I want my system to be a pipeline that transmits the original recording to my ears without changing it in any way other than the amplification level I choose to listen at.

So, given a baseline quality system - one that can produce a flat, audibly undistorted signal - and a set of speakers or cans that can put out a reasonably flat signal from 40hz-20khz, and a bit of EQ to get even closer to neutral...I'll let my ears adapt to that rather than to a sound that is colored or altered in some way by the system. Measurements definitely can point in the right direction for that..

And the nice thing is that you can get there without spending a whole bunch of money at all.

 

[amirm]

How about Dayton B-652 speakers and Lepai amp Steve once recommended, which he said does not sound terrible at all? This combination costs $50 bucks for new gear. If I was a young kid and that was all money I had that would make a great first little system. Can you “easily” do any better within that budget? I doubt it.

The problem is that he recommends everything. Have you seen a video where he outright rejects a product as I do? You don't. If it is given to him or makes up the content for the daily video to get more subscribers, it gets a positive review.

This is why I said he has no audio compass. Nothing to tell him the truth about audio products. By shunning measurements and not having critical listening skills, he recommend any and all audio products.

If he was going to recommend a $50 anything, he should show reliable data of comparing one setup to a number of other alternatives. That is not what is done.

And then there is the bit about "here is a great $100 DAC but wait, it is not as good as the $1000. Mind you, it is a great DAC. But it is missing that something that the $1000 one has." Let me know if you have seen him say the $100 cleans the floor with the $1000 one he has already reviewed. I have watched a lot of his videos but this is not what you are going to see from him.

Steve's asset is that he has a friendly tone and hence is likely to be trusted with what he says. That, however, has nothing to do with whether a speaker or amp is good or not.

Reminds of making a few dishes from a cooking show on TV. In every case the food tasted terrible. But boy, the show made it seem tasty due to the presentation. One was an Indian curry where you put the spices in at the end and just warmed up the dish. Ah, that made it quick to make but boy, who has heard of Indian curry being about dumping spices in a dish and just warming it?

 

[dshreter]

So, given a baseline quality system - one that can produce a flat, audibly undistorted signal - and a set of speakers or cans that can put out a reasonably flat signal from 40hz-20khz, and a bit of EQ to get even closer to neutral...I'll let my ears adapt to that rather than to a sound that is colored or altered in some way by the system. Measurements definitely can point in the right direction for that..

The trouble is there is no such thing as neutral when music is played through speakers. There are a number of additive concepts in the direction of neutral, but as far as I know there isn't a true reference.

There are some concepts that are somewhat well established to contribute to "neutrality":

• Flat on axis anechoic frequency response
• Smooth off axis anechoic frequency response
• Minimal distortion and resonance

There are some concepts though that don't have a well established reference:

• Dispersion - wide vs narrow
• In room response / room curve - flat vs tilted vs curved
• Home listening reference SPL - what I've seen thrown around is way too loud, and this also throws off what would be preferred for a room curve
• Room size and overall acoustics - liveness of the room, absorption approach for each wall, diffusion

Without a truly established reference - if I master audio X it will sound like Y at reference playback - it necessarily leaves room for a subjective set of measurable preferences within that space.

 

[DonH56]

John Atkinson is talking about a simple test in minute 15:20 so called Dual mono pink noise test to measure if a speaker system has more or less any imaging capabilities. Is it not an idea if you test speakers to take this along your measurements because there is so much talk on ASR regarding Imaging, transparency etc.
Great video.

It's a fairly common technique but depends very much upon the environment (room) as well as the speakers. It is also not great at telling exactly the problem and how to fix it; is it the speakers, room reflections, both, ??? I used to do it all the time; now, I usually just let the room correction do it's thing. But a good reminder; I should pull out my pink noise source and see how things stand.

Aside: Do not use white noise for this test unless you like blown tweeters at worst and at best ear-piercingly bright sound in the treble when you get the bass loud enough to hear.

 

[Matias]

The problem is that he recommends everything. Have you seen a video where he outright rejects a product as I do? You don't.

All the audio review industry works this way. There are no negative reviews, period. Manufacturers advertise and pay them to review. If the reviewers don't like something, they simply do not publish anything about it and privately give the manufacturer a feedback. Or often reviewers say that they only ask samples of products they heard in a show and liked, or that someone tipped them that a given product is awesome, or from a brand he knows and trusts, etc. That is what I read them answering time and again.

 

[MattHooper]

I figured that video was going to be grist for the ASR mill

I was also disappointed (but ultimately not surprised) by JA avoiding answering the claim Steve brought of "speakers that measure well but sound poor."

Wait...what???

This sounds like the hoary old audiophile trope of amplifiers that "measure correct/well but sound awful." Why should this claim be accepted as anything but in the imagination of said audiophile?

Steve's claim should be been pushed. What does Steve mean by "measuring well?" Does he perhaps have an older concept that "speakers that measure flat on-axis are speakers that measure well?" If so, that's a naive understanding of speaker measurements, where it's possible to have a speaker measure relatively flat on axis but which have left problems in off-axis radiation. If you actually know what to measure and understand the measurements, then the measurements can explain WHY the speaker Steve has in mind "sounds poor."

Or is Steve actually more fluent with the current state of knowledge on speaker measurements and for instance accepts the research from Toole et all on what measurements make for good/preferred sound? Well, if Steve is saying that speakers that measure well on and off-axis in the way Toole describe can still sound "poor" or bad, what speakers would he point to and what evidence? All the actual scientific evidence about what people rate as "good" sounding speakers predicted by their measurements would be AGAINST Steve's claim, and I'd like to see an actual example of one of his "speakers that measure well but sound poor" so we can see exactly where he's misunderstanding things and promoting audiophile myths.

 

[Thomas savage]

All the audio review industry work this way. There are no negative reviews, period. Manufacturers advertise and pay them to review. If the reviewers don't like something, they simply do not publish anything about it and privately give the manufacturer a feedback. Or often reviewers say that they only ask samples of product they heard in a show and liked, or that someone tipped them that a given product is awesome, or from a brand he knows and trusts, etc. That is what I read them answering time and again.

If you can read between the lines there's often negative comments but they are hidden.

The ' review' is published with the consent of the manufacturer and or distributor, it's a team effort lol .

It's not a honest industry.

 

[Purité Audio]

Hey Savage, some of us a thin sliver of integrity, not many mind.
Keith

 

[pozz]

Isn't "measures well but sounds bad" said kind of unthinkingly when considering just on-axis response, or just sound power, or THD without stating boundary conditions? As some kind of simplification or disingenuousness.

 

[Sgt. Ear Ache]

The trouble is there is no such thing as neutral when music is played through speakers. There are a number of additive concepts in the direction of neutral, but as far as I know there isn't a true reference.

There are some concepts that are somewhat well established to contribute to "neutrality":

• Flat on axis anechoic frequency response
• Smooth off axis anechoic frequency response
• Minimal distortion and resonance

There are some concepts though that don't have a well established reference:

• Dispersion - wide vs narrow
• In room response / room curve - flat vs tilted vs curved
• Home listening reference SPL - what I've seen thrown around is way too loud, and this also throws off what would be preferred for a room curve
• Room size and overall acoustics - liveness of the room, absorption approach for each wall, diffusion

Without a truly established reference - if I master audio X it will sound like Y at reference playback - it necessarily leaves room for a subjective set of measurable preferences within that space.

Yes, but I'm only concerned about the sound at my listening position. I want that to be as close to neutral as possible. I'm aware of what goes into that - room dynamics, etc...but still I want to take whatever measures I can to achieve neutral at my ears. That starts for me with equipment that measures well.

 

[Dogen]

I have to admire (?) Atkinson’s ability to withstand cognitive dissonance. He seems very knowledgeable and level-headed on one hand, but on the other colludes, at least passively, with the worst of the snake oil salesmen to basically rip off customers. His love affair with MQA and his hand in outright misrepresentation was extremely disappointing to me. Still, he’s one of the last of a breed - an audio “ journalist” who knows $hit from shinola. Wish he’d be more forthcoming with his real employer, the audio consumer.

 

[restorer-john]

If it stops people with a limited budget wasting money on gear that can't be a basis for a system: yes.

Absolutely correct. The Lepai an absolute piece of junk and telling people to stay away from it is a community service.

It is a perfect bench/test amp for other projects, which is what I use mine for.

With respect, not a single Class D amplifier is remotely useful as "bench/test amplifiers" due to the RF sitting on their waveforms. You can't use appropriate test gear with them. DSOs/CROS or mV distortion meters anywhere near them, unless you have a pile of filters and everything compensated for those filters.

Most of them are also floating output (non ground referenced/BTL style) so common ground test gear will short them out. You can't use them as a reference as they are inadequate in that respect. About all you could use one for is feeding LF into a woofer you were re-rolling for VC gap placement.

I use a Rotel pre and power amp on my bench which have known, excellent characteristics, to drive or be-driven by, devices under test.

 

[Matias]

If you can read between the lines there's often negative comments but they are hidden.

The ' review' is published with the consent of the manufacturer and or distributor, it's a team effort lol .

It's not a honest industry.

Yes, they are a team effort, but I would not say it is dishonest, only that they tell part of the story, omitting things.

Very subtle negative observations here and there, yes, but always overall positive review. No way they bash something as broken or stealing money like our host Amir does. Which makes ASR reviews sooo fun to read.

 

[dshreter]

Yes, but I'm only concerned about the sound at my listening position. I want that to be as close to neutral as possible. I'm aware of what goes into that - room dynamics, etc...but still I want to take whatever measures I can to achieve neutral at my ears. That starts for me with equipment that measures well.

I'm not refuting that as a goal though. My point is a precise definition of neutral at the listening position really doesn't exist. Dispersion doesn't just impact the sound as you move around the room, it changes the balance of direct vs reflected energy which has an impact on how things sound even in the sweet spot. Similar with the room curve. Flat speaker response does not translate to flat in room response, and the target response in the room isn't well defined.

 

[Thomas savage]

Hey Savage, some of us a thin sliver of integrity, not many mind.
Keith

Ha ha, you ( and I ) know how it works.

 

[Sgt. Ear Ache]

I'm not refuting that as a goal though. My point is a precise definition of neutral at the listening position really doesn't exist. Dispersion doesn't just impact the sound as you move around the room, it changes the balance of direct vs reflected energy which has an impact on how things sound even in the sweet spot. Similar with the room curve. Flat speaker response does not translate to flat in room response, and the target response in the room isn't well defined.

Why isn't it well defined? Can we not use a known entity such as pink noise and measure at the listening position as a guideline for a bit of EQ? Now I totally get that you can't remove the room from the equation. But, you can try and get as close as possible. And, knowing that the sound leaving the speakers isn't colored or altered at all is a good place to start.

 

[dshreter]

Why isn't it well defined? Can we not use a known entity such as pink noise and measure at the listening position as a guideline for a bit of EQ? Now I totally get that you can't remove the room from the equation. But, you can try and get as close as possible. And, knowing that the sound leaving the speakers isn't colored or altered at all is a good place to start.

A good explanation for this is the counterfactual. Listening to speakers in an anechoic room to hear only on-axis sound isn't considered "neutral". Part of this is being in an anechoic chamber just feels uncomfortably quiet and weird, but there's apparently also preference for some amount of reflected sound too.

Reflected sound will have a different arrival time to the ear than direct sound, so it changes the overall sound characteristics. All real rooms have this behavior of acoustic decay to a greater or lesser extent depending on size, construction, and treatment.

Then there's the matter of room curve. The objectively flat speaker if placed close to a wall will have more bass, away from the well less bass. How much is the right amount if there's not a known reference for target in room response? Flat is not considered a desirable goal, and targets are only loosely defined.

 

[Sgt. Ear Ache]

A good explanation for this is the counterfactual. Listening to speakers in an anechoic room to hear only on-axis sound isn't considered "neutral". Part of this is being in an anechoic chamber just feels uncomfortably quiet and weird, but there's apparently also preference for some amount of reflected sound too.

Reflected sound will have a different arrival time to the ear than direct sound, so it changes the overall sound characteristics. All real rooms have this behavior of acoustic decay to a greater or lesser extent depending on size, construction, and treatment.

Then there's the matter of room curve. The objectively flat speaker if placed close to a wall will have more bass, away from the well less bass. How much is the right amount if there's not a known reference for target in room response? Flat is not considered a desirable goal, and targets are only loosely defined.

As I said, you can't remove the room totally. As close to neutral as possible is what I want but you can't EQ the room away. There's no room in headphones right? I am aware of the nature of room dynamics. But this discussion is about the measurements of the equipment. Once the sound leaves the speaker, we have the issue of room dynamics to deal with, but I still want the system to produce a neutral, un-colored signal, and then I work with it from there.

 

[Hugo9000]

Isn't "measures well but sounds bad" said kind of unthinkingly when considering just on-axis response, or just sound power, or THD without stating boundary conditions? As some kind of simplification or disingenuousness.

I think it might refer to the cases where Stereophile has JA's measurements that might look rather awful but where he writes that they are good (or sometimes says they show "good engineering" lol), like some of the Wilson reviews or Devore or whatever. The problem is, those "bad sounding" (to many people, just look at the polarizing response across audio forums to those brands) products don't actually measure well, they just get the bad measurements glossed over/minimized.

You can take your pick of nearly any of the tons and tons of Wilson speakers that the magazine has pushed through the years. I looked at a random selection of five reviews of Wilsons, and they all feature rather bad measurements and amazing contortions to excuse them or occasionally simply say they are "good." (Yvette supposedly good, Watt/Puppy 8 odd contortionist conclusions, Alexandria XLF might be too big to be measured correctly lol, Sophia supposedly excellent lol, Alexia supposedly measures well).

I think the "measurement" thing at that mag is just a form of gaslighting, honestly. "Oh, but look, Stereophile has reviewed every model of Wilson speaker in history, and always gives glowing recommendations of them, and you know that Stereophile does real measurements, so Wilson must be a great company with spectacular engineering prowess!" Occasionally, something has truly great measurements and gets a positive review, but all too often it seems that the measurements and the conclusion/summary of those measurements by JA don't correlate well (never mind trying to correlate the measurements to the subjective review itself lol). If he honestly thinks there is no way to truly or accurately characterize much of this gear through his measurements, then he shouldn't have bothered with those measurements in the first place.

All that said, if someone is not used to hearing their favorite recordings reproduced with a flat/even frequency response (as just one example of something that can be measured), that person may certainly feel that "measures well" means "sounds bad." It must take some getting used to, if you are accustomed to decades of hearing a favorite album with bass bumps and suckouts or peaks at various frequencies, to suddenly hear it without those anomalies.

 

[murraycamp]

I loved the comment of liking something that’s wrong. I fear that I fall in that camp with my love of my primaluna tube amp and vinyl setup. I’ve been following these forums for a long time but I’m hesitant to take the plunge into the accuracy world. I love what I have and although it may be wrong, I still love it.. what if I get it right, but I still prefer what’s wrong?

You have hit the nail on the head on the whole subjective vs objective deal. The main point is acknowledging that subjective preference does not necessarily equate to accuracy or fidelity to the source. Even though output of the system may be compressed, noisy and/or distorted, you may very well like that sound subjectively. Most people on this forum will not assign fault to that unless you contend that your subjective preference (even with poor measurements) is "better," "more accurate," or "high fidelity."

Also, you may find that higher accuracy ends up sounding "better" subjectively. I did. The Toole, Harman, et al. research also supports that conclusion as far as it goes.

My goal with my own system is to achieve audible transparency with the digital chain, and then have fun with compressed, noisy and distorted vinyl. Most of the vinyl I listen to anyway is jazz recorded before 1968, and most of that is live, so even assuming state of the art (circa 1960-ish) portable recording equipment, the source is compressed, noisy and distorted. So I just have fun with it. At least my accurate digital system will accurately reproduce the compression, noise and distortion of vinyl. Best of both worlds. HTH, YMMV, IMHO, etc.