Was this aimed at ASR?

[CtheArgie]

My missus told me there was a mention of this column in a recent PS Audio daily dribble, and mentioned as a criticism of this site. I can now read it as it was not then availble online.

There are a couple of interesting issues here. First, I am not familiar with Toole's methodology and I am not an expert on what he did. But I did do a lot of work on questionnaire design and studied with some of the world's experts on research methodology, so I am a little cautious to comment on Toole. It may be totally obvious that he ended up with the conclusions he reached. He tested loudspeakers in a specific room. But I don't know if he tested the sound of instruments on the room to figure out which speakers sounded "closer" to the musical instruments. If I go to a classic music concert, the sound will vary depending on where I seat. I prefer the way I hear in Carnegie Hall than the sound at Disney Hall. Is it possible that the architecture biases me to what I "prefer"? Having read a lot of Kahnemann's work too, which incidentally has a big overlap with some research methodologies and techniques (conjoint analysis being the most relevant), I am very aware constantly at how biases affect my presumed preferences and decision making.

However, I tend to be more convinced of the work of Toole as explained by all of you at ASR. If most speakers are designed to a specific model, then we can compare to the "real" sound. I also have to admit that the real sound is a very elusive issue. Is King Crimson playing live the real sound? What about John Pizzarelli? I remember telling his bass playing brother after a concert in LA to tell John to let him have his bass sound louder. It gets lost even in a small venue. Do I listen to Pizarelli different then as he doesn't not want his bass to sound "fair"?

Speakers that modify what is been fed to them act as musical instruments. Period. People may like to modify the sound of their systems and hear the way they want to, but this is not accuracy. Same with DACs, pre-amps, amps, etc. This is why I assume that the smallest modification or distortion is best.

Lastly, I wish producers/masters would state which speakers they used while mastering a record, and the performance of those speakers on their rooms, this way, when I listen to my set up, if I want to be accurate, I can compare to what they used to approve the master.

 

[zelig]

"All truth passes through three stages. First, it is ridiculed. Second, it is violently opposed. Third, it is accepted as being self-evident".

- Arthur Schopenhauer

 

[Soundstage]

Who has the courage to read the lengthy sound impression published by Stereophile?
I, for one, jump first to the specifications to find out the price and measurements . Then I read the measurements. Then, sometimes, I take note of the song records used. Most of the time these records did not stay for long on my bucket list.
Besides the measurements, I also read the music review section. However this one lacks measurements. At least on pop rock blues and jazz records we should have some dynamic range measurement.
Finally, while I have been reading Stereophile for years, I have learned incredibly more on ASR in 6 months than in years at Stereophile. Stereophile could do a lot more in educating. They assume the reader knows everything! That’s really a shame. But Stereophile (thanks to John Atkinson) is one of the best magazine out there.

 

[pozz]

I don't like the "some critics out there" approach. If you have a specific beef, raise it. But OTOH there is nothing wrong with the concept that legitimate preferences by any one person may not be the same as an average. We are dealing with sensory experience, first of all, so there's z10% certainty that some of us experience a different FR curve than others. Further, my choice of music might reward deep bass more than yours.

I've previously mentioned this kind of preference in the context of wine: There are biological differences in the way we each taste and some aspects of wine preference are driven by that. Perhaps, someday, when we have a bigger database of both products and listeners, we can score ourselves and then match our style preference to reliably determined "flavor styles" of speakers. That could be completely scientific yet not one-size-fits-all.

I could see it working if speakers had a mechanism to select directivity and you could upload personal HRTFs/HRIRs and auditory thresholds, and provide room information.

 

[Jorj]

Transducers are hard. So many configurations of drivers, crossovers, cabinets, you name it. At the end of the day, they are trying to reproduce sounds faithfully, and we have good reason to say that measurements objectively describe a speakers' ability to do so. Given the ability of a room to make a well-designed speaker sound bad, and conversely, make a poorly-designed speaker sound better, I tend to give very little weight to subjective reviews.

I still regularly use my links to the archives of the Audio Critic and SoundStage! data for older speakers, and am thrilled that Amir is testing speakers with a repeatable methodology. Data truly is king. If you want to tell me how amazing your speakers sound, I will nod and smile and congratulate you on being happy with your results, but don't tell me that a subjective opinion is worth more than the electrons that carried it to my monitor. It is meaningless except as a personal anecdote of an experience, not as a review of any import whatsoever.

 

[Ron Texas]

Don't let yourself be overcome by the numbers. Stereophile is definitely paying attention to what is happening here.

 

[Midwest Blade]

Yeh! I am still with F. E. Toole on this one, with standard measurements at least we can establish a baseline from which to begin. The article sort of pitches their abilities to describe the sound..."golden ears".

 

[nemanja_t]

The problem is not in the testing methodology, the problem is independance from manufacturers. Everybody just got too comfort by noding heads with every new product, and publishing same old reviews over time.

 

[Mountain Goat]

Yeh! I am still with F. E. Toole on this one, with standard measurements at least we can establish a baseline from which to begin. The article sort of pitches their abilities to describe the sound..."golden ears".

Yeah, when I hear "trained listener" I'm Audi 5000.

 

[Neddy]

Objectively, I'd say 'quote the proper phrase', darn it! Bah.
It should be "Hoist with your own Petard"
https://en.wikipedia.org/wiki/Hoist_with_his_own_petard

I just don't get all the angst over subjectivism in audio - everybody's ears, rooms, and musical preferences are different (and changing all the time).
One - or several - subjective reviews tell me almost nothing about whether I'd like a particular product.
Amir's objective, repeatable measurements are about the only clues I have to guide me in choosing what to audition, or take a risk on.
Music reviews are entirely subjective - and there's nothing wrong with that, either.
Both have a place, just keep them there, and be happy!

 

[pozz]

I saw it as a kind of maintenance-dose comment, saying, "Yeah, engineers and researchers, as is inevitable, are slowly converging on a consensus ... but we want to keep the bubble alive, so let's all agree there's plenty of wiggle room for us to operate in."

That message isn't just in the magazine's interest - lots of consumers want it to be true too, because they love the way they do things. It's a kind of benign conspiracy, or a willing collective suspension of disbelief, enjoyable because none of it really matters. It's a kind of participatory showbiz, where people feel special, and it will never go away.

This is a good point.

Many have moved on from purely expert or word-of-mouth recommendations. The emphasis on understanding what you're buying and why, and understanding the supply chain and industry practices, is fairly commonplace (and very disruptive).

 

[Canuck57]

I look at Stereophile for the pictures (audio porn) and read Playboy for the articles!

 

[JeffS7444]

Ahh, the one-sided plea for "open minds" once again. I think ASR has been very open-minded to notions such as $100 DACs offering excellent performance on an absolute scale, not merely "for the money".

 

[teched58]

The schizoid nature of the Stereophile business model -- suck up to high-end advertisers with glowing subjective reviews, while stifling grumblings about this from readers (usually by calling them "jealous") -- is showing. On the one hand, we have Jim Austin's column weak-tea defense of Stereophile's way of conducting reviews, AT THE VERY SAME TIME (well, ok, three weeks ago) Herb Reichert calls the Tannoy Revolution XT 6s "thin, metallic, herky-jerky, dull, and rolled off completely below about 90Hz." John Atkinson's detailed measurements verify this poor performance. But then to highlight the difference between ASR and Stereophile to correct Reichert's deviation into honesty, he closes the review by "highly recommending" the speakers. As we say in New Jersey, WTF?

 

[amirm]

How does the author know this?

The appeal often comes from improper listening tests and subjective impressions of reviewers with same faults. When put in controlled tests, this is what we get:

And there is this:

Then how come we put him in controlled tests, the results are these:

All the groups similarly ranked these speakers. Error bars are there by the way to account for listener variation and it doesn't change the rankings.

The reviewers had their shot at showing differing preferences but failed.

 

[Feyire]

"All truth passes through three stages. First, it is ridiculed. Second, it is violently opposed. Third, it is accepted as being self-evident".

- Arthur Schopenhauer

 

[Jimbob54]

Give me a like if you had to google the meaning for "petard"

I havent googled and I believe it is some kind of military accoutrement/ garment. I am now going to google and find out how wrong I am.

 

[Inner Space]

Ahh, the one-sided plea for "open minds" once again. I think ASR has been very open-minded to notions such as $100 DACs offering excellent performance on an absolute scale, not merely "for the money".

True, and from a magazine's point of view, this is the only lethal danger that measurement presents. Translated, the one-sided plea is, "Please keep us in business." Which won't happen if components become interchangeable commodities at prices too low to allow for an advertising margin. We've seen their defense already, with cables - truly interchangeable commodities - and I suppose we can expect the same with electronics.

(Pausing a second now, shaking my head - your post somehow emphasized to me we live in a market where a DAC with flawless sci-fi numbers can be had for $100, and an RCA cable for $10,000.)

 

[Jimbob54]

I havent googled and I believe it is some kind of military accoutrement/ garment. I am now going to google and find out how wrong I am.

Close but no cigar!

 

[Racheski]

Close but no cigar!

It's like an IED to breach doors or gates. Also apparently "Hoist with his own petard" is a Hamlet quote - look how cultured they are!