• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Was this aimed at ASR?

b1daly

Active Member
Forum Donor
Joined
Dec 15, 2018
Messages
210
Likes
358
@amirm, again, I have no problem accepting this as a reference. I actually think it is extremely helpful to have it as a reference. This is an independent issue of accuracy to live unamplified music, but I don't think there is a practical way of doing this (to date).

And, no, preference is not more important than accuracy. But we don't know how to reach it. I thought this was ASR and not a subjective site. We can still have opinions about the science though.
For audio reproduction for fun, pleasure listening, preference is the only thing that matters. It just turns out that more ‘accurate’ speakers are preferred. You are mixing up two concepts of accuracy. The sense it used in these discussions is simply how accurately it reproduces the input signal. It has nothing to do with how accurately it represents acoustic live performance. That is the job of the producer, and if producer and listener have more or less accurate systems, this will determine if the listener will hear the work of the producer and whatever ’musical accurac’ they may have managed to embed into the signal.

This is the best argument for accurate/flat speakers. It gives a common reference point on both ends of the ’production pipe.’ If all studio monitors were made with a deliberate frequency response curve, then and ‘accurate’ playback system would fail completely.
 

pozz

Слава Україні
Forum Donor
Editor
Joined
May 21, 2019
Messages
4,036
Likes
6,827
In the Harman speaker test is the listener asked to simply rank the speakers by preference or is the suggestion to rank based on perceived accuracy also included?

I know for me this would be a significant deviation.
Research showed that preference correlated to accuracy. I.e., listeners would rank speakers based on what they liked, and that ended up corresponding to how well-designed the speaker was in terms on consistency of axial response, lack of mechanical resonances and other factors.
 
Last edited:

Jimbob54

Grand Contributor
Forum Donor
Joined
Oct 25, 2019
Messages
11,066
Likes
14,697
Well what did your other fish in the sea comment mean then?

I would guess he is saying if Stereophile publish a bad review so manuf stops advertising there, there are plenty more manufs who will buy the ad space.

There is an interesting point here. Lots of us feel there is a link between the slant of the reviews in a publication and the fact that the publisher needs ad revenues from the manufs of the reviewed products. An inbuilt conflict for sure. This may play out badly for objectivity in some instances. However, let us not forget the manufs need somewhere to advertise. As do their competitors. Somebody will always want the back page of Stereophile whilst ever it has a readership that manufs wants to appeal to.
 

Kal Rubinson

Master Contributor
Industry Insider
Forum Donor
Joined
Mar 23, 2016
Messages
5,273
Likes
9,789
Location
NYC
Well I guess your enigmatic aphorisms are over my head because I don’t understand your meaning.
Someone, in this thread I think, already asked about the potential impact of manufacturers refusing to submit new products if/after a bad review as you did. My reply to the question was the same indicating that there is a sufficient number of other manufacturers that makes this not a problem.
 

North_Sky

Major Contributor
Joined
Oct 31, 2019
Messages
2,741
Likes
1,553
Location
Kha Nada

mt196

Active Member
Joined
Apr 7, 2019
Messages
229
Likes
156
Location
Rome, Italy
I agree with the fact that judging a speaker not good just because it is not balanced is stupid and I cannot understand why a lot of times amir does it.

I read the graphs in the reviews, but completely jump the conclusion since reading " I cannot suggest these speakers because they have not much power" or "they have not a flat response" is something I don't understand.

Maybe there are people that do not want a flat speaker? Never thought of it? While I 100% agree with not suggesting stuff that has distortion and resonances
 

JustAnandaDourEyedDude

Addicted to Fun and Learning
Joined
Apr 29, 2020
Messages
518
Likes
819
Location
USA
I agree with the fact that judging a speaker not good just because it is not balanced is stupid and I cannot understand why a lot of times amir does it.

I read the graphs in the reviews, but completely jump the conclusion since reading " I cannot suggest these speakers because they have not much power" or "they have not a flat response" is something I don't understand.

Maybe there are people that do not want a flat speaker? Never thought of it? While I 100% agree with not suggesting stuff that has distortion and resonances

Surely, you jest?

Although, I admit I do fancy a Marilyn Monroe shaped speaker singing "happy birthday" to me.
 
Last edited:

March Audio

Master Contributor
Audio Company
Joined
Mar 1, 2016
Messages
6,378
Likes
9,317
Location
Albany Western Australia
.

"It was John Crabbe who defined for me the relationship between a magazine's editorial integrity and the advertisers who financially support it (readers, sadly, are never a significant source of income, given the high costs of distribution): 'If you tell the truth about components you review, there will always be a small percentage of companies at any one time who are not advertising in your pages. But if you publish the truth, you will have a good magazine. And if you have a good magazine, you will have readers. And as long as you have readers, disgruntled advertisers will eventually return. But if you don't tell the truth, you won't have a good magazine. And if you don't have a good magazine, you won't have readers, at least not for long. And if you don't have readers, you won't have advertisers.'"

A philosophy that is as true now as it was a half-century ago.

John Atkinson
Technical Editor, Stereophile

Sorry to be pedantic but can we just make one important distinction here.

The subjective reviews you publish are not the "truth", they are the reviewers opinion.

There are many reasons why one may, and almost certainly will, diverge from the other.
 
Last edited:

Robin L

Master Contributor
Joined
Sep 2, 2019
Messages
5,209
Likes
7,588
Location
1 mile east of Sleater Kinney Rd
Sorry to be pedantic but can we just make one important distinction here.

The subjective reviews you publish are not the "truth", they are the reviewers "opinion".

There are many reasons why one may and almost certainly will diverge from the other.
Yeah, but the measurements aren't "opinion", they're data.
 

b1daly

Active Member
Forum Donor
Joined
Dec 15, 2018
Messages
210
Likes
358
Someone, in this thread I think, already asked about the potential impact of manufacturers refusing to submit new products if/after a bad review as you did. My reply to the question was the same indicating that there is a sufficient number of other manufacturers that makes this not a problem.
The issue I‘m talking about isn’t manufacturers being willing to submit new product for review: my guess is that there will never be a shortage of such manufacturer. It’s the purchasing of ads that is the biasing factor. My contention is that many of the products advertised and reviewed in Stereophile are wastes of money at best and pure snake oil at worst. Actively demonstrating this in the way ASR is doing would damage the business model which is based on symbiotic alignment of interests, not rigorous and honest appraisal.

This isn’t necessarily a matter of deliberate deception. The multiple factors that bias human behavior are so powerful that even full self awareness is not enough to overcome them.

My biggest complaint is not even the praise of “inaccurate” equipment which I think is besides the point. Granting the best intentions, if a reviewer heaps praise upon an ‘inaccurate‘ speaker there are little grounds to challenge that. A subjective opinion like this can’t be wrong.

The problem is praising as ’sonic superior’ products that are sonically indistinguishable in properly controlled listen tests. The issue here is not the truthfulness of the reported experience: ‘placebo effects’ in audio actually cause a different perception, they can really make things ’sound better.’ This can be forgiven in a naive listener because this is not widely understood. In an experienced audio pro, who is purporting to be a source of authority on sound quality and value, this is disingenuous at best. I think for authors who have been doing this for years it’s rather scandalous.

I don’t ascribe bad motives to most people. So when I see intelligent, experienced reviewers peddling this nonsense year after year, my search for an explanation leads me to postulate the kinds of distorting relationships I outlined.
 

North_Sky

Major Contributor
Joined
Oct 31, 2019
Messages
2,741
Likes
1,553
Location
Kha Nada
There are two kinds of audio ...
1. Audio business, including porno gear, and pro audio.
2. Audio music enjoyment; open field for all classes of music lovers...with or without data.

Can you mix the two together? ...The art of the deal; it's been tested in the past and it failed.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,388
Location
Seattle Area
In the Harman speaker test is the listener asked to simply rank the speakers by preference or is the suggestion to rank based on perceived accuracy also included?
Maybe they give instructions to testers during studies but the two times I took them, they just gave us score sheets to fill in. This was my sheet early in the testing:

Harman Voting.jpg


As you barely see, we were told to score from 0 to 10. It is up to you to decide how you figure that out.
 

Robin L

Master Contributor
Joined
Sep 2, 2019
Messages
5,209
Likes
7,588
Location
1 mile east of Sleater Kinney Rd
On the one hand, I really want the issues of recording/playback to have solutions that are scientifically determined, repeatable. I want audio gear that is reasonably priced and sensibly designed.

On the other hand, I've heard systems that were expensive, weirdly designed and amazing sounding. A number of combinations of single ended triodes, coupled to high efficiency speakers, playing back LPs exceeded my expectations. I can't say the sound they produced was "realistic", though to be honest, I gave up on realism a long time ago. But the sound was suductive, I could understand why someone would persue that kind of gear. I suspect that If I had that kind of gear long enough, the "audible illusion" would go away eventually. But I think that's the issue Stereophile contends with, beyond the usual issues of what can be measured, what can be described, those unknown unknowns of sound. At the end of the day I'm happier with gear that lets me know an AKG 451 was used on the guitar, a Neumann u-47 on the vocal. Because, hearing something that sounds like a microphone feed is more believeable to these ears than a system that's attempting to fool me into thinking it's "the real thing". I already know it's not the real thing.
 

b1daly

Active Member
Forum Donor
Joined
Dec 15, 2018
Messages
210
Likes
358
Exactly the case. When I joined Hi-Fi News magazine in 1976, the magazine's editor was John Crabbe, from whom I learned my craft. John passed away in December 2008 and in an essay for Stereophile in which I commended on his passing - see https://www.stereophile.com/asweseeit/communities/index.html - I wrote: "Soon after becoming HFN's editor in 1964, John had written that a magazine's reviewers 'have a mandate to write what they think, even if some feelings are hurt.' John taught me that this is the ground on which a review-based magazine stands.

"It was John Crabbe who defined for me the relationship between a magazine's editorial integrity and the advertisers who financially support it (readers, sadly, are never a significant source of income, given the high costs of distribution): 'If you tell the truth about components you review, there will always be a small percentage of companies at any one time who are not advertising in your pages. But if you publish the truth, you will have a good magazine. And if you have a good magazine, you will have readers. And as long as you have readers, disgruntled advertisers will eventually return. But if you don't tell the truth, you won't have a good magazine. And if you don't have a good magazine, you won't have readers, at least not for long. And if you don't have readers, you won't have advertisers.'"

A philosophy that is as true now as it was a half-century ago.

John Atkinson
Technical Editor, Stereophile

Well, I really have to challenge this because I think Stereophile does not tell the truth, as a rule, in the reviews. The reason you can get away with this is because of psychological effects and the challenges of honest appraisal of audio equipment. If a reviewer proclaims audio nirvana with some gigabuck DAC, even though it could not even be distinguished in blind testing, never mind consistently preferred, that is a form of sophisticated lying.

A listener purchasing such gear will have no reason to be dissatisfied because the audio performance is probably good enough and their own investment will bias them to like the gear. That doesn’t mean you have provided a value to the consume if they spend money they didn’t have to. (Clearly you would have provided value to the manufacture!)

I just want to reiterate that my opinion here is not based on some philosophical debate over whether gear should be ‘accurate’ vs imparting a ‘signature sound’. My critique does not apply to the speaker reviews. It applies to everything else, and foundation is simply that the lack of inability to distinguish gear like amplifiers, converters, and cables.

The lack of conducting such a test does not let you ’off the hook’ because enough testing has been done to show that most well designed audio gear is in fact sonically indistinguishable. I have a passing familiarity with your long anecdotal experience where you feel that ABX testing fails to capture actual, perceptible, sonic differences in audio gear.

I highly doubt this contention, and as far as I know some research has confirmed the same factors at play in ABX testing are a factor in long term listening.

If there truly was some mystery here about how can people fail to hear real sonic differences in double blind AB or ABX testing that become palpable in extended listening that would merit some skepticism that so called ‘scientism’ can explain these elusive nuances in human musical perception. But there are so many biasing ‘placebo’ effects that are easily demonstrated that there is no mystery. If the placebo effects
explain perceived sonic differences in short time scale testing, there is no reason to think it stops working under longer time scale listening.

Is it your contention that preferences in audio gear that reveal themselves over extended time but not in ABX testing would survive a proper blinding? I highly doubt this. Extraordinary claims still require extraordinary evidence.

Or is it the contention that blind testing is simply invalid as a way to ascertain sonic performance? That would be a profoundly irrational view, akin to believing that blind testing causes the sound-faeries to be sad and abandon their posts.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,388
Location
Seattle Area
The issue I‘m talking about isn’t manufacturers being willing to submit new product for review: my guess is that there will never be a shortage of such manufacturer. It’s the purchasing of ads that is the biasing factor. My contention is that many of the products advertised and reviewed in Stereophile are wastes of money at best and pure snake oil at worst. Actively demonstrating this in the way ASR is doing would damage the business model which is based on symbiotic alignment of interests, not rigorous and honest appraisal.
This is the subscription stats for Stereophile:

1595386592098.png


Whether we impact their business or not depends on who the 75,000 are. If most of them are the high-end audiophiles who have no use for audio science, then their position is safe. If on the other hand there are a lot less of those people, then folks better pay attention to what we are doing here.
 

b1daly

Active Member
Forum Donor
Joined
Dec 15, 2018
Messages
210
Likes
358
Maybe they give instructions to testers during studies but the two times I took them, they just gave us score sheets to fill in. This was my sheet early in the testing:

View attachment 74584

As you barely see, we were told to score from 0 to 10. It is up to you to decide how you figure that out.
Instructions to pick the most ‘accurate’ speaker could hypothetically bias results, though I would not expect this in naive listeners.

I am just confounded by my experiences with ‘accurate’ speakers. We have Genelec 3040s in our main studio and to my ear they are uncannily accurate. If you have mics live in the recording space the playback on the speakers sounds very much like the actual sound in the room.

But on most commercially recorded music they sound ’incorrect’ to me. There are a number of plausible explanations for this:

- the recording was produced on inaccurate monitors which introduces coloration exposed by the Genelecs. There is perhaps some association with eras of popular music.

- the music is produced/mastered for compromised playback systems. This could dovetail with above point. For example, I think the Beatles original recordings sound quite ‘wrong’ on studio monitors. Artists of that era needed to make sure for satisfying listener experience on cheap radios

- taking again The Beatles, I grew up listening to them on all kinds of systems but none remotely like studio monitors. So I have a ‘sense memory’ of how they should sound. Ironically, I find the Beatles recordings remarkable in how well they translate across a bizarrely diverse set of playback environments. Studio monitors are one of the few places I don’t think they translate well.

- whatever perceived deficiency I experience with the Genelecs is simply biased perception that would completely disappear in a blind test. This is highly likely, but the effect is persistent enough for me to question it. I do have a source of bias here in that I associate studio monitors with work, which is stressful, and makes me feel grumpy at times.

- Genres of music, I know this was considered in the Harman studies, but I still think it could be a factor. One issue with the concept of ‘accurate’ monitors in the musical sense is that the way rock music is produced involves a lot of techniques to convey ‘loudness’ at all playback levels. I think most people don’t realize what a profound illusion this. My theory is that playback systems which have some perceptible resonance can ’enhance the illusion.’ Our perception of volume is based in part on our innate ability to perceive the effects of sound energy vibrating physical matter. Even though much of this experience can be captured purely timbraly, having the sound energy of the speaker vibrate the matter of the speaker cabinet itself might help provide some ‘versimaltude’ to the effect.

I have a portable speaker that I quite like, and like most of these speakers the whole physical system is perceptibly energized. This makes rock music sound ‘tough’ especially vintage seventies rock like Aerosmith or AC/DC.

- I am listening to the Genelecs in a recording studio environment. It is highly damped. The speakers are pretty isolated from the floor by heavy stands. The net effect is that the speaker does not energize the room in the way it would in a normal listening environment. In a home listening environment playing a speaker at a relatively high level causes reflections and vibrations that provide cues about the ‘loudness’ of the music that give some of the same effect as the vibrations of the enclosure I discussed. The Genelecs sound pretty ‘inert’ I detect virtually no cabinet resonance at all from them.

For anyone pondering these things who are rock music fans, think about how AC/DC ’should sound.’ It should be shakin’ the walls!

- nostalgia, the speakers I enjoy listening to are from the era I grew up in. This may be purely a ‘placebo’ effect, or it could be nostalgia for actual sound characteristics.
 

North_Sky

Major Contributor
Joined
Oct 31, 2019
Messages
2,741
Likes
1,553
Location
Kha Nada
This is the subscription stats for Stereophile:

View attachment 74586

Whether we impact their business or not depends on who the 75,000 are. If most of them are the high-end audiophiles who have no use for audio science, then their position is safe. If on the other hand there are a lot less of those people, then folks better pay attention to what we are doing here.

Amir, how can you impact someone's else audio business when you're into audio science? To the contrary, you're helping everyone's else audio business.

Unless that by impacting you meant beneficially.
 

Thomas savage

Grand Contributor
The Watchman
Forum Donor
Joined
Feb 24, 2016
Messages
10,260
Likes
16,298
Location
uk, taunton
I would guess he is saying if Stereophile publish a bad review so manuf stops advertising there, there are plenty more manufs who will buy the ad space.

There is an interesting point here. Lots of us feel there is a link between the slant of the reviews in a publication and the fact that the publisher needs ad revenues from the manufs of the reviewed products. An inbuilt conflict for sure. This may play out badly for objectivity in some instances. However, let us not forget the manufs need somewhere to advertise. As do their competitors. Somebody will always want the back page of Stereophile whilst ever it has a readership that manufs wants to appeal to.
There's not many distributors though!
 
Top Bottom