[YouTube] The Big Measurement & Listening Mistake Some Hi-Fi Reviewers Make - SoundStage! Real Hi-Fi

[MattHooper]

Whaaaaat??? You were a subjectivist maganize reviewer? All this time you didn't tell us this? You didn't think it was important to share that you were defending fellow subjectivist reviewers? I was wondering what the source of tenacity was. Now I know.

Amirm...

Why did you quote this part of my post:

MattHooper: "When I wrote reviews for an on-line audio magazine,"

While leaving off this part:

MattHooper: "As I've mentioned before: When I wrote reviews for an on-line audio magazine, I faced no pressure to write anything other than what I thought about a product."

And then suggested that I have never told anyone I did some subjective reviews? As if I was hiding it for some reason.

I don't expect you to have seen or memorized my every post, but I DO expect at least the charity of not cutting off obviously pertinent parts of what I write, which should get you to at least stop and consider what it meant, e.g.: "He said he'd mentioned it before...maybe he's not lying about that and I happen to have missed when he's brought it up. I'll ask him about when he's mentioned it before, rather than assuming he's hidden it." A bit of charity can go a long way rather than leaping to disparaging conclusions.

Aside from the fact I mentioned it earlier in this very thread as well, I have mentioned my old reviews and the fact I still know some reviewers numerous times on the forum, where it seemed relevant:

From my "introduction" on the Why Are You Here thread:

https://www.audiosciencereview.com/forum/index.php?threads/why-are-we-here.10181/post-279559

But I always had a strong rational/empirical side and always took the more dubious or controversial audio claims with a big grain of salted skepticism. Along those lines, when i was offered a gig doing reviews for a fledgling on-line audio mag, I stuck to reviewing speakers and refused to review cables/amps/tweaks. My conscience wouldn't allow it ;-)

Other examples:

-------------------------------

https://www.audiosciencereview.com/forum/index.php?threads/unusual-speaker-designs.12255/post-481607

Long version - My old review of the Waveform Mach Solo:

http://stereotimes.com/speak020200.shtml

(That's one of several speaker reviews I wrote "back in the day" - you may find it interesting in that it delves a little in to the design, though that was a subjectivist rag and hence no measurements)

https://www.audiosciencereview.com/...eve-guttenberg-audiophiliac.14695/post-460855

It's a very common assumption among the cynical that the preponderance of good reviews points to shenanigans between the reviewer and manufacture.

However, I can say as someone who did a bit of audio reviewing for an on-line mag years ago, I was never pressured to write anything other than what I thought about a product.

https://www.audiosciencereview.com/...odern-day-snake-oil-salesman.5800/post-163066

For a little while, about 20 years ago, I did some reviews for a high end audio on-line mag. I got on board on the condition that I'd only review speakers, and those of my choosing. Even back then I just couldn't with good conscience review things like cables (even amplifiers) given how aware I was of the reasons to doubt the claims associated with them. And no cable manufacturer was going to give me their expensive cables to blind test them


https://www.audiosciencereview.com/forum/index.php?threads/who-do-we-trust.7506/post-187732

For a little while I did reviewing for an on-line mag. It was a totally subjectivist mag, but I had refused to "review" things like cables or even amps...I stuck only to speakers.

https://www.audiosciencereview.com/...r-a-subjective-rating-system.8536/post-215991

When I was reviewing I tried as best I could to transcribe "what the speaker sounded like" to words, and I very often had responses along the lines of "Wow, I too have auditioned that speaker before and didn't end up liking it as much as you; nevertheless you described exactly the sound I heard!" I've seen lots of similar success in agreeing on sonic descriptions between audiophiles.


https://www.audiosciencereview.com/...le-talks-about-measurements.11004/post-311026

I used to do a bit of reviewing and have some pals who still do it.


https://www.audiosciencereview.com/forum/index.php?threads/audio-jewelry.9831/post-266204

My only experience in this is simply being an audiphile, having known a great number of audiophiles, and having been a denizen of audiophile forums since the 90's (and also having been part of the high-end reviewing crowd, so seeing some behind the scenes). So I hope to simply bring to a conversation like this what I have observed in terms of the behaviour and apparent motivations of actual audiophiles.

-----------------

And, aside from implying I never mentioned it and was hiding the fact for some reason, what exactly would your point be about my having done some subjective reviews? Is it to impugn the motivation I have for my arguments? I stopped reviewing 20 years ago. Should we assume yours are impugned because they are just defensive reactions given you have a web site to defend? I hope not. The reason you defend your position is that you think it makes sense. It's why you started a web site to defend it. The reason I defend *some* aspects of reviewing is that *some* aspects of subjective reviewing make sense to me. The arguments either stand or fall on their own, not on implications of ad hominem or bias or appeal to emotional motivations, correct?

 

[USER]

This is getting boring.

 

[MattHooper]

15,000 words of passive aggression in this thread are instantly explained.

On that same logic, and lack of charity, one could say that all of Amirm's "passive aggression" in defending his reviews against so many other reviewers is instantly explained by the fact he's a reviewer with a web site to defend and justify. "Oh, you constantly argue against all those other review magazines? I see you are a reviewer yourself with your own agenda, so I can dismiss your arguments as mere passive aggressive emotion. so I don't have to tend to the actual points you are making."

Sound good?

No. Because that's a facile, uncharitable reaction. The arguments are either good or they aren't. Does this site really want to be on the side of Ad Hominem rebuttals?

(And note that I have ceded MUCH more to the points AGAINST subjective reviewing than Amirm and some others have ceded to my arguments, which seems about zero. So I'm not being dogmatic).

 

[MaxBuck]

I appreciate the input provided by John Atkinson here. Let me say that no one should expect Stereophile to be an ultimately objective reviewer of audio equipment, any more than they should expect Road & Track to be an ultimately objective reviewer of automobiles. Both publications are limited to some extent by the imperative to maintain advertising revenue, which consumers need to recognize. Consumer Reports unfortunately doesn't review hifi equipment any more (and when they did, not sure they understood what they were doing).

 

[GaryH]

Consumer Reports unfortunately doesn't review hifi equipment any more (and when they did, not sure they understood what they were doing).

They certainly didn't, their loudspeaker ratings actually having a negative correlation of -0.22 with actual listener ratings in Harman's double-blind tests

Recently, their headphone ratings came out on top however among these few:

Still worse than Harman's predicted preference ratings calculated purely from objective measurements of course.

 

[Ken1951]

This is getting boring.

Indeed. Ground has been plowed, raked, re-plowed, re-raked, etc...

 

[amirm]

Who in this thread is a part-time vegetarian? I don't think anyone here is against what you are saying about knowing the cost, model, and design can be possible sources of bias, as well as knowing the measurements before doing the subjective review. If you take away one of them the better, like not knowing the measurements before listening, and if you take away all the rest, it's even better.

No, there is no research or science to back any of this, that taking this or that out is "better." Proper speaker testing is multi-way, comparing speakers against each other. In that sense, the comparison needs to be blind so that the look, design, etc. doesn't interfere. You are also supposed to test speakers in mono configuration for most discrimination.

A reviewer listening to a pair of speakers, sighted and without comparison to any other speakers, is doing it all wrong. To say that "I am not looking at measurements therefore I am more scientific" is all wrong. There is nothing scientific. You either do things right, or you are not.

With proper measurements having high correlation with listening preference of large population, we can at least steer the reviewer in the right direction. No longer will they be ready to declare a completely broken speaker design as one of the best they have heard. Or even that it sounds great for price. Your doctor has a ton of experience and even he starts with your weight, blood pressure, temperature, etc.

I have measured and listened to speakers that are atrocious. Yet they have thousands of 5 star reviews on Amazon. Those people listened without looking at measurements. In what way was their assessment more accurate than mine? Don't you think if they learned to read measurements and then listened, they would be less inclined to say that is a great speaker and that would be the more correct assessment?

I don't understand why this concept is so hard to understand once explained not once, but a dozen times.

 

[amirm]

If you happen to be a hardcore objectivist, then it's likely that knowing the measurements before giving your subjective verdict will give you an even bigger impact bias-wise, depending on what you saw in the measurements.

That is lay intuition speaking, not reality. Why would a "hard core objectivist" listen to a speaker if that is true? I will answer that for you: they do because they want to know if the measurements speak the truth. That is why I do it. We all know that some aspects of the speaker performance we can't measure or measure directly. So we listen to see what the real deal is.

I assess a speaker by using equalization to correct is measured performance. By definition I need to look at the measurements to develop the EQ. But once there, I can do what no other reviewer can or is doing: I can perform instant, blind tests if needed, of that EQ. I can bring a bunch of other people to listen just as readily and find out if the measurement is speaking the truth or not.

In that way, I am able to correct whatever gaps there are between measurements and listening. I am not after discovering random, totally unreliable things like "how congested instruments are below 1 kHz in this track." That is voodoo nonsense.

Look at the bottom score in the speaker review index:

There are four speakers there with awful preference score that I liked and recommended. So clearly the measured performance did not bias me to do otherwise.

Nothing is more clear on this than in headphone testing where I routinely give the worst scores to uneq version and great to eq version. This is the only way to verify efficacy of measurements.

So the goals here are very different. The subjectivist reviewer wants you to think that in totally unscientific method, he can tell you if a speaker is good or not (although most avoid every saying a speaker is not good). My goal is to quality aspects of speaker measured performance.

Great example of this is in the KEF LS50 Meta I just tested. It showed variations in response which I corrected with EQ but it made no difference. So I overrode the aberration there and gave it high marks.

 

[amirm]

Talking about things that can go both ways even if you see the measurements before you do the subjective part of the review. Did you ever get to a definitive conclusion why you did find the overall experience of the KEF R3 "unexciting and unengaging for lack of a better word", did you manage to tweak them to your liking and get them more exciting and less unengaging?

That was very early in the process for me until I landed on the methodology I am advocating and using now. That aside, no, I am always willing to tell you that I can't figure out what the sound is telling me. Try to get that out of any other reviewer. They always have an answer for you and most of the time it is positive.

 

[amirm]

And then suggested that I have never told anyone I did some subjective reviews? As if I was hiding it for some reason.

You were hiding it. In this back and forth with me, you should have made this front and center in your posts. That you speak for yourself and other reviewers like you. Instead, you kept talking about other reviewers as if you are not one of them. This lack of transparency is super disappointing but does explain the great lengths to which you have gone to defend the population you are part.

 

[MattHooper]

You were hiding it. In this back and forth with me, you should have made this front and center in your posts. That you speak for yourself and other reviewers like you. Instead, you kept talking about other reviewers as if you are not one of them. This lack of transparency is super disappointing but does explain the great lengths to which you have gone to defend the population you are part.

Unbelievable.

This is frankly disgusting, Amirm. When confronted with the bold fact you were wrong about my not mentioning having reviewed before, you double down and accuse me again of "hiding" the fact.

In this VERY THREAD I mentioned reviewing earlier, before you brought it up. You were just presented with a long list showing I'd written numerous times about having reviewed long ago. (Which also included explanations for why I wouldn't review things like cables and tweaky stuff). And I have often mentioned some friends who are reviewers, in various threads, to make various points. Yet you are accusing me of a "lack of transparency?"

I'm not "one of them" as I am NOT an audio reviewer, haven't been one for 20 years, and even then it was just a handful of reviews.
So that is PRECISELY how I have represented myself. It would be more misleading to present myself as actually "being one of them" implying I'm still a reviewer.

What do you need? A disclaimer at the bottom of every post "I Once Reviewed Some Speakers 20 Years Ago." ??

What the heck is the actual RELEVANCE of your exasperated sounding comments? Like, if you knew (instead of ignored) that I'd reviewed some speakers 20 years ago, on that basis my arguments can simply be dismissed? Or that I'm not arguing for a position on the reasons I believe I have for holding the position...like you and literally everyone else here?

 

[amirm]

What do you need?

Simple. When you interact with me on this topic, you make that front and center. You know who I am front and center. I had no idea who you were and read that as a pure reader of reviews. Nothing about the tone of your responses owned up to this. It was quite a surprise to read that you had such a history.

What the heck is the actual RELEVANCE of your exasperated sounding comments?

I would address you very differently knowing what I know now. I would have probed you as a subjective reviewer, just as you probe me as who I am here.

You may have different ethical standards as I. Mine says you say front and center who you are. That is why I put what I put in my signature. This is why the second sentence in ever review says where I got the product under test. This is why there is a prominent notice at the top of the review if there is remotely a conflict of interest. You seem to think that in a topic where we are discussing subjective reviewers, the fact that you were one is just "oh by the way." It is not. It is the very topic of the thread and I would have expected you to make that clear in the first response to me and repeated in later responses.

If you think this reaction is "disgusting," it is nothing of what I think of your conduct in this regard. I suggest moving on and getting someone who doesn't mind such lapses in transparency and ethical conduct.

 

[Newman]

16,000 and mounting, and Matt’s passive aggression is escalating.

 

[Robin L]

Simple. When you interact with me on this topic, you make that front and center. You know who I am front and center. I had no idea who you were and read that as a pure reader of reviews. Nothing about the tone of your responses owned up to this. It was quite a surprise to read that you had such a history.


I would address you very differently knowing what I know now. I would have probed you as a subjective reviewer, just as you probe me as who I am here.

You may have different ethical standards as I. Mine says you say front and center who you are. That is why I put what I put in my signature. This is why the second sentence in ever review says where I got the product under test. This is why there is a prominent notice at the top of the review if there is remotely a conflict of interest. You seem to think that in a topic where we are discussing subjective reviewers, the fact that you were one is just "oh by the way." It is not. It is the very topic of the thread and I would have expected you to make that clear in the first response to me and repeated in later responses.

If you think this reaction is "disgusting," it is nothing of what I think of your conduct in this regard. I suggest moving on and getting someone who doesn't mind such lapses in transparency and ethical conduct.

This is what I mean by "puritanical objectivist."

Matt has let a lot of folks here know on numerous occasions that onceuponatime he wrote subjective reviews.

It's not his fault that you haven't paid attention.

 

[amirm]

16,000 and mounting, and Matt’s passive aggression is escalating.

Indeed. This is his first response to me:

Complete defense of subjective reviewers but didn't think to mention he was one himself?

 

[amirm]

Matt has let a lot of folks here know one numerous occasions that onceuponatime he wrote subjective reviews.

It's not his fault that you haven't paid attention.

Come again? You don't think he needs to start his arguments with me by saying so? I was supposed to have read his past posts in who knows where to know him?

 

[Robin L]

Come again? You don't think he needs to start his arguments with me by saying so? I was supposed to have read his past posts in who knows where to know him?

If you've read anything by him, you would know he's a subjectivist.

Nobody expects a Spanish Inquisition.

 

[Robh3606]

Nobody expects a Spanish Inquisition.

 

[MarkS]

Although I am a hard-core objectivist (PhD in physics from Stanford, using 16-gauge lampcord for speaker wire since 1979), I find the notion that any and all subjective reporting has no value to be quite absurd.

Of course it's possible to tell good sound from bad sound by listening alone. If it wasn't, we could just go buy the cheapest speakers available (which is what I actually do with electronics, since I am convinced that there is no sonic difference). If you can't tell the difference by listening, why pay more for good measurements only?

My anecdotal experience: in the early 90s, I went to a big audio show at a hotel in Los Angeles. Spent two days wandering from room to room. In one room, the music sounded almost magically lifelike. I inquired about the apeakers in use (which annoyed the exhibitor, an amp company that wanted to tout their amps, but I didn't care about that at all). They were Virgos by Audio Physique, a German company that I had never heard of. These speakers then went on to be very well reviewed by the audio press, and IIRC were eventually the least expensive speakers named on Stereophile's Class A list.

I believe this experience was real. The Virgos were very good speakers, and I was able to hear that.

 

[amirm]

Nobody expects a Spanish Inquisition.

You would if you had spent a ton of time answering him, and suffering through his long missives....