First of all thanks very much for your comment. And “ long winded” will certainly follow me to my gravestone
I’m just going to reply to this, and if Rick finds it too off topic he can end up deleting it.
Yes, this is a particularly long one, and I certainly understand anybody taking a FRAT response. But I feel that laying out my case requires a number of examples, so I’m just putting it out there.. as a summation of my position on this..
I don’t think I’m working with the level of naïveté you are inferring.
I have propounded as much as anybody here the nature and relevance of bias effects. Bias effects in audio are well demonstrated by the type of research cited here all the time. I’ve experienced my own bias effects vanishing under blind testing . (In fact my own work in sound involves exploiting bias effects).
So the conclusions I’m drawing are not in the context of ignoring bias effects, including my own, but they are trying to keep a coherent picture of what type of reasonable inferences we can still make in informal listening
while considering the possibility of bias effects. Because most of the time we are operating in conditions where we cannot control for bias effects, we therefore have to arrive at practical, even if not fully certain conclusions.
So in my view, the following is reasonable:
1. If you are a scientist looking for scientific levels of certainty, then you may as well disregard any uncontrolled listening. You can even just throw it in the “bias” bin because it simply doesn’t offer the reliability for the confidence levels you are seeking in order to understand what’s going on.
2. An ASR member can simply disregard any uncontrolled listening reports from reviewers or other audiophiles and when choosing gear can even disregard his own impressions as untrustworthy. “
I’m looking for the most reliable information on which to make my decisions - uncontrolled listening won’t cut it, and I’m looking towards measurements or at the very least controlled listening tests” is a perfectly reasonable approach for somebody who just looks to measurements.
So “Listening in uncontrolled conditions, especially without supporting measurements, is unsuitable for gaining the level of confidence I’m seeking” is entirely reasonable.
What goes too far is the idea that listening in uncontrolled settings is always uninformative and nobody can be justified in making any inferences under such a situations.
THAT is what I push back against. And this level of scepticism - it’s probably just your imagination so I don’t have to take what you’re saying seriously - is often enough thrown around here when people don’t feel like accepting a claim, even for the sake of argument .
We can and do come to reasonable conclusions with lower than scientific confidence levels all the time. Otherwise, we couldn’t get through the day. I’m bringing mini instances of my own experience and asking what explanations make the most sense of those impressions.
And unfortunately, I find that sometimes the default move to “
it’s likely just sighted bias effects / imagination” starts to look fairly hand wavy when it gets to brass tacks.
So when it comes to putting any stock in my or some other audiophile’s informal, listening impressions I work on basic heuristics;
Extraordinary claims require extraordinary evidence. Is the impression of the gear implausible? Does it suggest something technically implausible? Or does it go against any known measurements of that gear?
And:
What is the best or most reasonable hypothesis or explanation for a given subjective impression?
So sticking just to evaluating loudspeakers:
If the hypothesis about somebody’s subjective impression is that it is a “ bias effect” then that suggests “
the loudspeaker doesn’t really sound like you think it does; it sounds different than what you perceive.”
Well, if that’s going to be one hypothesis, its explanatory power should be put against another possible hypothesis “
The sonic impressions are to some relevant degree accurate; because that really is what the speakers sound like!”
So let’s see these different hypotheses in action:
I auditioned the Revel Performa speakers a few times. I went in knowing how Revel speakers generally measure. So that could have biased my perception. Except the first audition didn’t go well - I was surprised by the substandard performance in which the speakers didn’t sound smooth, but somewhat rough in the highs and uneven in the bass. But I quickly realized this was due to a poor set up - as the speakers were placed too close to the back wall and one was near a large reflective wall of glass. The poor set up seemed to overpower any bias I might’ve had that the speakers would sound better.
However, in a different store the speakers were set up much better.
And I evaluated them in my normal way: listening to some of the same tracks I’ve used on countless different loudspeakers, and I evaluate the sound from further seating distances, middle and nearer field distances to see how the sound holds up - not every loudspeaker sounds coherent up close. I evaluate the sound in the vertical domain from kneeling down, sitting, standing to see if there’s any “venetian blind” effect or obvious changes, roll off in the highs or whatever. I walk around the speaker, listening from different angles to check off axis performance, both in terms of changes in tonality and changes in imaging - does the sound glom in to one speaker we’re moving off axis, or maintain a sense of spaciousness and imaging between the speakers? Etc.
What I perceived from the Revels was a beautifully balanced sound. Well controlled and even sounding from top to bottom. No obvious colorations. Even if there were room nodes they were not intrusive. Smooth off axis performance. Very neutral while being smooth and easy to listen to.
And all of this is predicted by and consonant with the way those speakers measure.
So what’s the best explanation for my impression? Was it just that I was biased to hear them that way and they didn’t really sound that way? And if they didn’t sound the way I perceive them how did they actually sound? What objective evidence disputes my impressions? It seems to be objective evidence in terms of measurements actually support my impressions.
It seems to me at least as reasonable, Occam's razor style , that the reason I had those impressions is because that’s how the speakers actually sounded. Very much as their measurements help predict.
The same could be said for the number of times I listened to the
B&W 804 D4 (and D3) loudspeakers.
What I heard was a very open, very detailed spacious sound, with generally well controlled and not over-rich bass, quite “ free of the box” sounding from top to bottom, but also a lack of warmth in the midrange and some peakiness in the upper mids and treble region. It clearly wasn’t neutral, but instead that modern sculpted B&W sound.
This was especially obvious when I heard them the same day that I heard the same tracks earlier on the more neutral Kii Audio Three speakers.
So what’s the best explanation for my impressions? Is it better explained as a bias effect because I do know how B&W tends to measure? Could be.
On the other hand… the measurements DO generally describe how the speaker will sound, and also can comport with my listening impressions.
So is the best explanation for why I heard the B&Ws to be less neutral and more peaky sounding in the upper frequencies than the Kiis a bias effect or… that I was actually hearing what they sound like, which is predicted by the measurements. Put that together with the fact that John Atkinson also report reported the same impressions about their lack of neutrality.
Again, it seems to me entirely reasonable to provisionally conclude I was perceiving the essential characteristics of that loudspeaker.
But then there’s the many other examples of where I listened to speakers before I was aware of any measurements.
I auditioned the
Paradigm Personas when they were brand new in a local high-end store.
I found plenty to admire in terms of their amazing clarity, and they seemed generally very well, balanced, similar to the Revel.
With the exception that I kept noticing a sharp peak somewhere in the highs that over time was wearing me out. It didn’t seem to be showing up in vocal sibilance so much as being somewhere maybe higher up. At the end I found I want to keep turning the volume down and ultimately found my ears fatiguing, so I gave up on those speakers.
And later Kal reviewed those speakers in stereophile. His description was almost word for word in terms of the qualities I heard in those speakers INCLUDING his noting a peak in the highs that he guessed would be around 10 K. And sure enough in the measurements there it was. A sharp peak right at 10 K!
So what’s the best hypothesis for both my and Kal’s sonic impressions both made before seeing the measurements?
We just happen to have the same bias that produced precisely the same sonic impressions including a peak in the highs… and all the sonic impressions just happening to lineup with the measurements?
Seems to me something like Occam's razor allows the practical inference that we were simply actually hearing what that speaker really sounds like.
That happened again with the
PMC Fact 8 speakers. My friend had those in for review and I listened to them. I was impressed by their open spacious sound, which sounded very clean and detailed.
But I was largely turned off overall because they sounded, to use that old term “ too hi-fi” - in the sense of exaggerating the artifice in recordings in the highs and not really sounding as natural as I like. In particular they sounded very “cool” and reductive, lacking warmth somewhere maybe in the lower mids or upper bass, I didn’t know, but it lacked body and warmth for male vocals and anything that usually had more body that I’m used to other speakers like my own.
After that I saw Kal’s review of the same speakers in stereophile and… again… his descriptions matched what I heard all the way down to him mentioning the same lack of warmth “ in the upper and mid bass.”
And there it was the Stereophile measurements! A bit of a roller coaster in the on axis frequency and JA’s comments only in room response he measured “….
the PMC fact.8s' in-room response is shelved down in the lower midrange and bass and has significantly less presence-region energy.“
Again… best explanation for why Kal and I perceived the same characteristics and flaws in the speakers that turned out to be continent with the measurements?
Seems reasonable that our perception was relatively accurate to what the speaker actually sounds like.
I can keep going with all sorts of examples. The
YG floor standing speakers that my friend reviewed, where he asked me over to listen without telling me what he thought.
These were sizeable floor standing speakers and I was expecting on my test tracks to hear something probably down to 35 Hz or so a similar presentation that I’ve heard from similar size speakers like my own. And yet I was shocked at the lack of bass, and also the consequent sense of emphasis in the upper mids and higher frequencies. These matched my friends own impressions exactly which is why he was having so much trouble with the speaker.
And when they were measured sure enough, the bass was very underdamped - they actually start sloping down around 200 Hz and fall down fairly steeply after 50hz.
And this was ameliorated by placing them near the corners.
Again… just a bias effect, or just some form of coincidence that my friend and I perceived the same characteristics, which were surprising to both of us given the size of the speaker, but which happened to be consonant with the measurements?
Seems entirely reasonable we simply heard what the speaker really sounded like in that room.
What about my own Joseph Perspective speakers? I didn’t even know those speakers existed let alone having read reviews before I heard them for the first time at the dealers. And my impressions have remained the same from that first time all the way to my owning them for years now. And they are consistent with what JA heard and measured in Stereophile (with the exception that the highs did not bother me in the first version of the model as much as they did JA, even though I could hear that they were tipped up).
Once I had my Joseph speakers dialled in at home and I had experimented with some acoustic diffusors I was blown away by the complete disappearing act of the speakers, the massive, enveloping sound stage (on appropriate record recordings), the richness yet tight quality of the bass, the beautiful clarity of the mids and the incredible smoothness and airiness of the highs. And especially with the diffuser, an amazing solidity and palpability to the images appearing in the vast sound stage.
Once I had that set up, I invited my reviewer pal over and just sat him down to listen and give me his own impressions before I told him my own. We do this kind of thing when we have new speakers, double checking our impressions with what the other guy hears.
He was completely shocked and basically said “ how the fuck did you do this?” When I asked him to describe the sound he described what I hear to a “T” - his first comments were the crazy sound staging and imaging, he noted that on tracks with the lowest bass the bass was a little bit rich but that he loved it anyway, because it was still really tight and rhythmic. And he commented on the general clarity and especially on the incredibly relaxed quality of the highs “ I can just keep turning up the sound and it feels like I could just listen to this all night without any fatigue… the highs are buttery smooth.”
Exactly what I perceived in the same recordings. Just coincidence? My friend listens to plenty of speakers that sound great, is there some reason that these speakers cause a particular bias effect where we both perceive the sound characteristics the same way?
And his stereophile review of the Perspective 2s JA noted wide full range sweep of sound as well as the clarity and smoothness in the highs even during complex passages, and also noted a slightly rich and yet still tight and punchy bass quality.
None of this seems disputed in the nature of the Stereophile measurements.
So are we all suffering the same bias effect?
It seems at least as reasonable to conclude, provisionally, that we are largely perceiving the actual characteristics of this loudspeaker.
So those are just a few of many different examples along the same lines, where I and other listeners impressions seem to converge fairly well on the general character of loudspeaker, and when measurements are available very often the impressions are consonant with the measurements, at least certain characteristics.
I’m not talking all the time perfection here. But it seems to happen often enough. So from my own experience, I conclude that even in the context of its known liabilities, informal listening CAN provide some useful information where the impressions are technically plausible. It can be at least reasonable to draw some conclusions, with lower confidence levels and caveats, even if such conditions won’t suffice for scientific levels of confirmation and insight. Sighted listening to loudspeakers is significantly less reliable than blind listening, but not necessarily wholly unreliable or wholly uninformative.
Cheers.
(I should print this on a scroll and be buried in this one…)
