Master Thread: Are measurements Everything or Nothing?

[Jimbob54]

My Spidey-senses are practically throbbing.

 

[Ken1951]

My Spidey-senses are practically throbbing.

Trolls will definitely do that!

 

[dshreter]

But designing audio reproduction electronics, that keep their coloration below human perceptibility, is not a task that require extraordinary talent, IMO.

This is a true statement... but only because what can be designed today builds upon the work of a century of geniuses that did have extraordinary talent.

 

[MattHooper]

Trolls will definitely do that!

I think it’s worth reserving that term to where it would it would really apply.

Everything I have seen from polocolo suggests he is sincere in believing his arguments are worthwhile for consideration. Which is not characteristic of being a Troll.

 

[dshreter]

I should have specified that I meant for things that do sound different – like speakers. We can dive deeper into things that mesure similarly after we've interrogated the relevancy of ABX for things that have actually a particular sound signature.

For things that sound different and where measurements would also indicate they sound different... this is similarly detectable in ABX tests. There is no controversy.

 

[Mart68]

Ok, so, maybe I should say it again (for the fifth time?): I'm not talking about things for which we have no evidence of sounding different (like cables) but of speakers.
Pages 690 to 713 or so were about blind-testing speakers vs sighted listening. So I was still riding that wave.

And then again: yes. Psychology tends to show that emotionally engaging situations tend to change your perceptual skills. So why not here?

I mean it's a pretty well known fact for instance that stress induces "narrow" or "tunnel" vision – selecting the useful information from the field of experience.

If you're trying to say there's no perfect way of selecting a loudspeaker then I agree with you. Not sure there is anyone here who won't.

 

[dshreter]

Ok, so, maybe I should say it again (for the fifth time?): I'm not talking about things for which we have no evidence of sounding different (like cables) but of speakers.
Pages 690 to 713 or so were about blind-testing speakers vs sighted listening. So I was still riding that wave.

And then again: yes. Psychology tends to show that emotionally engaging situations tend to change your perceptual skills. So why not here?

I mean it's a pretty well known fact for instance that stress induces "narrow" or "tunnel" vision – selecting the useful information from the field of experience.

You're correct that one's psychological state significantly influences perception and performance.

The stress of a blinded listening session very well may alter the experience of the listener. But... there's no specific reason to believe this provides advantages to speakers that "truly" sound worse over ones that "truly" sound better. There's no reason to think blind testing favors cheap equipment over expensive equipment. So even if the listener is stressed in some way, is this introducing an error-causing bias?

It is well known that expectation bias can color our perceptions, and so there is a clear rationale for controlling for it. We do that with blind testing. The stress of blind testing can also color our perceptions in some way, but I don't think you have presented a compelling argument for why this is actually problematic.

 

[ahofer]

if we are biased all the time, isn't there a risk that, constructing a context with the goal in mind to eradicate as much bias as possible, we might add some other biases in the context? and if so, how do we know that bias state is more relevant than the one we trying to avoid?

It strikes me as improbable that the biases created by a blind test would sum to more than the sum of the biases from the non-hearing senses and the knowledge of which component is playing. I’d treat this as a rather remote possibility - Russell’s Teapot - until I see some applicable research. Furthermore, I suspect those types of biases are unstable and therefore error might disappear over the large population of blind tests.

In the meantime, controlling for non-audible sensory inputs and direct awareness seems like the way to go.

By the way, there have been a handful of blind tests conducted over long periods of time in peoples’ homes. Not of speakers to my knowledge.

 

[Blumlein 88]

Ok, so, maybe I should say it again (for the fifth time?): I'm not talking about things for which we have no evidence of sounding different (like cables) but of speakers.
Pages 690 to 713 or so were about blind-testing speakers vs sighted listening. So I was still riding that wave.

And then again: yes. Psychology tends to show that emotionally engaging situations tend to change your perceptual skills. So why not here?

I mean it's a pretty well known fact for instance that stress induces "narrow" or "tunnel" vision – selecting the useful information from the field of experience.

It wasn't clear to me that you were referring to only speakers. And I'm not sure that makes a difference.
Second conclusion/first question: does that not entail that ABX tests are not suited to assess the ability of individuals to distinguish/assess/listen critically?

So this is the hypothesis: ABX tests are not a reliable way to test one’s ability to hear differences in music reproduction because the situation of the test itself involve great psychological bias.

It seems most of your first post on this thread is working on the assumption ABX tests don't show differences.

One reason not to think your hypothesis fits is that people do in fact discern quite small differences in blind testing of gear other than speakers. So emotional quality of using music does not seem to prevent them from doing this. This reliably works at levels of difference far smaller than is seen with speakers. Why would the emotional quality not prevent one from hearing small differences in amplifiers, but then prevent one from hearing larger differences in speakers?

Speaker testing is not so easily done blind, but people have been able to discern differences and preferences. Blinding the listeners improved their consistency of evaluation, and the agreement of groups upon which speakers were preferred. This consistency held even when people with various native languages were compared.

 

[ahofer]

It wasn't clear to me that you were referring to only speakers. And I'm not sure that makes a difference.
Second conclusion/first question: does that not entail that ABX tests are not suited to assess the ability of individuals to distinguish/assess/listen critically?

So this is the hypothesis: ABX tests are not a reliable way to test one’s ability to hear differences in music reproduction because the situation of the test itself involve great psychological bias.

It seems most of your first post on this thread is working on the assumption ABX tests don't show differences.

One reason not to think your hypothesis fits is that people do in fact discern quite small differences in blind testing of gear other than speakers. So emotional quality of using music does not seem to prevent them from doing this. This reliably works at levels of difference far smaller than is seen with speakers. Why would the emotional quality not prevent one from hearing small differences in amplifiers, but then prevent one from hearing larger differences in speakers?

Speaker testing is not so easily done blind, but people have been able to discern differences and preferences. Blinding the listeners improved their consistency of evaluation, and the agreement of groups upon which speakers were preferred. This consistency held even when people with various native languages were compared.

I believe rapid comparisons also helped improve discernment. A lot of people think you have to listen for a long time, but rapid switching tends to more easily detect the smaller differences. Mostly an auditory memory thing, I gather.

 

[Blumlein 88]

I believe rapid comparisons also helped improve discernment. A lot of people think you have to listen for a long time, but rapid switching tends to more easily detect the smaller differences. Mostly an auditory memory thing, I gather.

Yes rapid switching helps though if one does not believe it they can do otherwise. If you've done much of this, you quickly learn rapid is better. I would think if the emotional quality of music is some issue, rapid switching diminishes that.

 

[Doodski]

Yes rapid switching helps though if one does not believe it they can do otherwise. If you've done much of this, you quickly learn rapid is better. I would think if the emotional quality of music is some issue, rapid switching diminishes that.

It is difficult to get involvement from the people listening to the test if there isn't rapid switching. They must have the easiest possible test method available and that means rapid switching.

 

[MattHooper]

One reason not to think your hypothesis fits is that people do in fact discern quite small differences in blind testing of gear other than speakers. So emotional quality of using music does not seem to prevent them from doing this. This reliably works at levels of difference far smaller than is seen with speakers. Why would the emotional quality not prevent one from hearing small differences in amplifiers, but then prevent one from hearing larger differences in speakers?

Yes, that’s what I was trying to get at too.

Yes rapid switching helps though if one does not believe it they can do otherwise. If you've done much of this, you quickly learn rapid is better. I would think if the emotional quality of music is some issue, rapid switching diminishes that.

Agreed.

That issue also puts me in the mind of the concept that one needs to live with a loudspeaker for significant period of time in order to assess or review it (this is very common and the subjective reviewing game).

I’ve never felt that way. I mean, certainly one could come to find that overtime your taste may change regarding the loudspeaker so you may become less satisfied with some aspect. But in terms of getting the general measure of a loudspeaker I think that can happen quite quickly.

It reminds me a little of when I was a caricature artist for several years at Canada’s largest amusement park. We would always do caricatures of people in profile. After a while of seeing people sideways and getting the gist of their profile, I could do this astonishingly fast. By that I mean, I merely had to glance at someone’s profile for a for a moment, even someone passing by, and I could render a pretty accurate full caricature drawing.

Similarly, it doesn’t seem to take long at all, playing a bunch of familiar tracks on a loudspeaker to quickly get the gist of its character. I don’t think it takes weeks, much less months.

(though I would throw a caveat in there that, yeah, sometimes I can take a while of experimentation in terms of set up and dialling in so you fully realize the capabilities)

 

[Doodski]

Yes, that’s what I was trying to get at too.




Agreed.

That issue also puts me in the mind of the concept that one needs to live with a loudspeaker for significant period of time in order to assess or review it (this is very common and the subjective reviewing game).

I’ve never felt that way. I mean, certainly one could come to find that overtime your taste may change regarding the loudspeaker so you may become less satisfied with some aspect. But in terms of getting the general measure of a loudspeaker I think that can happen quite quickly.

It reminds me a little of when I was a caricature artist for several years at Canada’s largest amusement park. We would always do caricatures of people in profile. After a while of seeing people sideways and getting the gist of their profile, I could do this astonishingly fast. By that I mean, I merely had to glance at someone’s profile for a for a moment, even someone passing by, and I could render a pretty accurate full caricature drawing.

Similarly, it doesn’t seem to take long at all, playing a bunch of familiar tracks on a loudspeaker to quickly get the gist of its character. I don’t think it takes weeks, much less months.

(though I would throw a caveat in there that, yeah, sometimes I can take a while of experimentation in terms of set up and dialling in so you fully realize the capabilities)

I've heard a lot of speakers and compared them instantly via switching. I find that with a proper selection of product and selecting the several speakers that all sound pretty good takes a few minutes and then after comes the more extended decision making process of what speaker to take home and learn to like. I also found by winning many speakers in sales contests and buying speakers that they all sounded good once they where in my home and that I learned to like all of them because even though the character of them was different they still presented a entertaining immersive experience. It's all about the experience and one becomes accustomed to the sound of a speaker and then all other sound foreign until replacement time comes and then the process repeats.

 

[krabapple]

Oh ok. Maybe I misunderstood some of the stakes of the conversation then.

What I got from the whole debate (like pages 690 to 710 or so) was: you can't trust sighted listening for speakers you should trust only blinded listening. And I was wondering if we weren't adding a whole other set of biases because of the influence of context on perception.

Blind testing of preference is a perfectly conventional research tool. It certainly has been used to study preference for loudspeaker sound.

Are you aware of any of that research?

 

[MattHooper]

I've heard a lot of speakers and compared them instantly via switching. I find that with a proper selection of product and selecting the several speakers that all sound pretty good takes a few minutes and then after comes the more extended decision making process of what speaker to take home and learn to like. I also found by winning many speakers in sales contests and buying speakers that they all sounded good once they where in my home and that I learned to like all of them because even though the character of them was different they still presented a entertaining immersive experience. It's all about the experience and one becomes accustomed to the sound of a speaker and then all other sound foreign until replacement time comes and then the process repeats.

I have essentially two modes when I’m evaluating audio equipment in particular loudspeakers. On one hand, I’m always curious to hear a system, just to hear the characteristics of music through that system.
I find it very entertaining, which is why I love auditioning, different loudspeakers, going to shows visiting audiophiles homes, etc.

The other is when I’m evaluating a loudspeaker in terms of “ how much do I actually like listening to music through this?” And other words “ do I want to own this?”
And then I get to a whole different level of picky.

I have found that, in that sense, I do not acclimate to speakers over time. If I’ve been able to get a gist of a speaker from playing a bunch of music through it, I can tell very quickly whether I like it or not. And that impression never changes. If a speaker doesn’t grab me at first it never does.

I’ve had a couple of loudspeakers in my room who is characteristics didn’t make me want to keep sitting there, listening to music… usually due to a type of tone or timbre that I don’t care for. And that never changed over weeks or months which is why I got rid of them.

And on the other hand if a speaker “ speaks to me” quickly in an audition, the factors that grabbed me remain very constant, and if I end up owning that loudspeaker those characteristics seem very constant over years.

 

[MAB]

Blind testing of preference is a perfectly conventional research tool. It certainly has been used to study preference for loudspeaker sound.

Are you aware of any of that research?

We keep suggesting he read up on actual studies, of anything.
He seems immune to that.

I actually think he made the classic mistake:
1) Enter a room knowing nothing about both the contents of the room
2) Get on the table in the center of the room and orate a long winded manifesto making a slightly comical fool out of one's self
3) Double down

It makes it even funnier that his rant was in the dumpster thread. Goes back to entering a room and knowing nothing...

Step 3 could be different, but none of this is sincere, despite his repeated statements to the contrary. Which is what doubling down is.

 

[Doodski]

I have essentially two modes when I’m evaluating audio equipment in particular loudspeakers. On one hand, I’m always curious to hear a system, just to hear the characteristics of music through that system.
I find it very entertaining, which is why I love auditioning, different loudspeakers, going to shows visiting audiophiles homes, etc.

The other is when I’m evaluating a loudspeaker in terms of “ how much do I actually like listening to music through this?” And other words “ do I want to own this?”
And then I get to a whole different level of picky.

I have found that, in that sense, I do not acclimate to speakers over time. If I’ve been able to get a gist of a speaker from playing a bunch of music through it, I can tell very quickly whether I like it or not. And that impression never changes. If a speaker doesn’t grab me at first it never does.

I’ve had a couple of loudspeakers in my room who is characteristics didn’t make me want to keep sitting there, listening to music… usually due to a type of tone or timbre that I don’t care for. And that never changed over weeks or months which is why I got rid of them.

And on the other hand if a speaker “ speaks to me” quickly in an audition, the factors that grabbed me remain very constant, and if I end up owning that loudspeaker those characteristics seem very constant over years.

That sounds like my experience with the KEF C Series and the KEF Reference Series that I've owned and paid for with my own money. Upon listening in the store for extended time periods I found them to be a relaxed sound that was non-confrontational. They are also the speakers that I owned for the longest time period too. Very easy to sit and be absorbed for hours at a time. Those required up to months to select out of the available good choices that I had available at the time. I could have chosen from the better lines that we sold which where JBL, Polk, ENERGY, Mission, MB Quart, Yamaha, Infinity and B&W although after comparing my favorites of all those brands the KEFs where my go to speakers. I could have gone with Infinity but they where too amp dependent, the JBL rocked really good but where a bit confrontational/shouty, the B&W where too crispy on the top end, The ENERGY where in the final running but when compared to the Polks they sounded a bit flat and the Polks a bit bright and lacking punch, the MB Quart (Which I owned/won 2 models of.) where very detailed in the highs and so much so that I decided I did not want to chance them with my own money, the Yamaha where midrange oriented and the Missions I never thought stood out in the crowd for my taste. I could have been happy/acclimate with any of them but as you say I was choosing with my wallet and my experience level and the immersion factor. Over the years that I owned many speakers I enjoyed them all but they where all short term experiences as I sold them and either bought or won more of them. It was relieving to have the KEFs because I could depend on them for anything I played through them and no more swapping out gear and never getting full satisfaction for up to a complete day of listening at a time.

 

[Blumlein 88]

In the past some speakers grabbed me and I knew I liked them. In my home it sometimes took time to work around issues. A couple over the years would be love/hate. I liked what I liked while other aspects were hard to live with and not solvable. That is something so nice with Revel or similar. They're easy to work with. They may not grab your attention as much with some areas of superlatives, but get the below 300 hz worked out then you are good to go.

 

[MattHooper]

In the past some speakers grabbed me and I knew I liked them. In my home it sometimes took time to work around issues. A couple over the years would be love/hate. I liked what I liked while other aspects were hard to live with and not solvable. That is something so nice with Revel or similar. They're easy to work with. They may not grab your attention as much with some areas of superlatives, but get the below 300 hz worked out then you are good to go.

After living with the Quad ESL 63s on top of gradient, subwoofers for quite a while for several reasons, I decided to replace them with skinnier profile box speakers. After living with a spacious boxless-sounding presentation, any trace of boxiness and loudspeaker stuck out and I was trying to avoid that. I ended up replacing the quads with Von Schweikert VR4 Gen 2 speakers which had blown me away in terms of sounding ultra three-dimensional, full range and lacking a boxy signature. The one demerit was that I found the overall timbre of the sound, voices instruments, a bit bland.
I really generally liked those loudspeakers, but the perception of their tonality never changed, and overtime that was why I sold them, especially to replacement with a speaker whose tonality I loved.