Master Thread: Are measurements Everything or Nothing?

[polocolo]

The test though, that I figured we are talking about, has to do with detecting Sonic differences not whether one is being moved by music or not


I mentioned in my first reply that if one is suggesting that the music itself, due to the type of emotional investment you suggest it demands, creates a variable, not only is that variable controlled for by keeping it constant, but separate tests, such as that I suggested can test for the claim about that variable. So as far as I can tell, the point you were bringing up was already addressed and my reply.




OK, I still think there seems to be a bit of slippage in terms of what is being tested for , engagement or the audibility of Sonic differences.

But as I understand it here you are saying that engagement level with the music can affect the ability to discriminate between Sonic differences. And that if the ABX conditions subdue that engagement, that may be lowering the discrimination abilities of the participant.

I guess I’d say that, since you’d want to be careful about begging the question here, you would want to supply evidence for that claim.
I’m actually not familiar with standpoint theory. But I think some evidence would have to be presented not only for that theory, but that the phenomenon uncovered is likely to be a problem variable in the specific type of testing we are discussing. And what type of audible threshold might we be talking about in terms of this “ lack of engagement” lowering discrimination? My wife has plenty of engagement with Taylor Swift’s music. I have essentially none. But I’m pretty sure I could use a Taylor Swift track to discriminate between audible differences in an ABX test.



I’m not totally clear whether you’re argument is laser targeted on the ABX format, or whether it is to apply to any proper blind, listening test.

But since you were asking about my experience, here’s some experience:

I had a couple CD players and an outboard DAC in the 90s. They seemed to sound slightly different to me. I would play plenty of my favourite tracks on those CD players depending on which I had in my system, which I’m very engaged in.
I decided to do a proper blind test between them (volume level matched at the speaker terminal, random switching with a helper, etc.)
It turned out the same audible properties I was identifying in normal sighted listening were there to be heard in blind and so I could easily tell them apart. These were subtle (obvious to me, but still subtle ) differences.

This suggests the “ stress” of the blind testing did not Interfere with hearing the type of subtle Sonic differences I perceived during my informal listening.

How does that fit with your argument?

Well, of course, if I understood correctly your point, the fact that one can successfully identify blind does not mean that every one can. I'm just saying it may be generally more difficult.

All I'm saying is something like (to make it more obvious) : is it really unlikely that someone in an emotionally engaging and vulnerable mental state could perceive more or less consciously things that they can never clearly identify in a (more clinical) blind-test situation? Which would open the door to the hypothesis that one's musical experiment at home under "normal circumstances" is quite different from others.

 

[Blumlein 88]

First conclusion: one may be even more subjected to biases when listening to music than for an other audio signal.

First conclusion: one may be even more subjected to biases when listening to music than for an other audio signal.

Okay let us start with your first conclusion. Why would this cause any problems otherwise in testing? The tests for emotional signals or non-emotional signals could be done the same. It wouldn't prevent a blind test. I mean somehow people get all emotional and hear wire and cable make their enjoyment different. Night and day different, but when tested the emotional aspect means suddenly they cannot tell a difference?

 

[polocolo]

First conclusion: one may be even more subjected to biases when listening to music than for an other audio signal.

Okay let us start with your first conclusion. Why would this cause any problems otherwise in testing? The tests for emotional signals or non-emotional signals could be done the same. It wouldn't prevent a blind test. I mean somehow people get all emotional and hear wire and cable make their enjoyment different. Night and day different, but when tested the emotional aspect means suddenly they cannot tell a difference?

Ok, so, maybe I should say it again (for the fifth time?): I'm not talking about things for which we have no evidence of sounding different (like cables) but of speakers.
Pages 690 to 713 or so were about blind-testing speakers vs sighted listening. So I was still riding that wave.

And then again: yes. Psychology tends to show that emotionally engaging situations tend to change your perceptual skills. So why not here?

I mean it's a pretty well known fact for instance that stress induces "narrow" or "tunnel" vision – selecting the useful information from the field of experience.

 

[JustJones]

Isn't this a test for hearing ( does A sound different than B) not how did A make you feel? Did B elicit a different emotional response than A?

 

[polocolo]

Isn't this a test for hearing ( does A sound different than B) not how did A make you feel? Did B elicit a different emotional response than A?

yeah, that's why it was addressing the conversation about sighted listening vs blinded listening when it comes to choosing speakers.

 

[JustJones]

yeah, that's why it was addressing the conversation about sighted listening vs blinded listening when it comes to choosing speakers.

OK, we're not discussing anything stressful.

 

[polocolo]

OK, we're not discussing anything stressful.

well, 1) I think your answer shows some ignorance about what perceptual psychologists call "stress" (which is not "omg I'm so stressed") and 2) this was just an exemple of one kind of emotion that alters perception.

concentration also alters perception.

context alters perception.

why do we discard the idea that, creating a context to eliminate bias, we're not creating others?

 

[NTK]

I'm sorry that does not address the core of my (maybe extremely weak and stupid) hypothesis.
if this hypothesis is so lame, it should be easy to attack the reasoning at its core, I feel.

I now see that you are talking about speaker testing. When you say ABX, it is a test for audible differences (to identify whether X is A or B), not preferences. For loudspeaker, I doubt you can find speakers from different manufacturers and/or models that sound audibly identical so ABX is not the correct test to use.

 

[kharan]

But are these effects stable? That's a critical question. I think we've all had the experience of being impressed by something, but then fatigued or unimpressed over longer periods. When your cool-looking gizmo is a well-known part of your living room, does it affect your listening the same way?

I say they are not stable. I can't prove it, of course. But that's why controlling for non-auditory biases makes sense to me, even as a matter of preference. Your sighted impression is simply the auditory impression plus an unknown and probably unstable variable.

That's a good question. For my own decisions, "it measures badly" is automatically disqualifying, but I can see a situation where the initial positive impression builds up overtime as you continuously associate good feelings with a piece of gear. Especially if it's a piece of gear that is mostly audibly transparent like an amplifier with okay but not great performance.

 

[polocolo]

I now see that you are talking about speaker testing. When you say ABX, it is a test for audible differences (to identify whether X is A or B), not preferences. For loudspeaker, I doubt you can find speakers from different manufacturers and/or models that sound audibly identical so ABX is not the correct test to use.

Oh ok. Maybe I misunderstood some of the stakes of the conversation then.

What I got from the whole debate (like pages 690 to 710 or so) was: you can't trust sighted listening for speakers you should trust only blinded listening. And I was wondering if we weren't adding a whole other set of biases because of the influence of context on perception.

 

[Doodski]

Psychology tends to show that emotionally engaging situations tend to change your perceptual skills. So why not here?

yeah, that's why it was addressing the conversation about sighted listening vs blinded listening when it comes to choosing speakers.

I have done many listening tests where the customer is blind to what the amp is or in some cases which speaker is selected and then using a switchbox to instantly switch over and then most of them could not tell what model they where listening to until it was revealed to them. Before the tests I advised them to listen to the instruments and don't concern themselves with the emotion of the music or the mood. Solely concentrate on finding the speaker that to their ears has the best representation of the instruments. Many people where hesitant to engage the test but upon urging them to do this for the sake of finding the best choice for them they took the tests. After a test or two was completed in short order all of them wanted to test more gear and see what is up. So people can and do disconnect from the beat and the mood/emotion of the music and listen analytically to the instruments. It's not difficult to do when one is presented with gear that is all setup on switchboxes and requires little to no time for setup.

 

[JustJones]

I have done many listening tests where the customer is blind to what the amp is or in some cases which speaker is selected and then using a switchbox to instantly switch over and then most of them could not tell what model they where listening to until it was revealed to them. Before the tests I advised them to listen to the instruments and don't concern themselves with the emotion of the music or the mood. Solely concentrate on finding the speaker that to their ears has the best representation of the instruments. Many people where hesitant to engage the test but upon urging them to do this for the sake of finding the best choice for them they took the tests. After a test or two was completed in short order all of them wanted to test more gear and see what is up. So people can and do disconnect from the beat and the mood/emotion of the music and listen analytically to the instruments. It's not difficult to do when one is presented with gear that is all setup on switchboxes and requires little to no time for setup.

Back in the day when shops were around I was on the customer end of tests like that. Some shops/salesmen were really nice others tried to push stuff.

 

[polocolo]

I have done many listening tests where the customer is blind to what the amp is or in some cases which speaker is selected and then using a switchbox to instantly switch over and then most of them could not tell what model they where listening to until it was revealed to them. Before the tests I advised them to listen to the instruments and don't concern themselves with the emotion of the music or the mood. Solely concentrate on finding the speaker that to their ears has the best representation of the instruments. Many people where hesitant to engage the test but upon urging them to do this for the sake of finding the best choice for them they took the tests. After a test or two was completed in short order all of them wanted to test more gear and see what is up. So people can and do disconnect from the beat and the mood/emotion of the music and listen analytically to the instruments. It's not difficult to do when one is presented with gear that is all setup on switchboxes and requires little to no time for setup.

Interesting, thank you.
I don't doubt the ability of people to change the way they perceive according to context – on the contrary, that's the basis of my questioning!
You can consciously change because of context (at will), or you can do it subconsciously as well.

 

[polocolo]

Well, it's not for lack of sincere efforts but I can tell I more or less completely failed to draw some interest to my questions!
I'm gonna go for tonight and will check tomorrow if there are other answers.

I guess I'll try one last way of saying it:

if we are biased all the time, isn't there a risk that, constructing a context with the goal in mind to eradicate as much bias as possible, we might add some other biases in the context? and if so, how do we know that bias state is more relevant than the one we trying to avoid?

 

[Doodski]

Interesting, thank you.
I don't doubt the ability of people to change the way they perceive according to context – on the contrary, that's the basis of my questioning!
You can consciously change because of context (at will), or you can do it subconsciously as well.

The factor that made it difficult on the initial suggestions to test this way was was that the people wanted to be in control. I asked them to be in control in their choices but to let me setup and organize the tests for them. They agreed to that and the objections where overcome.

 

[Doodski]

if we are biased all the time, isn't there a risk that, constructing a context with the goal in mind to eradicate as much bias as possible, we might add some other biases in the context? and if so, how do we know that bias state is more relevant than the one we trying to avoid?

If one is spending money for gear they will be concerned about getting it right the first time and not having hassles and stuff like that. When offered multiple choices which are all good choices they come around and form bias because they need to for those choices to be made according to their ears preferences. Bias is a need.

 

[Doodski]

Back in the day when shops were around I was on the customer end of tests like that. Some shops/salesmen were really nice others tried to push stuff.

I did active demos to all that wanted them whether they where buying or not. It was my way of doing business and I was very successful at selling speakers. I did not care if it was a dirty clothed smelly individual or a suit wearing analytical difficult person. They all worked hard for their money and all of them wanted the same thing. They wanted the experience and to have nice gear. Often times tradesmen would come in mid day as they where driving by or after a hard day at work and they spent large because they could afford it with a steady job and decent pay. I loved doing demos and walking people.... I had many return customers from that methodology.

 

[MAB]

ahaha, the smugness.

You know, when talking about something, there is always at least two expertises at stake. The expertise on the topic and the expertise of constructing a rational proposition self aware of its own limits and how the probabilities that ground it.

if you have just the former, you're petty much certain to talk foolishly.
if you have just the latter, you can avoid most danger.

Thanks. The irony made my day.
I really encourage you to study some literature on these subjects.

 

[antcollinet]

you can't just affirm this. Scientists never stop questioning their protocols/conclusions. Trying to fall the flaws in methodologies is at the core of a scientific practice.


Why the need to act smug?
have I not manifested goodwill in my answers?
I'm not trying to be clever, I'm questioning the foundation of one belief.
You make fun of me but you don't address the proposition.

You keep accusing others of being smug - but your posts come across, to me, as the smuggest I've ever read here.

 

[MattHooper]

All I'm saying is something like (to make it more obvious) : is it really unlikely that someone in an emotionally engaging and vulnerable mental state could perceive more or less consciously things that they can never clearly identify in a (more clinical) blind-test situation? Which would open the door to the hypothesis that one's musical experiment at home under "normal circumstances" is quite different from others.

Nothing wrong with raising such questions.

However…

That is sounding very suspiciously like the type of claims many subjectivist audiophiles make on behalf of many false or dubious claims, to reject the relevance of blind testing.

It seems to share the same quality of vagueness and as far as I can tell so far, it seems unfalsifiable, similar to the way subjectivist audiophiles provide unfalsifiable claims based on their epistemology.

You mentioned you were talking about audible differences, such as loudspeakers.

Let’s say someone in their normal emotional listening to music believes they have identified certain characteristics in a loudspeaker. How would this be falsifiable?

Well, it seems we could appeal to objective measurements. if the person believes that the speakers have a distinct emphasis in frequency response in a certain range, you can check the measurements to see if their impression is accurate… whatever their emotional state when sighted listening.

Or I suppose there is an outcome that in principle could lend evidential weight to your hypothesis.

Let’s say someone in their casual sighted listening to a loudspeaker claims to hear an obvious rise in the frequency response between 2k and 4K.

You could do a blind test to see if they really are identifying those Sonic characteristics.
But what if in blind testing - at one point they are listening to this very same speaker, but don’t know it’s identity - they do not recognize any such frequency emphasis?

Again, measurements to the rescue: let’s stipulate the speaker measurements actually do establish that two to 4K emphasis in the frequency response that the person claimed to hear during sighted listening.

So, in this case, you would have the measurements suggesting this persons sighted impressions are more accurate than the blind listening impressions.

So it could suggest less discrimination under the blind listening test conditions.

Would you agree this is one way that in principle he could lend evidence to your hypothesis?

The thing is, I’m not aware of any blind vs sighted tests that have come back with such results.