Paper called "What Do Blind Evaluations Reveal?" about selection of submissions for journals etc.

[MaxwellsEq]

This is an interesting paper. Although actually off-topic for audio, it's on-topic for how our biases affect the decisions we make.

Haruka Uchida of the University of Chicago compared academic paper submissions, both blinded and non-blinded. Non-blinded there was a positive selection bias for submissions by more senior male contributors. Blinded, the scoring differences between senior and student contributions was less. However, interestingly, good submissions still came to the top, so blinding did NOT reduce the quality of the best papers which were chosen. This refutes the argument by those who consider that blinded testing hides higher quality audio components; allegedly bringing everything to a mediocre common quality.

https://haruka-uchida.github.io/websitefiles/uchida_haruka_blinding.pdf

 

[DVDdoug]

Well, you can't easily reject a paper from a known & respected professor or scientist.

This refutes the argument by those who consider that blinded testing hides higher quality audio components; allegedly bringing everything to a mediocre common quality.

"Audiophiles'" often make all kinds of excuses for "failing" blind listening tests.

Personally, I can't accept that making a test blind can ever make it less reliable. And we know that sighted listening tests are often unreliable. That's been demonstrated when an A-B switch doesn't do anything, or when "night and day" differences magically go-away when the test is done blind.

Not everything has to be done blind. Some things are truly obvious... If you are hearing a buzz from your speakers or getting no sound out of the left channel I'm not going to suggest a blind test. Amir does sighted testing as part of his routine reviews but he backs-up his impressions & perceptions with measurements.

 

[JanL]

This refutes the argument by those who consider that blinded testing hides higher quality audio components; allegedly bringing everything to a mediocre common quality.

https://haruka-uchida.github.io/websitefiles/uchida_haruka_blinding.pdf

It does no such thing. The test was for classifying papers, not testing the sense of hearing.
That being said, someone should devise a test to actually test the sensitivity to absolute audio quality sighted versus unsighted. If people track objectively superior quality better when sighted than unsighted then there would be some confirmation to the argument that unsighted guessing blocks something in the audio perception.

 

[MaxwellsEq]

It does no such thing. The test was for classifying papers, not testing the sense of hearing.
That being said, someone should devise a test to actually test the sensitivity to absolute audio quality sighted versus unsighted. If people track objectively superior quality better when sighted than unsighted then there would be some confirmation to the argument that unsighted guessing blocks something in the audio perception.

I recommend that you read Dr Floyd Toole's book. This is one of the things that his research demonstrated.

 

[JanL]

I recommend that you read Dr Floyd Toole's book. This is one of the things that his research demonstrated.

I am familiar with dr. Toole's findings. He showed that blind impressions are superior to impressions where the subjects know what product or brand they are listening to. My suggestion is a little different.
Take the sighted tests where the reproduction quality objectively tracks preferences and run them again unsighted. See if the preferences stay the same or become more random. That would be a way to verify the claims in question.

 

[MaxwellsEq]

I am familiar with dr. Toole's findings. He showed that blind impressions are superior to impressions where the subjects know what product or brand they are listening to. My suggestion is a little different.
Take the sighted tests where the reproduction quality objectively tracks preferences and run them again unsighted. See if the preferences stay the same or become more random. That would be a way to verify the claims in question.

Presumably you have not read the book.

 

[Head_Unit]

Presumably you have not read the book.

Book Shmook, I can see what sounds better! It's obvious!

Blind evaluations are time consuming and difficult, I guess that's why they seldom happen.

 

[sqrl]

It does no such thing. The test was for classifying papers, not testing the sense of hearing.
That being said, someone should devise a test to actually test the sensitivity to absolute audio quality sighted versus unsighted. If people track objectively superior quality better when sighted than unsighted then there would be some confirmation to the argument that unsighted guessing blocks something in the audio perception.

That would be audiovisual perception by definition.