• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Actual Double Blind Studies

ahofer

Master Contributor
Forum Donor
Joined
Jun 3, 2019
Messages
5,154
Likes
9,379
Location
New York City

Sal1950

Grand Contributor
The Chicago Crusher
Forum Donor
Joined
Mar 1, 2016
Messages
14,537
Likes
17,430
Location
Central Fl
Good double blind tests are very difficult just to get arranged and organized.

Subjective types don't want to be involved for fear of failing to hear the things they constantly claim to hear.

Objective types don't want to go thru all the work because TBH, so much of audio is a solved problem and many already
know what the outcome will be and feel it will be mostly a waste of time. Only in cases where determining the audible levels of things like say distortion, jitter, SINAD can you get much enthusiasm pumped up...

It's a strange, strange world we live in Master Jack. ;)
 

ahofer

Master Contributor
Forum Donor
Joined
Jun 3, 2019
Messages
5,154
Likes
9,379
Location
New York City
Good double blind tests are very difficult just to get arranged and organized.

Subjective types don't want to be involved for fear of failing to hear the things they constantly claim to hear.

Objective types don't want to go thru all the work because TBH, so much of audio is a solved problem and many already
know what the outcome will be and feel it will be mostly a waste of time. Only in cases where determining the audible levels of things like say distortion, jitter, SINAD can you get much enthusiasm pumped up...

It's a strange, strange world we live in Master Jack. ;)
Would you go to the trouble of reproducing experiments in what object falls faster in a vacuum? I mean, ok, the feather and the weight accelerated at the same speed, but whAt abOut a FRisbeE?

 

Sal1950

Grand Contributor
The Chicago Crusher
Forum Donor
Joined
Mar 1, 2016
Messages
14,537
Likes
17,430
Location
Central Fl
Would you go to the trouble of reproducing experiments in what object falls faster in a vacuum? I mean, ok, the feather and the weight accelerated at the same speed, but whAt abOut a FRisbeE?
Would I?
Nope, the science tells me the results would be the same. ;)
 

Sal1950

Grand Contributor
The Chicago Crusher
Forum Donor
Joined
Mar 1, 2016
Messages
14,537
Likes
17,430
Location
Central Fl
Why don't we just say it's lumpy and bumpy?
 

Duke

Major Contributor
Audio Company
Forum Donor
Joined
Apr 22, 2016
Messages
1,668
Likes
4,105
Location
Princeton, Texas
Good double blind tests are very difficult just to get arranged and organized.

Especially for loudspeakers!

"Double blind" means that even those administering the test do not know which speaker is being auditioned. Short of having a speaker shuffler that required zero human intervention or monitoring once the test begins, like what Harmon spent the big bucks to create, I don't see how a controlled double-blind test could be done.

Fortunately for loudspeaker testing, single-blind is sufficient as long as those administering the test have no direct or otherwise potentially bias-inducing interaction with the listeners during the test. This is not like drug testing where you don't want the nurse who administers the pills to know which ones are placebos.

I was once the behind-the-curtain "speaker shuffler" for a controlled single-blind test conducted by Earl Geddes, and would have had a really hard time doing it blind-folded, but even then I could have differentiated the speakers by weight.
 

Sal1950

Grand Contributor
The Chicago Crusher
Forum Donor
Joined
Mar 1, 2016
Messages
14,537
Likes
17,430
Location
Central Fl
Fortunately for loudspeaker testing, single-blind is sufficient as long as those administering the test have no direct or otherwise potentially bias-inducing interaction with the listeners during the test.
Very true, with speakers the differences are mostly wide enough that only preference needs to be determined among a large listening group. The "better" speakers can mostly be determined by measurement which is easily repeatable.
 

ahofer

Master Contributor
Forum Donor
Joined
Jun 3, 2019
Messages
5,154
Likes
9,379
Location
New York City
Would you go to the trouble of reproducing experiments in what object falls faster in a vacuum? I mean, ok, the feather and the weight accelerated at the same speed, but whAt abOut a FRisbeE?

Well, it looks like somebody decided to test astrology:

 

cavedriver

Addicted to Fun and Learning
Joined
Jul 10, 2020
Messages
745
Likes
844
Location
Maryland, USA
On page 78 is a quite famous test between some very well known high-end amplifiers published in Stereo Review:
I worked in a store that sold Levinson amps at the time, to be honest I wasn't surprised.
This test reminds me that I felt there was a very discernible difference between my Hypex 502MP and my vintage Onkyo tx-sr805 receiver. Both have plenty of power and I was using them to drive a variety of relatively small and easy to drive speakers, but it was all completely sighted so now I wonder what I was hearing. I would be interested in seeing ABX tests of class D vs class AB amps to see if some of the behaviors of the AB equipment vs the D are typically discernible. The two things that seemed remarkable to me would have been that the bass seemed much cleaner and tighter with the Hypex, and the highs were a bit more sterile and perhaps a little louder. Given that we're talking about a 10+ year old mid-grade receiver, there's no telling if the amp's frequency accuracy has measurable fall-off at the low and high ends of the range.

Also, regarding running double blind tests, I suppose having the two speakers sitting next to each other behind the screen and just switching the sources is not good enough since the listeners could rapidly discern, "oh, that's the one on the left". I suppose a pair of automated turn tables for the speakers to rest on could make the swap-outs fast. Something I've been wanting to set up at a trade show like CAF. Maybe this coming year.
 
Last edited:

SIY

Grand Contributor
Technical Expert
Joined
Apr 6, 2018
Messages
10,715
Likes
25,802
Location
Alfred, NY
This test reminds me that I felt there was a very discernible difference between my Hypex 502MP and my vintage Onkyo tx-sr805 receiver. Both have plenty of power and I was using them to drive a variety of relatively small and easy to drive speakers, but it was all completely sighted so now I wonder what I was hearing. I would be interested in seeing ABX tests of class D vs class AB amps to see if some of the behaviors of the AB equipment vs the D are typically discernible. The two things that seemed remarkable to me would have been that the bass seemed much cleaner and tighter with the Hypex, and the highs were a bit more sterile and perhaps a little louder. Given that we're talking about a 10+ year old mid-grade receiver, there's no telling if the amp's frequency accuracy has measurable fall-off at the low and high ends of the range.

Also, regarding running double blind tests, I suppose having the two speakers sitting next to each other behind the screen and just switching the sources is not good enough since the listeners could rapidly discern, "oh, that's the one on the left". I suppose a pair of automated turn tables for the speakers to rest on could make the swap-outs fast. Something I've been wanting to set up at a trade show like CAF. Maybe this coming year.
These days, it's a lot easier. Use a good recording interface or sound card and record the electrical signal at the speaker terminals at about the same sound level (precise matching isn't needed, just get it close). You want to use a couple resistors (say 10k and 1k) to divide down the voltage for protection of the sound card. Then use Deltawave software to look at the differences and see if they're significant. You can also use the software to run a level-matched ABX.
 

ahofer

Master Contributor
Forum Donor
Joined
Jun 3, 2019
Messages
5,154
Likes
9,379
Location
New York City
I suppose a pair of automated turn tables for the speakers to rest on could make the swap-outs fast. Something I've been wanting to set up at a trade show like CAF. Maybe this coming year.
Harman has that in their facility. Expensive.

200906_harman.jpg
 

12Many

Active Member
Joined
Jan 22, 2022
Messages
125
Likes
75
I have always been curious about 'training' for these tests.

I saw a visual test that used an aerial pic of a full Rose Bowl, and instantaneous ABX testing was used to compare two pics and seeing if people could notice the difference between them. The subjects couldn't do it.

Then, the tester pointed out the one difference: one person in a different color shirt(out of, what, 100,000 people?) and after that the subjects had a 100% success rate. They had been taught to spot the difference.

Obviously not the same for audio, but are there listening tests where people were taught in advance the differences and looking to see how smaller and smaller 'learned' differences could still be identified blind?

For electronics, with good headphones, perhaps there is a way for people to hear in more detail what we measure. Not claiming any knowledge of this, just chatting.

No idea if this has been done and to what degree. I have seen results looking at difference sonic thresholds, but I do not know if there is a further training effect that improves performance.
Good post. The biggest variable is the person listening, their experience, and hearing quality.
 

cavedriver

Addicted to Fun and Learning
Joined
Jul 10, 2020
Messages
745
Likes
844
Location
Maryland, USA
Harman has that in their facility. Expensive.
I think I could build this as simply two large circles so we could A/B two or three pairs of speakers at a time. Plenty of large lazy susan bearings with 1,000 lb weight capacities out there. If it were to be turned manually could build a pair in one day. Automated would take longer. Biggest hassle would be building damping into the platform to ensure there are no vibrations being perpetuated by the platform. Perhaps dense foam on top surface and a CLD layer between two sheets of 3/4" plywood, and then edge protection so you can roll the monstrosity in and out of trade shows. If the bearing is contributing to the vibration could use wood shop portable table feet that latch down to the baseplate during playback.
 
  • Like
Reactions: MAB

ahofer

Master Contributor
Forum Donor
Joined
Jun 3, 2019
Messages
5,154
Likes
9,379
Location
New York City
I think I could build this as simply two large circles so we could A/B two or three pairs of speakers at a time. Plenty of large lazy susan bearings with 1,000 lb weight capacities out there. If it were to be turned manually could build a pair in one day. Automated would take longer. Biggest hassle would be building damping into the platform to ensure there are no vibrations being perpetuated by the platform. Perhaps dense foam on top surface and a CLD layer between two sheets of 3/4" plywood, and then edge protection so you can roll the monstrosity in and out of trade shows. If the bearing is contributing to the vibration could use wood shop portable table feet that latch down to the baseplate during playback.
Sounds pretty formidable, but the *most* formidable obstacle is lack of a core audiophile constituency that actually wants this sort of thing to happen. They don't want windows opened, there are too many houses of cards in the room.
 

MAB

Major Contributor
Joined
Nov 15, 2021
Messages
2,228
Likes
5,063
Location
Portland, OR, USA
I think I could build this as simply two large circles so we could A/B two or three pairs of speakers at a time. Plenty of large lazy susan bearings with 1,000 lb weight capacities out there. If it were to be turned manually could build a pair in one day. Automated would take longer. Biggest hassle would be building damping into the platform to ensure there are no vibrations being perpetuated by the platform. Perhaps dense foam on top surface and a CLD layer between two sheets of 3/4" plywood, and then edge protection so you can roll the monstrosity in and out of trade shows. If the bearing is contributing to the vibration could use wood shop portable table feet that latch down to the baseplate during playback.
Like this!
 

cavedriver

Addicted to Fun and Learning
Joined
Jul 10, 2020
Messages
745
Likes
844
Location
Maryland, USA
Sounds pretty formidable, but the *most* formidable obstacle is lack of a core audiophile constituency that actually wants this sort of thing to happen. They don't want windows opened, there are too many houses of cards in the room.
Yeah, I could build it, but could I get my local audio club to help sponsor me to set it up in a room at CAF? Maybe, maybe not. I think it would be a cool "attraction" at an event like that. There's always those rooms where you wonder what they are selling and why they even have the room in the first place. A room with a sign out front reading, "TEST YOUR AUDIOPHILE CHOPS HERE!" would be fun, and I wouldn't have to use currently hot new speaker designs, just things that are well known or have interesting behaviors. Could give anyone that gets a certain score a prize or something. I think a lot of people would be interested in the challenge. We think we know what we know and some are curious and brave enough to test it.
 
  • Like
Reactions: MAB

ahofer

Master Contributor
Forum Donor
Joined
Jun 3, 2019
Messages
5,154
Likes
9,379
Location
New York City
some are curious and brave enough to test it.
some indeed. Obviously it would be much easier with electronics, which many at those shows think they know as well.

My guess is you'd get low participation and a bunch of quibbling with the test conditions. Like this

 
Top Bottom