• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Are DBTs the defacto standard for Audio Science research?

AJ Soundfield

Major Contributor
Joined
Mar 17, 2016
Messages
1,001
Likes
68
Location
Tampa FL
"Current" and/or past.

Members, Yes or No?
If No (DBTs are worthless/stupid/etc), should said member ever be able to cite auditory research results that are derived from DBTs??
Isn't that complete hypocrisy?

cheers,

AJ
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,658
Likes
240,924
Location
Seattle Area
In the strict sense, no, they are not the de-facto standard. Vast amount of psychoacoustics research gets conducted as single blind or for that matter, not blind at all.

Let's say we want to test the detection threshold for ticks. A bunch of university students are brought in as volunteers in a room, and ticks of various intensity played by the researcher and the point where the testers can't tell the difference is the difference anymore becomes the threshold. In that regard, the students know what is being tested, and the people conducting the test clearly are doing the work so not at all double blind.

The data however is considered very much valid because it is more or less bias-controlled. The people being tested have little bias interfering with their ability to answer truthfully. And researchers/proctors likewise have little to gain by gaming the outcome.

Read the Fastl and Zwicker text on psychoacoustics and you see that the above is pretty much the protocol for much of their work. BTW no statistical analysis is done either with group sizes pretty small (as small as four testers).

So the key thing here is be aware of what you are testing and what the sources and power of bias can be. If the latter factors are insignificant, then much simpler protocols suffice. If not, then you better be as formal as you can with double blind, etc.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,658
Likes
240,924
Location
Seattle Area
BTW, audiology hearing tests are also done like above. You sit in a booth, knowing full well you are being tested for hearing the tones. And so does the operator. And no statistical analysis. Yet the outcome is medically accepted as valid.
 
OP
AJ Soundfield

AJ Soundfield

Major Contributor
Joined
Mar 17, 2016
Messages
1,001
Likes
68
Location
Tampa FL
Ok, so to be clear Amir, a person can openly state that they reject DBTs over zero controls "listening" and DBTs are unnecessary to the point of stupidity...then cite auditory perceptual research results from DBTs, to support their beliefs?
 

SoundAndMotion

Active Member
Joined
Mar 23, 2016
Messages
144
Likes
111
Location
Germany
Ok, so to be clear Amir, a person can openly state that they reject DBTs over zero controls "listening" and DBTs are unnecessary to the point of stupidity...then cite auditory perceptual research results from DBTs, to support their beliefs?
Hi AJ, it's been a while. Can you be clear about whether you mean someone who rejects all ABX or DBT tests, or just those that are done by amateurs at home. I've had disagreements with one member here about the value of home-performed tests, but we agree that professionally done tests, as done in AES papers or other science cited here is valid.
Cheers, SAM
 
Last edited:

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,658
Likes
240,924
Location
Seattle Area
Ok, so to be clear Amir, a person can openly state that they reject DBTs over zero controls "listening" and DBTs are unnecessary to the point of stupidity...then cite auditory perceptual research results from DBTs, to support their beliefs?
No, one doesn't translate from this to the other. The keyword we need to use is "bias-controlled" testing, not DBT and certainly not ABX. Those are example methods of bias-controlled testing.

In that regard, any testing that has the potential for bias then is to be disregarded as far as proving a point in this forum.
 

h.g.

Active Member
Joined
Mar 14, 2016
Messages
110
Likes
8
Are DBTs the defacto standard for Audio Science research?
No.

Blinding and its form is a component part of an experiment to test a hypothesis. When the blinding becomes important and detached from the hypothesis being tested we know we are dealing with audiophiles of a subjective or objective flavour and not people involved in conventional research.

Scientific knowledge is predicting the outcome of experiments without having to perform them. So most of audio science research involves using existing knowledge with only a modest part involving experiments. When it comes to sound nearly all those experiments will involve microphone measurements rather than listening because they are lot more efficient and reliable. The tiny fraction that involve listening will often involve double blinding as one of a range of factors that influence sound perception. The neglecting of those other factors that influence perception and banging on about blinding is another big red flag indicating audiophile involvement.
 
OP
AJ Soundfield

AJ Soundfield

Major Contributor
Joined
Mar 17, 2016
Messages
1,001
Likes
68
Location
Tampa FL
Hi AJ, it's been a while. Can you be clear about whether you mean someone who rejects all ABX or DBT tests, or just those that are done by amateurs at home. I've had disagreements with one member here about the value of home-performed tests, but we agree that professionally done tests, as done in AES papers or other science cited here is valid.
Cheers, SAM
Hi Sam, the referenced posts were deleted by Amir in another thread, but yes, a member positing research derived from blind tests as an argument, while maintaining this:
http://www.diyaudio.com/forums/ever...easurements-show-not-show-14.html#post2267178 and saying here they favor anecdotal reports over controlled blind tests.
Yes AES type tests as I usually reference.

cheers,

AJ
 

fas42

Major Contributor
Joined
Mar 21, 2016
Messages
2,818
Likes
191
Location
Australia
No, one doesn't translate from this to the other. The keyword we need to use is "bias-controlled" testing, not DBT and certainly not ABX. Those are example methods of bias-controlled testing.

In that regard, any testing that has the potential for bias then is to be disregarded as far as proving a point in this forum.
Amir, could you elaborate a bit more on what exactly you understand "bias-controlled" as meaning?
 
OP
AJ Soundfield

AJ Soundfield

Major Contributor
Joined
Mar 17, 2016
Messages
1,001
Likes
68
Location
Tampa FL
In that regard, any testing that has the potential for bias then is to be disregarded as far as proving a point in this forum.
Ok and again to be clear, saying one rejects bias controlled tests/DBTs and then positing them to support a belief is hypocritical?
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,658
Likes
240,924
Location
Seattle Area
Amir, could you elaborate a bit more on what exactly you understand "bias-controlled" as meaning?
It means any factor that would change your perception of audio, other than audio. Examples include what something looks like, what it costs, its size, color, brand, material construction, whether the outcome goes against for for our beliefs, etc.

Bias controlled tests examine the risk factor in each of these categories and create a test where the dominant variable is the sound arriving at your ear and nothing else.
 

Mivera

Major Contributor
Joined
Mar 4, 2016
Messages
2,322
Likes
97
Location
West Kelowna
It means any factor that would change your perception of audio, other than audio. Examples include what something looks like, what it costs, its size, color, brand, material construction, whether the outcome goes against for for our beliefs, etc.

Bias controlled tests examine the risk factor in each of these categories and create a test where the dominant variable is the sound arriving at your ear and nothing else.

The only problem I can see is it takes all of those variables you mentioned out of the picture, which are variables that are present in the real world with 99.999% of people who listen to high end audio systems.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,658
Likes
240,924
Location
Seattle Area
The only problem I can see is it takes all of those variables you mentioned out of the picture, which are variables that are present in the real world with 99.999% of people who listen to high end audio systems.
And that is why we make poor/wrong decisions in our everyday evaluation.

What would happen if I closed my eyes. Should the fidelity go down because I can't see the prettiness of the cabinet? What if the newness of the shiny cabinet wears out. Is it OK to hear worse quality then?
 

fas42

Major Contributor
Joined
Mar 21, 2016
Messages
2,818
Likes
191
Location
Australia
Would you include in that, making a change where you expect an perceived improvement to occur? Over on diyAudio this is the standard put down of someone's findings.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,658
Likes
240,924
Location
Seattle Area
Would you include in that, making a change where you expect an perceived improvement to occur? Over on diyAudio this is the standard put down of someone's findings.
Whether you expect or not expect a change, the outcome can be corrupt if you have additional senses involved besides your ear. Lack of expectation does not fix corruption by bias.
 

fas42

Major Contributor
Joined
Mar 21, 2016
Messages
2,818
Likes
191
Location
Australia
I'm talking more in the sense of tweaking a system, no change occurs in an area where the other senses might pick up something - the only variation would be a possible auditory one.
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,764
Likes
37,617
Also not that many use abx even if blind. Many are an up down protocol. Some parameter is varied down by an amount then when the testee misses it goes back up once again decreasing until missed. At some point the testee is missing as often as not and that is considered the threshold.

I have tried to interest people informally in such testing. The problem is you have to really hear a difference before you can decrease it to threshold.
 

Mivera

Major Contributor
Joined
Mar 4, 2016
Messages
2,322
Likes
97
Location
West Kelowna
And that is why we make poor/wrong decisions in our everyday evaluation.

What would happen if I closed my eyes. Should the fidelity go down because I can't see the prettiness of the cabinet? What if the newness of the shiny cabinet wears out. Is it OK to hear worse quality then?

For pretty much everyone, they don't care. Even the higher form of objective folks on this forum. Look how everyone turned their nose up at the DIY Mirand DAC just because of the packaging. In this example where I demoed this system to my friend with the Devialet 400 system, had it not been hiding behind a piece of plywood, it probably would have sounded like a cheap Hypex plate amp. But since he didn't see it, it was better than his Devialet 400.

IMG_3721.JPG
 
Top Bottom