Are DBTs the defacto standard for Audio Science research?

[AJ Soundfield]

"Current" and/or past.

Members, Yes or No?
If No (DBTs are worthless/stupid/etc), should said member ever be able to cite auditory research results that are derived from DBTs??
Isn't that complete hypocrisy?

cheers,

AJ

 

[amirm]

In the strict sense, no, they are not the de-facto standard. Vast amount of psychoacoustics research gets conducted as single blind or for that matter, not blind at all.

Let's say we want to test the detection threshold for ticks. A bunch of university students are brought in as volunteers in a room, and ticks of various intensity played by the researcher and the point where the testers can't tell the difference is the difference anymore becomes the threshold. In that regard, the students know what is being tested, and the people conducting the test clearly are doing the work so not at all double blind.

The data however is considered very much valid because it is more or less bias-controlled. The people being tested have little bias interfering with their ability to answer truthfully. And researchers/proctors likewise have little to gain by gaming the outcome.

Read the Fastl and Zwicker text on psychoacoustics and you see that the above is pretty much the protocol for much of their work. BTW no statistical analysis is done either with group sizes pretty small (as small as four testers).

So the key thing here is be aware of what you are testing and what the sources and power of bias can be. If the latter factors are insignificant, then much simpler protocols suffice. If not, then you better be as formal as you can with double blind, etc.

 

[amirm]

BTW, audiology hearing tests are also done like above. You sit in a booth, knowing full well you are being tested for hearing the tones. And so does the operator. And no statistical analysis. Yet the outcome is medically accepted as valid.

 

[AJ Soundfield]

Ok, so to be clear Amir, a person can openly state that they reject DBTs over zero controls "listening" and DBTs are unnecessary to the point of stupidity...then cite auditory perceptual research results from DBTs, to support their beliefs?

 

[SoundAndMotion]

Ok, so to be clear Amir, a person can openly state that they reject DBTs over zero controls "listening" and DBTs are unnecessary to the point of stupidity...then cite auditory perceptual research results from DBTs, to support their beliefs?

Hi AJ, it's been a while. Can you be clear about whether you mean someone who rejects all ABX or DBT tests, or just those that are done by amateurs at home. I've had disagreements with one member here about the value of home-performed tests, but we agree that professionally done tests, as done in AES papers or other science cited here is valid.
Cheers, SAM

 

[amirm]

Ok, so to be clear Amir, a person can openly state that they reject DBTs over zero controls "listening" and DBTs are unnecessary to the point of stupidity...then cite auditory perceptual research results from DBTs, to support their beliefs?

No, one doesn't translate from this to the other. The keyword we need to use is "bias-controlled" testing, not DBT and certainly not ABX. Those are example methods of bias-controlled testing.

In that regard, any testing that has the potential for bias then is to be disregarded as far as proving a point in this forum.

 

[h.g.]

Are DBTs the defacto standard for Audio Science research?

No.

Blinding and its form is a component part of an experiment to test a hypothesis. When the blinding becomes important and detached from the hypothesis being tested we know we are dealing with audiophiles of a subjective or objective flavour and not people involved in conventional research.

Scientific knowledge is predicting the outcome of experiments without having to perform them. So most of audio science research involves using existing knowledge with only a modest part involving experiments. When it comes to sound nearly all those experiments will involve microphone measurements rather than listening because they are lot more efficient and reliable. The tiny fraction that involve listening will often involve double blinding as one of a range of factors that influence sound perception. The neglecting of those other factors that influence perception and banging on about blinding is another big red flag indicating audiophile involvement.

 

[AJ Soundfield]

Hi AJ, it's been a while. Can you be clear about whether you mean someone who rejects all ABX or DBT tests, or just those that are done by amateurs at home. I've had disagreements with one member here about the value of home-performed tests, but we agree that professionally done tests, as done in AES papers or other science cited here is valid.
Cheers, SAM

Hi Sam, the referenced posts were deleted by Amir in another thread, but yes, a member positing research derived from blind tests as an argument, while maintaining this:
http://www.diyaudio.com/forums/ever...easurements-show-not-show-14.html#post2267178 and saying here they favor anecdotal reports over controlled blind tests.
Yes AES type tests as I usually reference.

cheers,

AJ

 

[fas42]

No, one doesn't translate from this to the other. The keyword we need to use is "bias-controlled" testing, not DBT and certainly not ABX. Those are example methods of bias-controlled testing.

In that regard, any testing that has the potential for bias then is to be disregarded as far as proving a point in this forum.

Amir, could you elaborate a bit more on what exactly you understand "bias-controlled" as meaning?

 

[AJ Soundfield]

In that regard, any testing that has the potential for bias then is to be disregarded as far as proving a point in this forum.

Ok and again to be clear, saying one rejects bias controlled tests/DBTs and then positing them to support a belief is hypocritical?

 

[AJ Soundfield]

So most of audio science research involves using existing knowledge

Existing knowledge based on what? Sighted tests?

 

[amirm]

Amir, could you elaborate a bit more on what exactly you understand "bias-controlled" as meaning?

It means any factor that would change your perception of audio, other than audio. Examples include what something looks like, what it costs, its size, color, brand, material construction, whether the outcome goes against for for our beliefs, etc.

Bias controlled tests examine the risk factor in each of these categories and create a test where the dominant variable is the sound arriving at your ear and nothing else.

 

[Mivera]

It means any factor that would change your perception of audio, other than audio. Examples include what something looks like, what it costs, its size, color, brand, material construction, whether the outcome goes against for for our beliefs, etc.

Bias controlled tests examine the risk factor in each of these categories and create a test where the dominant variable is the sound arriving at your ear and nothing else.

The only problem I can see is it takes all of those variables you mentioned out of the picture, which are variables that are present in the real world with 99.999% of people who listen to high end audio systems.

 

[amirm]

The only problem I can see is it takes all of those variables you mentioned out of the picture, which are variables that are present in the real world with 99.999% of people who listen to high end audio systems.

And that is why we make poor/wrong decisions in our everyday evaluation.

What would happen if I closed my eyes. Should the fidelity go down because I can't see the prettiness of the cabinet? What if the newness of the shiny cabinet wears out. Is it OK to hear worse quality then?

 

[fas42]

Would you include in that, making a change where you expect an perceived improvement to occur? Over on diyAudio this is the standard put down of someone's findings.

 

[amirm]

Would you include in that, making a change where you expect an perceived improvement to occur? Over on diyAudio this is the standard put down of someone's findings.

Whether you expect or not expect a change, the outcome can be corrupt if you have additional senses involved besides your ear. Lack of expectation does not fix corruption by bias.

 

[fas42]

I'm talking more in the sense of tweaking a system, no change occurs in an area where the other senses might pick up something - the only variation would be a possible auditory one.

 

[AJ Soundfield]

http://seanolive.blogspot.com/2009/04/dishonesty-of-sighted-audio-product.html

 

[Blumlein 88]

Also not that many use abx even if blind. Many are an up down protocol. Some parameter is varied down by an amount then when the testee misses it goes back up once again decreasing until missed. At some point the testee is missing as often as not and that is considered the threshold.

I have tried to interest people informally in such testing. The problem is you have to really hear a difference before you can decrease it to threshold.

 

[Mivera]

And that is why we make poor/wrong decisions in our everyday evaluation.

What would happen if I closed my eyes. Should the fidelity go down because I can't see the prettiness of the cabinet? What if the newness of the shiny cabinet wears out. Is it OK to hear worse quality then?

For pretty much everyone, they don't care. Even the higher form of objective folks on this forum. Look how everyone turned their nose up at the DIY Mirand DAC just because of the packaging. In this example where I demoed this system to my friend with the Devialet 400 system, had it not been hiding behind a piece of plywood, it probably would have sounded like a cheap Hypex plate amp. But since he didn't see it, it was better than his Devialet 400.