Double Blind Testing FAQ Development

[audio2design]

This does complicate things when comparing DAC's. It would have to be done with fade out and fade in with a short 'gap' between them ?
The delay between different DACs would give them away.

While there are audio processing and other processing functions in the brain, decision process are much slower than 10msec, so I highly question the need for anything near 10msec switching time. If that is required to detect a difference you can probably already rule the difference out as inaudible.

 

[j_j]

Nope, can't do that.

Sorry, but even for testing wild*ss claims, you need some measure of test sensitivity for when the whining starts.

 

[j_j]

What is critical is the transition between the two, but that can also be a signal-silence-signal transition, which can usually be done much cleaner than an instant switchover.

Silence disrupts loudness memory, although not as bad as a click. Seamless transition is required for maximum sensitivity.

 

[j_j]

Wrong. How can you be familiar with the error when you don't know what it is, or if it even exists. I can train people to detect readily known errors, i.e. the "sound" of MP3, frequency anomalies, etc. I can essentially give them a toolset. However, I don't know always that there is a difference, or even what it is. I can let them listen to the two items, but familiarity implies a difference exists when it may not. That obviously does not work.

You want them to learn how to ABX properly if nothing else. Of course there may not be any real "signature" to the claim, but at least get them settled on how to do the test.

 

[audio2design]

Sorry, but even for testing wild*ss claims, you need some measure of test sensitivity for when the whining starts.

No, really you don't. The wild ass claim is just that. You only need to disprove the claim for the specific condition set, not for every condition imaginable.

 

[audio2design]

Silence disrupts loudness memory, although not as bad as a click. Seamless transition is required for maximum sensitivity.

I don't have any literature that meets your 10msec stated transition speed. Do you have some links?

 

[j_j]

While there are audio processing and other processing functions in the brain, decision process are much slower than 10msec, so I highly question the need for anything near 10msec switching time. If that is required to detect a difference you can probably already rule the difference out as inaudible.

Actually, decision processes are slow, but hearing processes are not. 1 millisecond ITD is disturbingly audible.

 

[audio2design]

You want them to learn how to ABX properly if nothing else. Of course there may not be any real "signature" to the claim, but at least get them settled on how to do the test.

That I can agree with.

 

[j_j]

I don't have any literature that meets your 10msec stated transition speed. Do you have some links?

Look up papers on "short term loudness memory".

 

[j_j]

No, really you don't. The wild ass claim is just that. You only need to disprove the claim for the specific condition set, not for every condition imaginable.

This is where the whining starts, annoyingly. Then when you show that something far, far smaller is trivially audible, that's when the swearing starts. Trust me on this one. After all of that, when you get down to brass tacks, that's when the slanders, libels, and appeal to ignorance come out in full swing.

Really, trust me on this one. Been there.

 

[audio2design]

Actually, decision processes are slow, but hearing processes are not. 1 millisecond ITD is disturbingly audible.

But in this case, it is a decision process, not a hearing process. You will remember that presence or lack of that 1msec ITD change for seconds if not 10's of seconds when doing a comparison between with and without.

 

[audio2design]

Look up papers on "short term loudness memory".

Not the same thing. This would be a forced level difference, which is something I can measure and does not require a blind test. If I am matching levels to 0.1db, then loudness is not valid difference except gross changes. Not to mention, if you jump in loudness over a 10msec period, by definition you are generating frequencies that did not exist in either signal which negates the transition.

 

[audio2design]

This is where the whining starts, annoyingly. Then when you show that something far, far smaller is trivially audible, that's when the swearing starts. Trust me on this one. After all of that, when you get down to brass tacks, that's when the slanders, libels, and appeal to ignorance come out in full swing.

Really, trust me on this one. Been there.

I have too, but I don't waste my time any more on idiots

 

[j_j]

Not the same thing. This would be a forced level difference, which is something I can measure and does not require a blind test. If I am matching levels to 0.1db, then loudness is not valid difference except gross changes. Not to mention, if you jump in loudness over a 10msec period, by definition you are generating frequencies that did not exist in either signal which negates the transition.

Short term loudness is what gets sent to your brain from the auditory nerve. Yes, it's precisely relevant.

 

[audio2design]

Short term loudness is what gets sent to your brain from the auditory nerve. Yes, it's precisely relevant.

Short term loudness with frequency discrimination that gets overall blended to a whole range of sensations, but back to my original point, if there is a difference and I transition fast, then I am generating new frequencies and negating the experiment. I have to maintain integrity of the original signal. I would argue that if one needs a 10msec transition to detect the difference then it is essentially inaudible anyway. That is not the same as 1 1msec or even 10usec ITD, which does not require a fast change to detect.

I went and found a paper I commented on where they did very fast changes, down to 1msec. As you noted, discrimination did not improve under about 10msec, but they were doing gross changes (5db). They did not have an answer when I asked how they ensured the spectrum did not change when the loudness changed that fast. It's a modulation function. The spectrum has to change. Then it becomes, what are you actually detecting?

 

[CMOT]

Folks. You are discussing how much the second signal masks the first signal. And what is the optimal interval to ensure that Signal 1 is well retained in auditory short-term memory, but isn't masked (or interfered with) by the onset of Signal 2. So too brief an interval and one gets auditory masking (aka interference) and too long a signal and one gets auditory memory decay. I need to go back to Moore's book and check, but since 2IFC designs are quite popular in the study of human audition, I am sure there are some established parameters for the length of signals and the interval between them. I will check some other literature as well, but I think this has all been well worked out.

 

[j_j]

Folks. You are discussing how much the second signal masks the first signal. And what is the optimal interval to ensure that Signal 1 is well retained in auditory short-term memory, but isn't masked (or interfered with) by the onset of Signal 2. So too brief an interval and one gets auditory masking (aka interference) and too long a signal and one gets auditory memory decay. I need to go back to Moore's book and check, but since 2IFC designs are quite popular in the study of human audition, I am sure there are some established parameters for the length of signals and the interval between them. I will check some other literature as well, but I think this has all been well worked out.

If you go to loudness differences (in partial loudness, of course) over time, and the ability to distinguish loudness differences inside an ERB, you will find that anything but a smooth, quick transition hides your data. This is not about masking, rather about short-term auditory memory.

 

[j_j]

Short term loudness with frequency discrimination that gets overall blended to a whole range of sensations, but back to my original point, if there is a difference and I transition fast, then I am generating new frequencies and negating the experiment.

This is why one windows during transitions. Goodness.

Remember, level is NOT LOUDNESS.

 

[audio2design]

This is why one windows during transitions. Goodness.

Remember, level is NOT LOUDNESS.

Except when you transition at 5 or 10 milliseconds, window or not, it is impossible to maintain signal integrity. Literally impossible. The act of switching between two sources is inherently a modulation, and you can't window that out. You can perhaps soften it, but then the transition is not really 10 msec. You will also accentuate any level mismatches, and if you are trying to do a true "audible" difference test, accentuate differences that would be masked under any normal conditions. At that point, you are turning music into test signals.

 

[amirm]

I would argue that if one needs a 10msec transition to detect the difference then it is essentially inaudible anyway.

You contend wrong. If there is an audible difference, there is an audible difference and claiming otherwise is a farce. You can ask for qualification from the tester on how significant that difference was and making some inferences, but not otherwise.

Also, remember that the term "inaudible" goes way beyond the borders of a test by one individual. What may be detectable only at 10msec switching by me, may be audible to others in worse situations. So best not to take short cuts and generalize.