The Ultimate Listening Test: Is It The Filtering or the Bandwidth?
In December 1996, I performed a listening test, with the collaboration of members of the Pro Audio maillist. The idea was to develop a test that would eliminate all variables except bandwidth, with a constant sample rate, filter design, DAC, and constant jitter. The question we wanted to answer was this: Does high sample rate audio sound better because of increased bandwidth, or because of less-intrusive filtering?
The test we devised was to create a filtering program that takes a 96 kHz recording, and compare the effect on it of two different bandwidth filters. The volunteer design team consisted of Ernst Parth (filter code), Matthew Xavier Mora (shell), Rusty Scott (filter design), and Bob Katz (coordinator and beta tester). We created a digital audio filtering program with two impeccably-designed filters which are mathematically identical, except that one cuts off at 20 kHz and the other at 40 kHz. The filters are double-precision dithered, FIR linear phase, 255-tap, with > 110 dB stopband attenuation, and < 0.1 dB passband ripple.
After the filter program was designed, I took a 96 kHz SR orchestral recording, filtered it and brought it back into a Sonic Solutions DAW for the comparison. I expected to hear radical differences between the 20 kHz and 40 kHz filtered material, but I could not! Next, I compared the 20 kHz filtered against "no filter" (of course, the material has already passed through two steep 48 kHz filters in the A/D/A). Again, I could not hear a difference! The intention was to listen double-blind, but even sighted, 10 additional listeners who took part in the tests (one at a time) heard no difference between the 20 kHz digital filter and no filter. And if no one can hear a difference sighted, why proceed to a blind test?
I tried different types of musical material, including a close-miked recording I made of castanets (which have considerable ultrasonic information), but there was still no audible difference. I then created a test which put 20 kHz filtered material into one channel of my Stax electrostatic headphones, and the time-aligned wide-bandwidth material into the other channel. I was not able to detect any image shift, image widening or narrowing—there was always a perfect mono center at all frequencies in the headphones! This must be a pretty darn good filter!
As a last resort, I went back to the list and asked maillist participant Robert Bristow Johnson to design a special "dirty" filter with 0.5 dB ripple in the passband. Finally, with the dirty filter, I was able to hear a difference...this dirty filter added a boxy quality that resembles the sound of some of the cheaper 44.1k CD players we all know.
This 1996 test seems to show that a "perfect 20 kHz filter" can be designed, but at what cost? Also note that as this test was conducted in the context of a 96 kHz sample rate, the artifacts of two other 48 kHz steep filters already in use may have obscured or masked the effect of the filter under test. Since I conducted my test, several others have tried this filtering program, and most have reached the same conclusion: the filter is inaudible. One maillist participant, Eelco Grimm, a Netherlands-based writer and engineer, performed the test and reported that there were no audible differences using the Sonic Solutions system, yet he and a colleague were able to pick out differences between filtered and non-filtered blind using an Augan workstation. He did not compare the sound of the 20 kHz versus 40 kHz filters, so we are not sure if he's hearing the filter or the bandwidth, but I believe he was hearing the filter, which must not be ideally-designed. I believe the reason he did not hear the differences on the Sonic system is perhaps its jitter was high enough to mask the other differences, which must be very subtle indeed!
Regardless of whether Eelco's group did reliably hear the bandwidth differences, it should be clear by now that the so-called "dramatic" differences people hear between sample rate systems are not likely to be due to bandwidth, but probably to the filter design itself. Ironically, it was necessary to make a high sample rate recording in order to prove that high sample rates may not be necessary.