I guess my beef is that this test isn't meaningless. While it certainly doesn't "prove" anything it suggests more tests are called for rather than, "nothing to see here."
The attitude from many of the more vocal on ASR is that this topic is "solved" and any exploration is a waste of time. I find that lack of curiosity surprising.
To me the fact that worldwide 10s of thousands of people have an alternate view of this, makes a deeper serious study worthwhile. The results of such a study may very well be that we can not distinguish between the mid level and the high end DACs, however this test while not conclusive, certainly leaves open the possibility that there may be sonic differences between the two devices under test.
I still have a MiniDSP SHD (DAC/Processor) to do another session.
Here is a list (thus far) of changes to the methodology under consideration for another blind test:
- Write a program to administer the blind test and record results, reducing listener pressure
- Put the DAC3 in optimal mode - use the default high gain settings instead of the 4 volt
- Use the DAC3 volume control instead of the Roon DSP
- Improve the test result presentation since the original method caused confusion
- Take breaks during the test, to reset your ears and fight fatigue and anxiety
- Perform additional test runs over multiple days
I think the level matching, blinding, automated fast switching all worked well.
The methodology is closer to the Harman model where listeners where blinded but in control of the source material and switching to determine their votes.
IMO, this better accommodates the listener than a strict ABX test.
- Rich