Archimago has a new DAC listening test

[DSJR]

When my ears were only twenty one years old or so and I was doing 'turntable comparisons' using the same stylus each time swapped from one deck to another, any perceived differences I felt I heard disappeared after two or three goes back and forth and after half an hour, I had to go and do something else for my ears and faculties to recover...

Moving forward forty five years or more, I'd urge anyone doing these comparisons to not continually swap between them in a prolonged session, but to try in a relaxed way and not concentrate too hard. If there's any difference at all, it'll creep up on you I feel...

 

[kemmler3D]

The test has nothing to do with sinad,

My bad, I should learn to read.

 

[melomane13]

i am sure than they sound the same, but the methodology is not ok: if the dac with which I listen is less competitive than those tested, how could I hear any differences?

 

[tmtomh]

i am sure than they sound the same, but the methodology is not ok: if the dac with which I listen is less competitive than those tested, how could I hear any differences?

You raise a good question, and to my eyes it's similar to what @Brian Hall commented earlier as well:

Is this really a valid test? Aren't we just hearing our own DACs version of what was recorded?

I think it's about audible transparency: if your current DAC converts digital to analogue with levels of noise, distortion, and frequency linearity that are equal to or better than human hearing thresholds, then your DAC should be good enough for you to be able to detect any audible differences in the output of the three DACs in Archimago's test.

Or to put it another way, if you believe that different DACs can be excellent performers when it comes to distortion and noise and linearity but still sound different in subtle ways then your DAC might impose its own "sound signature" on all three of Archimago's samples, but that doesn't mean your DAC's "sound signature" will obscure or obliterate the differences between the three DACs he used to generate his sample files. To use a crude visual analogy, say the three DACs' outputs are all very "clear," but one's output is slightly "blue," the other's is slightly "red" and the other's is slightly "green." And say your own DAC is also very "clear" but slightly "green." Your DAC's "green tint" will change how you perceive the color of the three DACs Archimago used in his test: the red one will look brownish, the blue one will look blue-greenish, and the green one will look extra green or darker green. But it won't change the fact that the three DACs produce perceptually different colors. So your own DAC could alter your preference among Arch's Samples A, B, and C, but that doesn't mean it will destroy your ability to detect a difference.

Now, in my opinion none of this matters because chances are that all three of the DACs Arch used, plus the DACs used by 99 to 100% of us who are participating in this listening test, are all audibly transparent. Not to mention, I know he used a very nice ADC to capture the analogue output of those DACs, but in general if there's a weak link in a DAC-ADC-DAC chain of processing in terms of measured specs of noise and distortion, it's usually the ADC anyway, not the DACs. But again, a very good quality ADC is IMHO also going to be audible transparent.

 

[Sokel]

If someone present an successful ABX between those files where he will show he can choose amongst them it would mean that he can either distinguish a difference between -120 and -160(!) db attenuation at 22Khz and about 1db difference in FR from 15Khz and upwards or the differences in phase. (The later (-160db) must have a really nice filter admittedly)

I would REALLY like to see that.

 

[DVDdoug]

Moving forward forty five years or more, I'd urge anyone doing these comparisons to not continually swap between them in a prolonged session, but to try in a relaxed way and not concentrate too hard. If there's any difference at all, it'll creep up on you I feel...

That's fine as long as the tests are blind, level-matched, and the results statistically valid/repeatable. There are no time limits in an ABX test but the people who do this kind of thing routinely, find that our short-term hearing memory is better so it's usually easier to reliably hear a difference with fast switching.

 

[melomane13]

You raise a good question, and to my eyes it's similar to what @Brian Hall commented earlier as well:



I think it's about audible transparency: if your current DAC converts digital to analogue with levels of noise, distortion, and frequency linearity that are equal to or better than human hearing thresholds, then your DAC should be good enough for you to be able to detect any audible differences in the output of the three DACs in Archimago's test.

Or to put it another way, if you believe that different DACs can be excellent performers when it comes to distortion and noise and linearity but still sound different in subtle ways then your DAC might impose its own "sound signature" on all three of Archimago's samples, but that doesn't mean your DAC's "sound signature" will obscure or obliterate the differences between the three DACs he used to generate his sample files. To use a crude visual analogy, say the three DACs' outputs are all very "clear," but one's output is slightly "blue," the other's is slightly "red" and the other's is slightly "green." And say your own DAC is also very "clear" but slightly "green." Your DAC's "green tint" will change how you perceive the color of the three DACs Archimago used in his test: the red one will look brownish, the blue one will look blue-greenish, and the green one will look extra green or darker green. But it won't change the fact that the three DACs produce perceptually different colors. So your own DAC could alter your preference among Arch's Samples A, B, and C, but that doesn't mean it will destroy your ability to detect a difference.

Now, in my opinion none of this matters because chances are that all three of the DACs Arch used, plus the DACs used by 99 to 100% of us who are participating in this listening test, are all audibly transparent. Not to mention, I know he used a very nice ADC to capture the analogue output of those DACs, but in general if there's a weak link in a DAC-ADC-DAC chain of processing in terms of measured specs of noise and distortion, it's usually the ADC anyway, not the DACs. But again, a very good quality ADC is IMHO also going to be audible transparent.

human hearing thresholds is the problem.
After some tests i have made and read serious paper on audibility of distortion, it is clear to me that the threshold is really high, -50db thd, more if only present of the second order and third thd.
I can't hear -20 db , 10% H2 on music!
in practice only noise can be a problem, not distortion and not frequency response, since it can be linearized

 

[tmtomh]

human hearing thresholds is the problem.
After some tests i have made and read serious paper on audibility of distortion, it is clear to me that the threshold is really high, -50db thd, more if only present of the second order and third thd.
I can't hear -20 db , 10% H2 on music!
in practice only noise can be a problem, not distortion and not frequency response, since it can be linearized

Agreed.

 

[MaxBuck]

... my guessing results were surprising - not because I guessed wrong, but rather because I guessed wrong every single time. I mean, what are the odds of that?

Based on my success in guessing what my wife really expects me to do, I'd say the odds can be pretty good.

 

[Ken Tajalli]

I just posted the same link in another thread, you know, the one about Goldensound listening test.
I didn't know this thread existed.
I took the test. I think I have found my preference. I am going to get my daughter to change the sample names, so I can take it again tomorrow, to see if I pick the same ones.

 

[Ken Tajalli]

Here we go!
We now know what the three DACs wee.
- A was a $9 Apple Dongle.
- B was a $3000 Linn Magic DS
- C was a $30000 Linn Klimax
My choice was A-C-B !
So I picked the $9 DAC as the best of the bunch. Now I had the Apple DAC and gave it away.
This was a comparison test, so for me, the conclusion was no vindication of the Apple dongle, but proof that the Linns were crap.
He should have included a Topping and a Chord DAC, then the results would have been more interesting.

 

[tmtomh]

Here we go!
We now know what the three DACs wee.
- A was a $9 Apple Dongle.
- B was a $3000 Linn Magic DS
- C was a $30000 Linn Klimax
My choice was A-C-B !
So I picked the $9 DAC as the best of the bunch. Now I had the Apple DAC and gave it away.
This was a comparison test, so for me, the conclusion was no vindication of the Apple dongle, but proof that the Linns were crap.
He should have included a Topping and a Chord DAC, then the results would have been more interesting.

The conclusion is that they're all audibly transparent.

 

[aon500]

I'm correct C-B-A .
Apple Dongle have a detail boost or something that sound harsh.

 

[tmtomh]

I'm correct C-B-A .
Apple Dongle have a detail boost or something that sound harsh.

 

[Palladium]

Here we go!
We now know what the three DACs wee.
- A was a $9 Apple Dongle.
- B was a $3000 Linn Magic DS
- C was a $30000 Linn Klimax
My choice was A-C-B !
So I picked the $9 DAC as the best of the bunch. Now I had the Apple DAC and gave it away.
This was a comparison test, so for me, the conclusion was no vindication of the Apple dongle, but proof that the Linns were crap.
He should have included a Topping and a Chord DAC, then the results would have been more interesting.

Subjectively preferring cheapos is wrong because *insert emperor new's clothes arguments"

 

[Ken Tajalli]

The conclusion is that they're all audibly transparent.

That's your conclusion, I take it you took the blind challenge.
Indeed when I did the test and reported back, I did mention that B and C sounded very much alike. to me it means (now that unveiled) that Linn DACs sound alike, possibly have a house-sound (that I don't like).

 

[solderdude]

105 respondents if 1/3 of them chose A to be the best, and 1/3 chose B to sound 'best' and the other 3rd chose C to sound the best one could draw the conclusion that the reason for selecting the order is random.

In the end there are only 6 possible choices one could make and with 4 of them one could easily state: The Linn DACs are audibly better (2/3 chance) and with 1/3 of the possible options you could say: Well the apple dongle is good enough for me.

The info that comes along with the responses is far more interesting though. I think that is what Archimago is really looking at but people who done the test only want to know if they got the correct 'value order' correct 1 out of 6 chance for that even without listening.

The best way to perhaps take this test (when one really wants to know) is to ABX A and B, and ABX A and C, and ABX B and C and see which one can be told apart reliably on your gears/ears.

PS... I did not do the test and did not submit any results. I am DAC deaf so there is no point for me nor do I really care for the differences at this magnitude anyway.

 

[pma]

100x, so 20dB? Not hard to find DACs that do 90dB or 110dB SINAD, yep turns out that was a mention of price, not performance... still that's

100x would be 40dB difference in voltage, noise or SINAD. Not 20 dB. I thought you would be one to know the basics .

 

[Ken Tajalli]

105 respondents if 1/3 of them chose A to be the best, and 1/3 chose B to sound 'best' and the other 3rd chose C to sound the best one could draw the conclusion that the reason for selecting the order is random.

In the end there are only 6 possible choices one could make and with 4 of them one could easily state: The Linn DACs are audibly better (2/3 chance) and with 1/3 of the possible options you could say: Well the apple dongle is good enough for me.

The info that comes along with the responses is far more interesting though. I think that is what Archimago is really looking at but people who done the test only want to know if they got the correct 'value order' correct 1 out of 6 chance for that even without listening.

The best way to perhaps take this test (when one really wants to know) is to ABX A and B, and ABX A and C, and ABX B and C and see which one can be told apart reliably on your gears/ears.

PS... I did not do the test and did not submit any results. I am DAC deaf so there is no point for me nor do I really care for the differences at this magnitude anyway.

It is a silly comparison test, wholly at the mercy of playback equipment.
If there are differences, then the playback device is a random element.
If there are no differences, then playback equipment equally makes no difference.
I believe that! was his point.
That a $9 dongle is at least comparable to top tier (pricewise) DACs.
I hope to see the full stats too.
One of the questions on the form was:
Do you believe there are differences in DACs sound?

 

[solderdude]

Differences could be shown to exist in DW between the DACs and even be made audible.
Have not seen any DW sorcery on the files yet (as with GS filter ABX)

Do you believe there are differences in DACs sound?

There is no need for 'believe' here but totally understand that question from Archimago as the vast majority believes just that.
Obviously DACs can be told apart as long as they differ in output within the audible range of the listener and with the right stimulus an gear.

But since you indirectly asked if I believe there are differences I can only say that the few DACs I tried, that were all more than decent and have 'proper' reconstruction filters, is that when they are level matched I can't tell those apart. (DAC deaf is what I call this).
I have not had the pleasure of toying around with purposely designed DACs that have a 'sound' and am not interested with playing with incorrect filters (some of my DACs have that option but why go for a 'sound' when the idea is to properly reproduce the music file.
If I want something to sound different then the usage of tone control, EQ or different transducers would be far more effective than the small (and only in upper extremes) different response a (non-defective) DAC can make.

In the Archimago test it really is not about the price of the DAC though but gathering intel from the respondents.
Always better to have 100 opinions than only 1 or 2 opinions.

People buying expensive gear want (and thus do) the expensive one to be better. People who don't want to buy expensive gear or don't hear it do want the cheap gear to be just as good (or even better). Most folks want something in between.... not the cheapest and not too straining on their wallet and believe that's more than good enough.

People and their choices.