Archimago has a new DAC listening test

[Ken Tajalli]

Differences could be shown to exist in DW between the DACs and even be made audible.



There is no need for 'believe' here.
Obviously DACs can be told apart as long as they differ in output within the audible range of the listener and with the right stimulus an gear.

But since you indirectly asked if I believe there are differences I can only say that the few DACs I tried, that were all more than decent and have 'proper' reconstruction filters, is that when they are level matched I can't tell those apart. (DAC deaf is what I call this).
I have not had the pleasure of toying around with purposely designed DACs that have a 'sound' and am not interested with playing with incorrect filters (some of my DACs have that option but why go for a 'sound' when the idea is to properly reproduce the music file.
If I want something to sound different then the usage of tone control, EQ or different transducers would be far more effective than the small (and only in upper extremes) different response a (non-defective) DAC can make.

In the Archimago test it really is not about the price of the DAC though but gathering intel from the respondents.
Always better to have 100 opinions than only 1 or 2 opinions.

People buying expensive gear want (and thus do) the expensive one to be better. People who don't want to buy expensive gear or don't hear it do want the cheap gear to be just as good (or even better). Most folks want something in between.... not the cheapest and not too straining on their wallet and believe that's more than good enough.

People and their choices.

I never asked the question!
It was one of the questions on Archie's report back form.
I believe the question was a psych evaluation.
If one does not "believe" that DACs can sound different (evidently such as yourself) then the chances may be that you won't find any.
However if one does have such a belief, chances are that he might do! the test may show if he actually can, successfully.
So it was a valid question, in that aspect.
But hey, if you have anything to say, say it to Archie.

 

[solderdude]

I do believe not all DACs actually sound the same.
Merely mentioned that the few DACs I listened to did not sound different to me. That is, all of those had proper reconstruction filters.

Having such a belief will make you think you heard a difference and may even hear one even when there is none.
While doing the test believing you can't hear a difference the 3 files might sound the same even if there are (minute) audible differences.
It goes both ways.

If one really wants to know then ABX (20 attempts) A & B, A & C and B & C. This will be more accurate if the goal is to find differences, you might even have a preference.

Archimago knows all the pitfalls of the test and for that reason I think he is more interested in the comments than finding the folks who guessed (or actually heard, we won't be able to tell) who got the prices/sound quality in the 'correct' order.

 

[aon500]

 

[board]

I agree with solderdude that you and @Ken Tajalli really should try to ABX the three samples to see if you can actually tell them apart. I'm not trying to sound antagonistic or anything - I'm just thinking that it's not uncommon for people to choose a preference even though there are no audible differences, case in point being a test where the same song was played twice in a row, and, if I remember correctly, 76 % of the listeners had a preference for one over the other.

 

[Ken Tajalli]

I agree with solderdude that you and @Ken Tajalli really should try to ABX the three samples to see if you can actually tell them apart. I'm not trying to sound antagonistic or anything - I'm just thinking that it's not uncommon for people to choose a preference even though there are no audible differences, case in point being a test where the same song was played twice in a row, and, if I remember correctly, 76 % of the listeners had a preference for one over the other.

ABX is not a magic bullet.
What I did was I did my auditioning at length and made a ranking decision.
A was the better one, with B and C sounding similar, but C having an edge.
Then my daughter changed the samples and their names and I took the test next day. I had picked the same ranking order.
Now, according to Archie we know that, B and C are from same manufacturer employing the same recon filter. While A is an Apple dongle with a different recon filter!
No ABX was needed nor used.
ABX for me gets confusing. after a few attempts, I don't know if I'm coming or going!

 

[danadam]

Then my daughter changed the samples and their names and I took the test next day. I had picked the same ranking order.

And how many times did you repeat that process?

 

[Ken Tajalli]

And how many times did you repeat that process?

2643 times .
Twice in total, while both my eyes were poked blind.
It was just a fun experiment.
I wasn't trying to prove anything.

 

[danadam]

It was just a fun experiment.
I wasn't trying to prove anything.

Ah, ok. For a moment I thought you were trying to say something about ABX tests. Probably my fault for reading too much from the post

 

[krabapple]

ABX is not a magic bullet.
What I did was I did my auditioning at length

You're totally able to do that with ABX

and made a ranking decision.

You could even do that, using ABX

A was the better one, with B and C sounding similar, but C having an edge.
Then my daughter changed the samples and their names and I took the test next day. I had picked the same ranking order.
Now, according to Archie we know that, B and C are from same manufacturer employing the same recon filter. While A is an Apple dongle with a different recon filter!
No ABX was needed nor used.

Ah, another anecdote from sighted listening, let's add it to the pile.
But it should be easy for you to ace an ABX, given this.

ABX for me gets confusing. after a few attempts, I don't know if I'm coming or going!

Which suggest maybe you just got lucky in those sighted tests.

 

[Ken Tajalli]

You're totally able to do that with ABX

I can do so many things with ABX.
Like avoiding it.
You like it, more power to you.
What sighted result?
I didn't know what A or B or C were, they could have been the same, or two of them could have been the same.
On the next day, I didn't even know which one was A or ....
Nothing was sighted, it was totally blind.
Blind tests are a must, but some people bash it to death, when it doesn't produce their preconceived results.

 

[krabapple]

I can do so many things with ABX.
Like avoiding it.
You like it, more power to you.

This is weird phasing. Like saying I 'like' a standard blood test as part of a physical. But I guess you mean ABX versus another blind protocol?

What sighted result?
I didn't know what A or B or C were, they could have been the same, or two of them could have been the same.
On the next day, I didn't even know which one was A or ....
Nothing was sighted, it was totally blind.

I wouldn't say *totally* blind without more details, but yes, sorry, I'll retract 'sighted', and call yours a blind preference test -- one halted well before it gained enough power to conclude anything, alas.

As I wrote, ABX accomodates long auditioning, no problem there. But it's designed for establishing difference between two sources, rather than preference; and certainly it's awkward for a three-way comparison, though that can be done. I suspect Archimago suggests using the foobar ABX comparator mainly for its quick switching, i.e., to audition A and B pairs and rank them by preference, there being no need for X in that case. There are better blind tests designed for preference, though.

Those here who want you to use ABX do so because if a preference is based on sound, there must be an audible difference , and ABX testing should detect it. Typically difference is established before anyone bothers to test for preference.

 

[pablolie]

...
If one does not "believe" that DACs can sound different (evidently such as yourself) then the chances may be that you won't find any.
...

I'd say if someone is convinced all DACs sound the same, they would not undergo the exercise to begin with. Why would they? I personally established many years ago, through long and unenjoyable testing to micro differences, that MP3 (>256k VBR) vs CD vs HD doesn't impact my ability to enjoy music one iota, and I'll never go through that ever again.

As it is, we know "some DACs" sound differently, by design intent. Or they allow for configuration - different harmonics and filters etc etc. So what? It doesn't in any way dispute the simple fact that things that measure the exact same through a variety of tests (frequency response, load dependency) will be utterly indistinguishable to the ear, if not to other senses.

 

[Ken Tajalli]

I'd say if someone is convinced all DACs sound the same, they would not undergo the exercise to begin with. Why would they? I personally established many years ago, through long and unenjoyable testing to micro differences, that MP3 (>256k VBR) vs CD vs HD doesn't impact my ability to enjoy music one iota, and I'll never go through that ever again.
As it is, we know "some DACs" sound differently, by design intent. Or they allow for configuration - different harmonics and filters etc etc. So what? It doesn't in any way dispute the simple fact that things that measure the exact same through a variety of tests (frequency response, load dependency) will be utterly indistinguishable to the ear, if not to other senses.

True words!
Just one observation, those who do believe all DACs sound the same, may be chosen as a member of an auditioning group at random, unless filtered out. I think that's why Archie was asking the question (my guess).
One thing though is established now, full transparency does not exist! but does it matter? is anothe question.

As I wrote, ABX accomodates long auditioning, no problem there. But it's designed for establishing difference between two sources, rather than preference; and certainly it's awkward for a three-way comparison, though that can be done. I suspect Archimago suggests using the foobar ABX comparator mainly for its quick switching, i.e., to audition A and B pairs and rank them by preference, there being no need for X in that case

By broad definition of the phrase ABX, I suppose I did do an ABX, but using such a device as Foobar with ABX addon, No - I don't like that, too cumbersome and confusing to me, but it is just me.
In this case, it was a matter of preference, but detecting differences between A or B (as you said) was not the goal here.
My stats is not that good, but I reckon in my case the probability of chance was about 3%, which is significant, but satisfactory to me.

 

[krabapple]

True words!
Just one observation, those who do believe all DACs sound the same, may be chosen as a member of an auditioning group at random, unless filtered out. I think that's why Archie was asking the question (my guess).
One thing though is established now, full transparency does not exist! but does it matter? is anothe question.

Where are you getting these ideas?

By broad definition of the phrase ABX, I suppose I did do an ABX, but using such a device as Foobar with ABX addon, No - I don't like that, too cumbersome and confusing to me, but it is just me.
In this case, it was a matter of preference, but detecting differences between A or B (as you said) was not the goal here.
My stats is not that good, but I reckon in my case the probability of chance was about 3%, which is significant, but satisfactory to me.

By no definition did you do an ABX (in which your task is to identify whether X is A or B). You did a blind test. There are several varieties of blind test.

And how do you reckon your result has a p=~.03?

 

[Ken Tajalli]

Where are you getting these ideas?


By no definition did you do an ABX (in which your task is to identify whether X is A or B). You did a blind test. There are several varieties of blind test.

And how do you reckon your result has a p=~.03?

I am wrong, you are correct.
my mistake.

 

[board]

One thing though is established now, full transparency does not exist!

Maybe I'm somewhat saying the same thing as Krabapple, but I would say that you're correct that full transparency doesn't exist, since any component will produce distortion and noise. But full transparency does exist within audibility, since many, maybe even most, components (especially DACs) have distortion and noise below the threshold of audibility.

 

[aon500]

1st time listen to C is the best far from B.
B need system that very good dynamic to show how good it is.
A is some cheap stuff that focus on sound detail.

But it will very hard to tell the different if we listen separately.

 

[solderdude]

I too found C to be the best sounding one by far. It was really easy to hear on my $ 500k system.
B was only slightly below the performance of C but still very good.
A was total crap and sounded crappy, a bit like my phone.

I can easily write this as the curtain has been lifted (not the veil) and I can safely post it as:
A: no one is gonna check what gear I have.
B: no one is gonna check how good my hearing is
C: I already new what DACs are used and can write my response accordingly.

 

[Ken Tajalli]

full transparency doesn't exist, since any component will produce distortion and noise. But full transparency does exist within audibility, since many, maybe even most, components (especially DACs) have distortion and noise below the threshold of audibility.

Well Cameron (goldensound) and Sharur (did I say that right?)
invalidated that statement.

 

[solderdude]

Well Cameron (goldensound) and Sharur (did I say that right?)
invalidated that statement.

Not exactly... what Cameron and Sharur showed is that they can successfully differentiate between 2 different filters (reproduced on the same DAC, amp and headphone)

This was not about the, by @board mentioned, aspects like distortion and noise which arguably were not tested and are below audible thresholds with any modern well designed DAC.