• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required as is 20 years of participation in forums (not all true). There are daily reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

DAC ABX shootout - unable to distinguish between 10$ and 15k$

Echoes

Member
Joined
Feb 21, 2022
Messages
21
Likes
142
Hello All,

First of all, thank you for the forum contributors who are sharing their knowledge here. I started reading ASR two month ago and just can't get enough of it.
I am sharing my experience with my first ABX testing. Last Friday me, together with a friend performed a double blind test on this systems:

1) Chord Dave + Upscaller from Chord2Go+2U - headphone output
2) Topping D90 + A90 from a laptop
3) Chord Mojo headphone output from Iphone.
4) Apple lightning 3.5mm adapter, output from Iphone.

Volume matching was performed with a dedicated microphone, nothing fancy, but still more reliable that our ears as proven below.
Switching between sources was done though a 4 position 3 pole rotary switch box, unbalanced 6.3mm cables.
Headphones used were the Seinheiser HD650, standard.
Switching was done randomly by my wife, who, with her scientific background, was happy to assist on this.

Result.. We could not tell the difference reliably between the systems. Which is.. proving either that we are both deaf or audio fools. We are repeating the test this Friday, I will post update if I can still type though my tears. Silly enough I can 100% reliably say which one is better when I see what system is connected.
 

Purité Audio

Master Contributor
Industry Insider
Barrowmaster
Forum Donor
Joined
Feb 29, 2016
Messages
5,237
Likes
4,716
Location
London
May I ask did you build or buy your switch box?
It is shocking when you first compare two ( properly engineered) dacs and realise that you cannot hear any difference, but it does allow you to focus on the aspects of reproduction that really can improve SQ.
Keith
 

dominikz

Senior Member
Forum Donor
Joined
Oct 10, 2020
Messages
487
Likes
1,692
Hello All,

First of all, thank you for the forum contributors who are sharing their knowledge here. I started reading ASR two month ago and just can't get enough of it.
I am sharing my experience with my first ABX testing. Last Friday me, together with a friend performed a double blind test on this systems:

1) Chord Dave + Upscaller from Chord2Go+2U - headphone output
2) Topping D90 + A90 from a laptop
3) Chord Mojo headphone output from Iphone.
4) Apple lightning 3.5mm adapter, output from Iphone.

Volume matching was performed with a dedicated microphone, nothing fancy, but still more reliable that our ears as proven below.
Switching between sources was done though a 4 position 3 pole rotary switch box, unbalanced 6.3mm cables.
Headphones used were the Seinheiser HD650, standard.
Switching was done randomly by my wife, who, with her scientific background, was happy to assist on this.

Result.. We could not tell the difference reliably between the systems. Which is.. proving either that we are both deaf or audio fools. We are repeating the test this Friday, I will post update if I can still type though my tears. Silly enough I can 100% reliably say which one is better when I see what system is connected.
Great work - always nice to see people going through the trouble to do blind listening!

What you got is IMO not a surprising result - perhaps you will find my online DAC ABX test interesting as well. That one compared an objectively flawed low-budget DAC (FiiO D03K) with one that is SOTA performing (Topping E50) and so far the large majority of participants seem to be struggling to detect any difference.

Just a suggestion on methodology - with DACs you'll be able to level match much more closely if you can measure the DAC electrical output directly with a voltmeter (or an ADC), instead of using a microphone to try and match the acoustical output from a transducer.
 
OP
E

Echoes

Member
Joined
Feb 21, 2022
Messages
21
Likes
142
The switch was built not bought, aluminum enclosure, 4x 6.3mm inputs 1x 6.3mm output. This: https://www.lemona.lt/Files/Instrukcijos/TI/En/Pdf/Galetiniai.pdf was used for the switching, 3 pole 4 way version. I have tried similar experiment with direct cabling, the switch box really does not degrade the sound quality, well... for my ears.

First of all I feel blessed just to able to listen to these piece of gear. Chord Dave is listening on its own its almost an euphoric experience. Going though different up sampling rates and even connecting up sampler to topping D90 seems to open up a brand new world in audio definition, but... when you do it blindly. Well I am lacking words to describe my disappointment.
 
OP
E

Echoes

Member
Joined
Feb 21, 2022
Messages
21
Likes
142
Great work - always nice to see people going through the trouble to do blind listening!

What you got is IMO not a surprising result - perhaps you will find my online DAC ABX test interesting as well. That one compared an objectively flawed low-budget DAC (FiiO D03K) with one that is SOTA performing (Topping E50) and so far the large majority of participants seem to be struggling to detect any difference.

Just a suggestion on methodology - with DACs you'll be able to level match much more closely if you can measure the DAC electrical output directly with a voltmeter (or an ADC), instead of using a microphone to try and match the acoustical output from a transducer.
Thanks for replying, I shall. Also will perform the suggested hearing threshold testing sequence on my free time. Well at least I can tell the difference in different headphones, as the blind testing.. is impossible with them!
 

BDWoody

Chief Cat Herder
Moderator
Forum Donor
Joined
Jan 9, 2019
Messages
5,350
Likes
15,400
Location
Mid-Atlantic, USA. (Maryland)
Which is.. proving either that we are both deaf or audio fools.

What it proves is that you are open to being a normal human like the rest of us.

How amused was your wife?

Welcome! Great first post!
 
OP
E

Echoes

Member
Joined
Feb 21, 2022
Messages
21
Likes
142
What it proves is that you are open to being a normal human like the rest of us.

How amused was your wife?

Welcome! Great first post!
Thanks for the moral support. My wife reaction was "if it measures better, it must be better, even though none of you can tell the kits apart". As long that don't purchase on credit or prioritize snake-oil cables over a family holiday my marriage is still strong.
 

BDWoody

Chief Cat Herder
Moderator
Forum Donor
Joined
Jan 9, 2019
Messages
5,350
Likes
15,400
Location
Mid-Atlantic, USA. (Maryland)
Thanks for the moral support. My wife reaction was "if it measures better, it must be better, even though none of you can tell the kits apart". As long that don't purchase on credit or prioritize snake-oil cables over a family holiday my marriage is still strong.

At the very least, this exercise will have you looking at typical reviews differently, won't it?

My audio life can easily be divided into before and after I did a controlled test.
 
OP
E

Echoes

Member
Joined
Feb 21, 2022
Messages
21
Likes
142
At the very least, this exercise will have you looking at typical reviews differently, won't it?

My audio life can easily be divided into before and after I did a controlled test.
Its a tough one to swallow, I still need to comprehend it. Somewhere deep I still blame the "mid-tier" HD650, although they have given me many, many hours of hi-fi pleasure. Soon to perform a test with Audeze LCD-5 to fully seal the deal on the DAC selection..
 

tvrgeek

Major Contributor
Joined
Aug 8, 2020
Messages
1,017
Likes
538
Location
North Carolinia
These tests , with all good intentions, are not conclusive beyond the OP. I can't tell him what he does or does not hear, and no one can tell me what I hear and do not hear.

I found on SOME tracks, on SOME speakers, on SOME DACs, there to be a clear, as in walk by glaringly clear, difference audible to SOME people. Not talking about some abstract "musicality" or pick your blog reviewer string of vague words. For me, it is something that causes a specific unpleasant "glare" in the midrange. Soprano vocals and more obviously the sound of a cymbal. To a lessor extent, trumpets and violins. Very clear to me and more so to my wife. "Good" and "bad" do not track with price. On serval sets of headphones, I too could hear no difference even listening carefully, eyes open, to the specific issue at hand. I do find it interesting all of the sounds are from instruments with very rich and broad overtones. I suspect that is a clue. I have theories but I could not correlate them to the tests Amir published. I suspect, on some much better headphones, with much younger ears, and some select recordings, differences in detail right at the threshold of hearing may be audible, but not for me. Differences in the source material dithering may or may not mask details at that level if the DAC is of sufficient quality ( Apple dongle or better). It seems a very bright EQ is preferred by some and that may make those differences to be real, not just bloggers filling space. Not for me.

In other words, it is still a bit more complicated. Not mentioned is some of the higher priced units have intentional distortions to add " warmth" or "color" as many believe these distortions to add to their enjoyment. Some of these are made up in advertising, some are clearly audible. The question is personal which is better for them. Some prefer tube amplifiers, some prefer LPs. I seem to prefer MOSFET power amps. I can't tell you why. Just over time, I seem to enjoy the music more. FWIW, my amplifiers actually do measure in the traditional SINAD very well but of all the amplifiers I have tried at home, the only bi-polar I liked was the Aragon. All the rest were MOSFET. Well, to think about it , my old Creek sounds fine, but I only have ever used it for background music. Not quite big enough for my main system. If we really knew what our brain interprets as musical, we could implement a set of transfer functions in DSP and we could pick the one that suits us best. We just don't know. It is science, not magic. With science, we should peruse the reasonings. Not claim they don't exist.

The good news for me is the JDS DAC+ does not have the issue I am sensitive to and I my tweeter testing suggests that after my swap, neither will my Schiit. So, had the Topping not had the driver incompatibilities and dropout issues, it too would have been fine and the feature set was worth the price to me. With the JDS I hear nothing wrong and find listening for hours to be enjoyable. That is the parameter that maters most to me. I kind of miss the remote volume control as some of my disks are from the loudness wars, and others much wider dynamic range so could use a boost to seem the same level. I would rather not have to get up to adjust.

A conclusion might be if you have money to spend, use it on the speakers and environment and not the DAC. I see questions on other forums asking if that $1000 DAC would help the "soundstage" on their $500/pair speakers. My answer will remain "No, but $50 worth of OC 705 and a used sheet will"
 

BDWoody

Chief Cat Herder
Moderator
Forum Donor
Joined
Jan 9, 2019
Messages
5,350
Likes
15,400
Location
Mid-Atlantic, USA. (Maryland)
These tests , with all good intentions, are not conclusive beyond the OP.

That's all they need to be.

While his testing wasn't airtight, it was enough for him.

With science, we should peruse the reasonings. Not claim they don't exist.

Who claims what doesn't exist? If a claim never gets beyond anecdotal, maybe that's a clue about what's worth pursuing in the first place. Pursuing every claim seems like a pretty inefficient way of advancing anything resembling science.
 

JayGilb

Addicted to Fun and Learning
Joined
Jul 22, 2021
Messages
608
Likes
826
Location
West-Central Wisconsin
We just don't know. It is science, not magic. With science, we should peruse the reasonings. Not claim they don't exist.
Yet none of your claims are backed by science, just your personal opinion. Setup a controlled abx shootout and present your findings and then the science can begin.
 

Shazb0t

Addicted to Fun and Learning
Joined
May 1, 2018
Messages
602
Likes
1,116
Location
NJ
We just don't know. It is science, not magic. With science, we should peruse the reasonings. Not claim they don't exist.
The burden of proof lies on the claimant. You, as well as the rest of the population making these claims, haven't been able to show that anyone can reliably hear these differences in controlled testing.

There is zero reason for people to investigate the cause of something which hasn't been shown to exist. It wouldn't matter what the actual cause was, people would care if there was ever reliable scientific evidence a difference existed. In the face of that lack of evidence your musings seem like the rantings of someone who doesn't get it.
 
Last edited:

tvrgeek

Major Contributor
Joined
Aug 8, 2020
Messages
1,017
Likes
538
Location
North Carolinia
The burden of proof relies on the claimant. You, as well as the rest of the population making these claims, haven't been able to show that anyone can reliably hear these differences in controlled testing.

There is zero reason for people to investigate the cause of something which hasn't been shown to exist. It wouldn't matter what the actual cause was, people would care if there was ever reliable scientific evidence one existed. In the face of that lack of evidence your musings seem like the rantings of someone who doesn't get i
That's all they need to be.

While his testing wasn't airtight, it was enough for him.



Who claims what doesn't exist? If a claim never gets beyond anecdotal, maybe that's a clue about what's worth pursuing in the first place. Pursuing every claim seems like a pretty inefficient way of advancing anything resembling science.
Your first statement is exactly what I said.

Who claims differences don't exist, ? A lot of folks on this sight. Why they read reviews if they don't; believe any differences I do not understand.
 

BDWoody

Chief Cat Herder
Moderator
Forum Donor
Joined
Jan 9, 2019
Messages
5,350
Likes
15,400
Location
Mid-Atlantic, USA. (Maryland)
Who claims differences don't exist, ? A lot of folks on this sight. Why they read reviews if they don't; believe any differences I do not understand.

It's more about understanding what the differences are, and what they mean. It's not that they don't exist.

At the point where measured differences are overwhelmingly likely to make zero audible difference, yet folks make claims to the contrary, they will often be asked for more support than 'because I say so.'

For some, this leads to much weeping and gnashing of teeth, but the reality is a lot of this stuff will not be distinguishable once filters/levels and bias is controlled for.

Actual evidence to the contrary is welcomed.
 

dominikz

Senior Member
Forum Donor
Joined
Oct 10, 2020
Messages
487
Likes
1,692
Who claims differences don't exist, ? A lot of folks on this sight
Differences absolutely exist - they are measurable. What is questionable is audibility (barring severely broken devices or misuse) under standard listening conditions.
Why they read reviews if they don't; believe any differences I do not understand.
Primarily to confirm that devices are not broken or poorly engineered, but also because there are other use-cases where the extra performance is indeed required.
E.g. I bought the Topping E50 not because it sounds better connected to my HiFi system (it sounds the same as my RME), but because I needed a DAC with ultra-low noise for high-gain guitar reamping purposes (112dB SNR of my RME was not enough to keep DAC noise floor beneath guitar amp baseline), and also to use as signal generator for audio measurements.

Having access to quality measurements like on ASR made my search much easier :)
 

DVDdoug

Addicted to Fun and Learning
Joined
May 27, 2021
Messages
999
Likes
1,214
My wife reaction was "if it measures better, it must be better, even though none of you can tell the kits apart".
But it's not better for the intended purpose (listening-to and enjoying music. ;)

...It's not hard to build an amplifier that goes to 1MHz but it wouldn't make sense to pay more for it. Even at the same price that doesn't (or shouldn't) make it more desirable than an audio amplifier that "only" covers the audio range. (...Probably a bad example because there are good for filtering-out RF.)

Or, you could have a school bus that goes 200MPH! :D

Speaking of frequency response... Sometimes people say that one reason vinyl is better than CD is that frequency response can sometimes extend higher. But CDs are flatter over the audible range. (Real world records usually just contain noise, and maybe distortion, in the ultrasonic range and extended frequency response doesn't make-up for all the other reasons vinyl is inferior.)
 
Top Bottom