• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required as is 20 years of participation in forums (not all true). There are daily reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

DAC ABX Test Phase 1: Does a SOTA DAC sound the same as a budget DAC if proper controls are put in place? Spoiler: Probably yes. :)

Volikovvv

Member
Joined
Nov 6, 2020
Messages
62
Likes
31
They are more than 30 dB apart in distortion, that is more than a factor of 30 voltage wise. That is a massive difference.
Interesting where is the 30db difference?
and
Maybe im missing something, please show me where to look plz.
 

voodooless

Master Contributor
Forum Donor
Joined
Jun 16, 2020
Messages
6,071
Likes
10,052
Location
Netherlands
Interesting where is the 30db difference?
and
Maybe im missing something, please show me where to look plz.
It clearly says so in the first post:
C6FA0AD6-DA6B-4FDE-A82E-2C0B94108178.jpeg

Or were you referring to another test?
 

voodooless

Master Contributor
Forum Donor
Joined
Jun 16, 2020
Messages
6,071
Likes
10,052
Location
Netherlands
Yes of course, you replied to my message about D1se and D90 dac, see below:)
View attachment 177848

Anyway, never mind:)
But those would add a few more dB at best. The test already has a 30 dB difference. What would be the point? Suddenly those few extra dB would make all the difference? Why not the other 30?
 

Volikovvv

Member
Joined
Nov 6, 2020
Messages
62
Likes
31
But those would add a few more dB at best. The test already has a 30 dB difference. What would be the point? Suddenly those few extra dB would make all the difference? Why not the other 30?
I noted before, that i don't care about measurments, so you better say if it should. But from my experience they sound different because of post filters, different output stage, different reaction of amplifier to different input voltages of the dac:) So i can distingish them easily, as i distinguished that fiio and e50, im not listening to db's or measurments, im listenings by ears and brain to semi-tones of the music provided by the system. these db say nothing for me, and to almost all people - as we can see from this test. I don't think this disccussion make sense, come at my city, lets do a comparison and blind test if you don't trust me:)

upd: We can get very similary measured su-9n and d1se, and give me a try. They sound very different to me, but when we look to the digits, they are the same... I know this forum don't like statements like that, so... come to Ukraine, Kyiv, i will ask for help in upcoming war.
 
Last edited:
OP
dominikz

dominikz

Addicted to Fun and Learning
Forum Donor
Joined
Oct 10, 2020
Messages
580
Likes
1,995
I noted before, that i don't care about measurments, so you better say if it should.
That is absolutely fine - it is a hobby and you should approach it as you like. :)

But it is also good to be aware that there is good science behind the technology, and a lot of solid predictions can be made when one understands some of the technical details. That is of course not to say that everyone needs to dive into the science to enjoy the hobby :)

You may have noted that in post #1 of this thread I wrote:
Though I consider this to be a difficult test to most people, I expect that those with well-preserved high frequency hearing (>10kHz) might still be able to hear a difference between the two DACs due to differences in their frequency response. Note that FiiO Taishan D03K has a slight sub-LF loss and a small top-octave peak that should result in a slightly brighter sound than the Topping E50, which could then be used to identify the "X".
So far that indeed seems to be the case, as most participants seem to be struggling to reliably hear a difference between the files, but very few (like yourself) are able to do better.
So the measurements, and the science behind them, seem to be nicely predictive of the outcome in this test :)
So i can distingish them easily, as i distinguished that fiio and e50, im not listening to db's or measurments, im listenings by ears and brain to semi-tones of the music provided by the system. these db say nothing for me, and to almost all people - as we can see from this test.
Similarly, the established science suggests that listeners will typically prefer slightly louder sounds, even when the difference is small enough not to perceive them as louder directly. Since both Topping D90SE and VMV D1se in 5V mode have slightly higher output levels than Topping E50, I don't doubt that most people would hear the difference and prefer either of them over E50 in a non-level-matched blind test. So that is also not really conflicting with measurements or science.

However in a blind ABX test with precise level matching I sincerely doubt anyone would be able to hear any difference between e.g. E50 and D90SE or D1se. It would perhaps be another interesting test to some, but unfortunately I don't own either of those two DACs so sadly can't prepare it. Perhaps someone else will :)
 

tonycollinet

Major Contributor
Joined
Sep 4, 2021
Messages
3,551
Likes
5,452
Location
UK/Cheshire
... I know this forum don't like statements like that, so...
It's not that the forum doesn't like them. It is just that they are meaningless except to the one person who has listened.

You can hear a difference - great. But the measurements tell us that the reason you hear a difference is very unlikely to be from the audio waves arriving at your ears. So what you hear tells us nothing about what anyone else will hear.
 

Volikovvv

Member
Joined
Nov 6, 2020
Messages
62
Likes
31
Since both Topping D90SE and VMV D1se
I compared d90 first revision non-mqa, so it is 4v. Thank you for detailed response, glad to hear this arguments. To be honest i did not read the first post, I'm just went through this test without reading, because there is a lot of technical info that im not interested in:)
 

Volikovvv

Member
Joined
Nov 6, 2020
Messages
62
Likes
31
But the measurements tell us that the reason you hear a difference
Maybe, but in this case looks a little bit strange for me why su-9n and d1se are so different in sound perception to me, difference is even bigger than d90 non mqa and d1se. I cant explain this by measurements which posted on this forum, maybe i lack some knowledge, don't know. When i noticed that my listening experience does not correlate with measurements i stopped to look to them.
So i fully support scientific approach, im an software engineer, but i dont understand why measurements does not correlate with what im hear? Of course difference are small in general, but big enough personally for me, and important to move me to the music nirvana.
 
OP
dominikz

dominikz

Addicted to Fun and Learning
Forum Donor
Joined
Oct 10, 2020
Messages
580
Likes
1,995
dont understand why measurements does not correlate with what im hear?
im an software engineer
It would be difficult to determine this remotely without having much deeper insights into the details of your test setup.
Somewhat like beeing asked for a root cause analysis of a reported SW issue without access to neither relevant logs nor source code - based on just a short, non-technical, end-user trouble report :)
There are various things that can mess-up uncontrolled subjective listening test results. :)
 

tonycollinet

Major Contributor
Joined
Sep 4, 2021
Messages
3,551
Likes
5,452
Location
UK/Cheshire
Maybe, but in this case looks a little bit strange for me why su-9n and d1se are so different in sound perception to me, difference is even bigger than d90 non mqa and d1se. I cant explain this by measurements which posted on this forum, maybe i lack some knowledge, don't know. When i noticed that my listening experience does not correlate with measurements i stopped to look to them.
So i fully support scientific approach, im an software engineer, but i dont understand why measurements does not correlate with what im hear? Of course difference are small in general, but big enough personally for me, and important to move me to the music nirvana.
Most likely what you hear is being generated in the wetware between your ears. I'd be very surprised if you still heard it under properly controlled blind test conditions. Most do not.

Look up confirmation bias etc.
 

audio2design

Major Contributor
Joined
Nov 29, 2020
Messages
1,769
Likes
1,793
I think if the test was performed with this fiio and more expensive dacs like d90 or d1se, i think the difference will be more noticeable for more people. I evaluated 9 dacs last year in 120-800$ price range, so from what i noticed, the difference between under 300$ dacs is small, that is what all people can hear from this ABX test. Going further to next tier range - make more people to pass this test. This is only my opinion, of course it can differ from others:)

I would put money on that not being the case.
 

yanm

Member
Forum Donor
Joined
Feb 10, 2021
Messages
65
Likes
52
Location
Switzerland
My theory is it might be due to Topping D90SE and VMV D1se '5V mode' which results in ~1,5dB higher output vs e.g. Topping E50 and similar balanced DACs (with their usual 4Vrms output).
+1,5dB may not seem louder when listening, but would sound different and definitely result in clear preference for the objectively louder unit in a non-level-matched test in most cases. This is why it is very important to measure and precisely level-match DAC outputs before listening. :)
Thanks, makes sense. I didn’t know about the difference in full scale voltage. It shows that choosing DAC by the ear is really tricky. Toole quite specifically wrote that a few dB louder music is indeed subjectively more pleasing. Hmm, I guess that’s also why some new releases are mastered so hot (for example, Adele 30 and 25 not to mention them).
 
OP
dominikz

dominikz

Addicted to Fun and Learning
Forum Donor
Joined
Oct 10, 2020
Messages
580
Likes
1,995
1.5dB (~15%) louder is clearly perceived as louder, at least I do. Otherwise I would look for a new hobby ;-)
That's great, but I'm not sure we can generalize that any and all listeners would easily identify a 1,5dB difference it as a pure level mismatch (rather than a tonality or sound quality difference) - as it is not a huge level difference perceptually.
My intention was just to warn readers about the pitfalls of doing listening comparisons without measuring and matching output levels. :)
 

btom

Member
Joined
Jan 11, 2019
Messages
7
Likes
5
12/4
Funnily enough first i did it 2 days ago at 11pm and i got 4 correct 12 incorrect,now not tired and a little louder the complete opposite.The bass and the highs are different/cleaner,at least thats what i try to focus on and i think i can hear it.I am 32yo my hearing is good for around 16,5-17khz(i can hear 17khz but barely)

Using Edifier S3000Pro USB input and monitor DSP.Treble and bass stock settings.Can't find anything about specs just that Signal-to-noise ratio is ≥85dB(A) soo i shouldn't be able to hear it?Maybe i don't considering the opposite results 2 days apart.
 
OP
dominikz

dominikz

Addicted to Fun and Learning
Forum Donor
Joined
Oct 10, 2020
Messages
580
Likes
1,995
12/4
Funnily enough first i did it 2 days ago at 11pm and i got 4 correct 12 incorrect,now not tired and a little louder the complete opposite.The bass and the highs are different/cleaner,at least thats what i try to focus on and i think i can hear it.I am 32yo my hearing is good for around 16,5-17khz(i can hear 17khz but barely)
Good work! 12 right out of 16 trials gives p-value 3,841%, which is better than the <5% condition, but doesn't satisfy the more strict <1% condition.
Using Edifier S3000Pro USB input and monitor DSP.Treble and bass stock settings.Can't find anything about specs just that Signal-to-noise ratio is ≥85dB(A) soo i shouldn't be able to hear it?
The relatively low SNR of your Edifier digital input would mask the SINAD differences (which are IMHO anyway probably less important for discrimination here), but it won't mask the significant measurable frequency response difference between the two DACs - so it should still be entirely possible to detect the differences.
Maybe i don't considering the opposite results 2 days apart.
IMO the main takeaway should hopefully be that the audible difference is definitely not 'night and day' - even if both the price and measurable differences are very significant. :) Hope you found the test interesting!
 
Last edited:

btom

Member
Joined
Jan 11, 2019
Messages
7
Likes
5
Good work! 12 right out of 16 trials gives p-value 3,841%, which is better than the <5% condition, but doesn't satisfy the more strict <1% condition.

The relatively low SNR of your Edifier digital input would mask the SINAD differences (which are IMHO anyway probably less important for discrimination here), but it won't mask the significant measurable frequency response difference between the two DACs - so it should still be entirely possible to detect the differences.

IMO the main takeaway should hopefully be that the audible difference is definitely not 'night and day' - even if both the price and measurable differences are very significant. :) Hope you found the test interesting!
Thanks,yeah there is no night and day difference at least on my system.It was def interesting test,if i listened to it with 5-10 sec stop there is no way i could tell.
 

solderdude

Grand Contributor
Joined
Jul 21, 2018
Messages
12,993
Likes
29,294
Location
The Neitherlands
I just flipped a coin 16 times.
6x heads, 10x tails

Also threw dice 16x and counted odd/even and did this 4 times
1: 7 even, 9 odd
2: 9 even, 7 odd
3: 10 even, 6 odd
4: 12 even, 4 odd
 
Top Bottom