- Joined
- Oct 10, 2020
- Messages
- 870
- Likes
- 2,784
This thread is meant to once and for all settle the age-old question on whether different non-broken DACs have a "sound" or if all DACs sound the same!
...just kidding - those debates will probably never end.
Let me start by providing the link to the ABX test.
Note: There are 16 trials in the test.
[EDIT 2024-09-17] Latest overview of participant test results can be found in post #196.
[EDIT 2022-01-17] Note that since test files are recorded in 44,1kHz sample rate it is recommended to set your audio output device sample rate to 44,1kHz as well to avoid resampling.
[EDIT 2021-12-29] For those wanting to use foobar2000 (in WASAPI exclusive mode) with ABX comparator plugin (16 trials suggested), here are direct links to the audio test files:
1. Topping E50 ABX sample file
2. FiiO Taishan D03K ABX sample file
If you do a test please report your results (copy/paste of ABX comparator result output) by posting in this thread or via a private message. Thanks!
Details of the test are provided below.
Introduction
Anyway, since I recently got a Topping E50 DAC and E1DA Cosmos ADC (both SOTA converters) I thought it might be interesting to prepare a controlled blind test between a SOTA DAC and one that is a relatively low-performing unit by todays standards (but not so much that one would call it "broken"). The ideas is to see if there are any audible differences in such an extreme comparison.
Many people might expect that this would be an easy test (given the price and spec difference), so let's start with that before trying anything else
Title states "Phase 1" because I plan to do a more difficult follow-up (E50 vs Babyface DAC) if results of this test show that several people can reliably tell the difference in this 'simple' test.
DACs under test
Meet the contenders:
Test equipment, SW and test track
The test track is the original 44,1kHz/24bit digital master wav file of the song "Farewell to Arms" (link to full song, available on various streaming services).
The song was mainly selected because I have the distribution rights and master files for it (shameless self-promotion alert! ).
The ADC used to record both DACs was the E1DA Cosmos ADC (full AP measurements available here) which, similarly to Topping E50, achieves measurement-equipment-grade conversion performance (SINAD ~120dB).
Head'n'HiFi Objective2 (O2) headphone amp in unity (1x) gain setting was used as an impedance buffer when recording the D03K - more on this below.
With both E50 and D03K the optical input was used to feed the DAC, and the source was the RME Babyface Silver Edition (1st gen) soundcard.
PreSonus Studio One 5 Professional was used to record the files. The project was configured to use the same 44,1kHz sample rate as the source track, and 32bit bit-depth to avoid any chance of loss of data.
ASIO4All driver was used so that the RME soundcard and E1DA Cosmos ADC USB devices can be used together as a single device for measurements and recording. 44,1kHz sample rate was used in all cases, to be consistent with the original recording and to avoid any resampling.
The online ABX test is constructed with the amazing abxtests.com web-based tool by @jaakkopasanen (see related thread). Thanks to @jaakkopasanen for building and providing this resource to the community!
Test file recording and preparation
In principle the concept is that of a simple loopback test - i.e. the output of both DACs was (separately) recorded by the same ADC, and resulting files level matched to better than 0,1dB accuracy.
Tone generator plugin (set to generate a -1dBFS 1kHz tone) was added to the track before the source file and used to calibrate the levels of both DACs when recording, and later to fine-tune the levels of the resulting recordings.
The recording chain was:
1) RME Babyface optical out -> Topping E50 optical in then balanced L/R out -> E1DA Cosmos ADC (set to 4,5V sensitivity) balanced L/R in
With this setup -1dBFS sine output results in -2,1dBFS input (-1,1dB loss in level).
Here's a short snip of the performance in this setup (used to reference with independent measurements and to make sure that the recording setup is correct):
We see THD+N is very low (at -115dB vs fundamental at 1kHz) and pretty close to maximum performance. Around 2dB is lost due to selected measurement input level, and additional ~3dB should be lost due to both ADC and DAC having similar SNRs and the noise summing in loopback.
2) RME Babyface optical out -> FiiO Taishan D03K optical in, then unbalanced RCA L/R out -> Head'n'Hifi O2 in 1x/unity gain (impedance buffer) -> E1DA Cosmos ADC (set to 1,7V sensitivity) unbalanced L/R input
With this setup -1dBFS sine output results in -3dBFS input (-2dB loss in level).
Note: E1DA Cosmos ADC has no input buffer and at 1,7V sensitivity has a very low input impedance (around 450 Ohms in unbalanced mode). If connected directly to D03K output this low input impedance is loading down the DAC's output, decreasing the level and increasing the distortion significantly. By using the O2 as buffer (1x gain with maximum volume pot position results in unity gain between input and output) its high input impedance and low output impedance allows for optimum impedance matching between the two devices. Since performance of O2 is much better than that of D03K there is no loss in signal transfer quality due to this.
Here's a short snip of the performance in this setup (used to reference with independent measurements and to make sure that the recording setup is correct):
Since D03K is not a very high performing unit (much worse performing than the rest of the measurement chain), this is fully in line with previous measurements, as well as the manufacturer's specification.
From the above diagrams we see that the total loopback recording chain 1kHz SINAD is:
Lastly let's compare the measured performance of the two DACs on FR and THD:
E50 FR is perfectly flat, while D03K has a ~0,3dB loss at 20Hz, and a ~0,7dB peak at 17,5kHz.
THD of E50 is more uniform across frequencies and >28dB better than that of D03K.
As we saw before, there is an overall ~0,9dB level difference in the raw recorded files, due to different analogue output levels of the two DACs and fixed sensitivity settings of the ADC.
To further fine-match the volume, Presonus Mixtool plugin (allowing gain adjustments to 0,01 dB precision) was used to tune the 1kHz tone level to maximum precision. Resulting files are therefore matched to significantly better than required 0,1dB (at 1kHz at least, note slight FR differences above).
Discussion and conclusion
As you can see, care was taken to achieve maximum performance from both DACs when preparing the test and to achieve very good level matching in all stages. This, combined with the blinded ABX test approach, ensures a controlled subjective comparison of audible differences between the two DACs.
Note however that an ABX test by itself will not tell us which DAC sounds 'better' (preference) - it will just tell us if there's any audible difference between them at all. This is still an important first step before it makes sense to investigate preference at all.
Though I consider this to be a difficult test to most people, I expect that those with well-preserved high frequency hearing (>10kHz) might still be able to hear a difference between the two DACs due to differences in their frequency response. Note that FiiO Taishan D03K has a slight sub-LF loss and a small top-octave peak that should result in a slightly brighter sound than the Topping E50, which could then be used to identify the "X".
I will wait for a week or so before posting preliminary result summary (unless a lot of people complete the test sooner and we have some significant results quickly). Anyone completing the test is of course free to post their results and impressions here at any time
If no listeners are able to reliably differentiate these two DACs I will not proceed to prepare the more difficult 'phase 2' test.
However if there are several listeners that can differentiate between these two DACs reliably, I will prepare the a similar comparison between the RME Babyface 1st gen DAC (flat FR, SINAD >100dB), representing average but solid DAC performance, and Topping E50 (flat FR, SINAD ~120dB) representing SOTA DAC performance.
Lastly, let me link again to the ABX test.
Note: There are 16 trials in the test.
[EDIT 2022-01-17] Note that since test files are recorded in 44,1kHz sample rate it is recommended to set your audio output device sample rate to 44,1kHz as well to avoid resampling.
[EDIT 2021-12-29] For those wanting to use foobar2000 (in WASAPI exclusive mode) with ABX comparator plugin (16 trials suggested), here are direct links to the audio test files:
1. Topping E50 ABX sample file
2. FiiO Taishan D03K ABX sample file
If you do a test please report your results (copy/paste of ABX comparator result output) by posting in this thread or via a private message. Thanks!
Enjoy!
[EDIT 2022-01-17] You can find an initial overview of results in post #69 (up to 01.01.2022.) and the latest one in post #130 (up to 17.01.2022.).
[EDIT 2022-02-18] Updated overview of results can be found in post #139.
[EDIT 2022-03-06] Updated overview of results can be found in post #163.
[EDIT 2022-05-15] Updated overview of results can be found in post #169.
[EDIT 2022-10-09] Updated overview of results can be found in post #180.
[EDIT 2024-09-17] Updated overview of results can be found in post #196.
...just kidding - those debates will probably never end.
Let me start by providing the link to the ABX test.
Note: There are 16 trials in the test.
[EDIT 2024-09-17] Latest overview of participant test results can be found in post #196.
[EDIT 2022-01-17] Note that since test files are recorded in 44,1kHz sample rate it is recommended to set your audio output device sample rate to 44,1kHz as well to avoid resampling.
[EDIT 2021-12-29] For those wanting to use foobar2000 (in WASAPI exclusive mode) with ABX comparator plugin (16 trials suggested), here are direct links to the audio test files:
1. Topping E50 ABX sample file
2. FiiO Taishan D03K ABX sample file
If you do a test please report your results (copy/paste of ABX comparator result output) by posting in this thread or via a private message. Thanks!
Details of the test are provided below.
Introduction
Anyway, since I recently got a Topping E50 DAC and E1DA Cosmos ADC (both SOTA converters) I thought it might be interesting to prepare a controlled blind test between a SOTA DAC and one that is a relatively low-performing unit by todays standards (but not so much that one would call it "broken"). The ideas is to see if there are any audible differences in such an extreme comparison.
Many people might expect that this would be an easy test (given the price and spec difference), so let's start with that before trying anything else
Title states "Phase 1" because I plan to do a more difficult follow-up (E50 vs Babyface DAC) if results of this test show that several people can reliably tell the difference in this 'simple' test.
DACs under test
Meet the contenders:
- Topping E50
- Price: 269$
- SINAD: ~120dB
- Full measurements and review link
- FiiO Taishan D03K
- Price: 20$
- SINAD: ~91dB
- Full measurements link
Test equipment, SW and test track
The test track is the original 44,1kHz/24bit digital master wav file of the song "Farewell to Arms" (link to full song, available on various streaming services).
The song was mainly selected because I have the distribution rights and master files for it (shameless self-promotion alert! ).
The ADC used to record both DACs was the E1DA Cosmos ADC (full AP measurements available here) which, similarly to Topping E50, achieves measurement-equipment-grade conversion performance (SINAD ~120dB).
Head'n'HiFi Objective2 (O2) headphone amp in unity (1x) gain setting was used as an impedance buffer when recording the D03K - more on this below.
With both E50 and D03K the optical input was used to feed the DAC, and the source was the RME Babyface Silver Edition (1st gen) soundcard.
PreSonus Studio One 5 Professional was used to record the files. The project was configured to use the same 44,1kHz sample rate as the source track, and 32bit bit-depth to avoid any chance of loss of data.
ASIO4All driver was used so that the RME soundcard and E1DA Cosmos ADC USB devices can be used together as a single device for measurements and recording. 44,1kHz sample rate was used in all cases, to be consistent with the original recording and to avoid any resampling.
The online ABX test is constructed with the amazing abxtests.com web-based tool by @jaakkopasanen (see related thread). Thanks to @jaakkopasanen for building and providing this resource to the community!
Test file recording and preparation
In principle the concept is that of a simple loopback test - i.e. the output of both DACs was (separately) recorded by the same ADC, and resulting files level matched to better than 0,1dB accuracy.
Tone generator plugin (set to generate a -1dBFS 1kHz tone) was added to the track before the source file and used to calibrate the levels of both DACs when recording, and later to fine-tune the levels of the resulting recordings.
The recording chain was:
1) RME Babyface optical out -> Topping E50 optical in then balanced L/R out -> E1DA Cosmos ADC (set to 4,5V sensitivity) balanced L/R in
With this setup -1dBFS sine output results in -2,1dBFS input (-1,1dB loss in level).
Here's a short snip of the performance in this setup (used to reference with independent measurements and to make sure that the recording setup is correct):
We see THD+N is very low (at -115dB vs fundamental at 1kHz) and pretty close to maximum performance. Around 2dB is lost due to selected measurement input level, and additional ~3dB should be lost due to both ADC and DAC having similar SNRs and the noise summing in loopback.
2) RME Babyface optical out -> FiiO Taishan D03K optical in, then unbalanced RCA L/R out -> Head'n'Hifi O2 in 1x/unity gain (impedance buffer) -> E1DA Cosmos ADC (set to 1,7V sensitivity) unbalanced L/R input
With this setup -1dBFS sine output results in -3dBFS input (-2dB loss in level).
Note: E1DA Cosmos ADC has no input buffer and at 1,7V sensitivity has a very low input impedance (around 450 Ohms in unbalanced mode). If connected directly to D03K output this low input impedance is loading down the DAC's output, decreasing the level and increasing the distortion significantly. By using the O2 as buffer (1x gain with maximum volume pot position results in unity gain between input and output) its high input impedance and low output impedance allows for optimum impedance matching between the two devices. Since performance of O2 is much better than that of D03K there is no loss in signal transfer quality due to this.
Here's a short snip of the performance in this setup (used to reference with independent measurements and to make sure that the recording setup is correct):
Since D03K is not a very high performing unit (much worse performing than the rest of the measurement chain), this is fully in line with previous measurements, as well as the manufacturer's specification.
From the above diagrams we see that the total loopback recording chain 1kHz SINAD is:
- ~115 dB with Topping E50
- ~90 dB with FiiO Taishan D03K
Lastly let's compare the measured performance of the two DACs on FR and THD:
E50 FR is perfectly flat, while D03K has a ~0,3dB loss at 20Hz, and a ~0,7dB peak at 17,5kHz.
THD of E50 is more uniform across frequencies and >28dB better than that of D03K.
As we saw before, there is an overall ~0,9dB level difference in the raw recorded files, due to different analogue output levels of the two DACs and fixed sensitivity settings of the ADC.
To further fine-match the volume, Presonus Mixtool plugin (allowing gain adjustments to 0,01 dB precision) was used to tune the 1kHz tone level to maximum precision. Resulting files are therefore matched to significantly better than required 0,1dB (at 1kHz at least, note slight FR differences above).
Discussion and conclusion
As you can see, care was taken to achieve maximum performance from both DACs when preparing the test and to achieve very good level matching in all stages. This, combined with the blinded ABX test approach, ensures a controlled subjective comparison of audible differences between the two DACs.
Note however that an ABX test by itself will not tell us which DAC sounds 'better' (preference) - it will just tell us if there's any audible difference between them at all. This is still an important first step before it makes sense to investigate preference at all.
Though I consider this to be a difficult test to most people, I expect that those with well-preserved high frequency hearing (>10kHz) might still be able to hear a difference between the two DACs due to differences in their frequency response. Note that FiiO Taishan D03K has a slight sub-LF loss and a small top-octave peak that should result in a slightly brighter sound than the Topping E50, which could then be used to identify the "X".
I will wait for a week or so before posting preliminary result summary (unless a lot of people complete the test sooner and we have some significant results quickly). Anyone completing the test is of course free to post their results and impressions here at any time
If no listeners are able to reliably differentiate these two DACs I will not proceed to prepare the more difficult 'phase 2' test.
However if there are several listeners that can differentiate between these two DACs reliably, I will prepare the a similar comparison between the RME Babyface 1st gen DAC (flat FR, SINAD >100dB), representing average but solid DAC performance, and Topping E50 (flat FR, SINAD ~120dB) representing SOTA DAC performance.
Lastly, let me link again to the ABX test.
Note: There are 16 trials in the test.
[EDIT 2022-01-17] Note that since test files are recorded in 44,1kHz sample rate it is recommended to set your audio output device sample rate to 44,1kHz as well to avoid resampling.
[EDIT 2021-12-29] For those wanting to use foobar2000 (in WASAPI exclusive mode) with ABX comparator plugin (16 trials suggested), here are direct links to the audio test files:
1. Topping E50 ABX sample file
2. FiiO Taishan D03K ABX sample file
If you do a test please report your results (copy/paste of ABX comparator result output) by posting in this thread or via a private message. Thanks!
Enjoy!
[EDIT 2022-01-17] You can find an initial overview of results in post #69 (up to 01.01.2022.) and the latest one in post #130 (up to 17.01.2022.).
[EDIT 2022-02-18] Updated overview of results can be found in post #139.
[EDIT 2022-03-06] Updated overview of results can be found in post #163.
[EDIT 2022-05-15] Updated overview of results can be found in post #169.
[EDIT 2022-10-09] Updated overview of results can be found in post #180.
[EDIT 2024-09-17] Updated overview of results can be found in post #196.
Last edited: