• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Review and Measurements of Benchmark DAC3

OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,376
Likes
234,551
Location
Seattle Area
It is important to figure out what went wrong with your tests. The results you posted are very strange and do not agree with the results obtain by us or by other reviewers.
John, as I wrote to you in private, you have an extremely positive review and set of measurements from me. From the original post:

Conclusions
The Benchmark DAC3 as expected is a state-of-the-art digital to analog converter. Other than one set of noise spikes in jitter test, the rest of the measurements show exceptional performance. No glaring faults are seen at all. Its higher output level can be useful in room EQ applications to boot.

So of course the DAC3 goes on my recommended list.


No way any process mistake allow you to get such superlative results and comments from me. The audio stack in non-bit-exact mode would have rendered the results similar to the cheapest $10 DAC. So there is little inconsistent between my review and others that have given you positive marks.

Yes, there are some sideband jitters. These show up in both balanced and unbalanced input, with USB out of the computer and Toslink out of Audio Precision APx555. The only explanation seeing how Jude doesn't see them is sample to sample variation. I don't know why you are not considering that, instead of assuming there must a methodology difference.

I know we don't know each other so the first reaction is to assume incompetence on my part. That's OK, even my own senior members think that every day of the week and twice on Sunday. :D But some things are unlikely and this is one of them. Of course if there is an error, I will correct but we can't get there by having Jude test some other unit, or look at your results with yet another unit and different methodology to boot. As I explained, I own a System 2 analyzer and am very familiar with how you ran that test. And no, I am not using bandpass filtering in my FFT (that is optional).

Regardless, at nearly -120 dB, this is not a concern. The question members have is around unbalanced performance. Can you explain to us what is going on there?
 

Thomas savage

Grand Contributor
The Watchman
Forum Donor
Joined
Feb 24, 2016
Messages
10,260
Likes
16,298
Location
uk, taunton
If there was something unique in amirs test setup that was causing these side bands would we not be expecting to see them present in other DACs tested?

I think we can rule out the driver being a issue as they show up on toslink and I belive that comes straight out of the AP.

It does not seem that complicated of a setup , just what is there that could be so wrong but cause such a tiny issue while leaving otherwise impressive measurement results .

It would be nice to test another one, maybe one that’s a known quantity.
 

gvl

Major Contributor
Joined
Mar 16, 2018
Messages
3,427
Likes
3,982
Location
SoCal
Hmm, maybe a heat gun? Sorry, couldn't resist.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,376
Likes
234,551
Location
Seattle Area
Hmm, maybe a heat gun? Sorry, couldn't resist.
I have graduated to this now:
180201123113-flamethrowers-780x439.jpg
 

John_Siau

Active Member
Technical Expert
Audio Company
Joined
Jul 17, 2018
Messages
184
Likes
1,407
Location
Syracuse, NY USA
John, as I wrote to you in private, you have an extremely positive review and set of measurements from me. From the original post:

Conclusions
The Benchmark DAC3 as expected is a state-of-the-art digital to analog converter. Other than one set of noise spikes in jitter test, the rest of the measurements show exceptional performance. No glaring faults are seen at all. Its higher output level can be useful in room EQ applications to boot.

So of course the DAC3 goes on my recommended list.

Yes, there are some sideband jitters. These show up in both balanced and unbalanced input, with USB out of the computer and Toslink out of Audio Precision APx555. The only explanation seeing how Jude doesn't see them is sample to sample variation. I don't know why you are not considering that, instead of assuming there must a methodology difference.
This is not sample-to-sample variation because every unit the leaves the factory must pass the following test (which is nearly identical to the test that you ran). This means that the unit has suffered some sort of failure after leaving the factory, or there is something wrong with the test methodology. The fact that Jude could not replicate your results also points to one of these two possibilities.

Jude's DAC3 test results:

https://www.head-fi.org/threads/schiit-yggdrasil-impressions-thread.766347/page-591#post-14366380

It looks like the most likely cause is a component failure and not a methodology problem. We are sending you a new unit from the factory that you can test so that we can get to the bottom of this.

Every DAC3 must pass the following test before leaving the factory. This is test 13 out of a 20-step comprehensive performance test. The gray line is the pass/fail. Typical results are shown with the red and magenta traces (L&R channels). There is very little unit-to-unit variation in this test.

DAC3_Final_Test_Step13.JPG
 

dallasjustice

Major Contributor
Joined
Feb 28, 2016
Messages
1,270
Likes
907
Location
Dallas, Texas
This is not sample-to-sample variation because every unit the leaves the factory must pass the following test (which is nearly identical to the test that you ran). This means that the unit has suffered some sort of failure after leaving the factory, or there is something wrong with the test methodology. The fact that Jude could not replicate your results also points to one of these two possibilities.

Jude's DAC3 test results:

https://www.head-fi.org/threads/schiit-yggdrasil-impressions-thread.766347/page-591#post-14366380

It looks like the most likely cause is a component failure and not a methodology problem. We are sending you a new unit from the factory that you can test so that we can get to the bottom of this.

Every DAC3 must pass the following test before leaving the factory. This is test 13 out of a 20-step comprehensive performance test. The gray line is the pass/fail. Typical results are shown with the red and magenta traces (L&R channels). There is very little unit-to-unit variation in this test.

View attachment 14033
The DAC3 @amirm tested was mine. Should I wait until we see the result of the new test for me to send it back to you?
 

jtwrace

Major Contributor
Forum Donor
Joined
May 31, 2017
Messages
1,225
Likes
1,401
Location
Orlando, FL
@John_Siau

How about you make an 8 ch remote volume controlled dac that can either work with a streamer or even better make it Roon Ready for Ethernet input. All XLR output too!
 

John_Siau

Active Member
Technical Expert
Audio Company
Joined
Jul 17, 2018
Messages
184
Likes
1,407
Location
Syracuse, NY USA
Here is the unbalanced tests from Benchmark DAC3. It is not the full suite due to shortness of time but should give you a flavor of what is going on.


View attachment 13388

While you can't see the linearity error itself, you can easily see the varying levels in the output of Benchmark DAC3 on the right. Clearly one or the other channel output is in error and that is what the Linearity measurement showed. The Topping on the other hand, most of the time produces identical output (once in a while one channel would be a bit lower).

As always, comments, correction and criticism are welcome.
Amirm,

This is not a normal test result for a DAC3. This unit would not pass final test if these results are correct. Given that this unit also produced sidebands, I suspect it has a problem.
 
D

Deleted member 65

Guest
Kudos to John Siau/Benchmark, setting an example for others to follow. Next DAC & Amp (I change a lot) purchase I'll have Benchmark on top of my list!
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,376
Likes
234,551
Location
Seattle Area
A loaned unit is being kindly sent from Benchmark, coming next week. We may take a brief trip the same time so it may be end of next week for new results.
 

etc6849

Active Member
Joined
Apr 28, 2018
Messages
132
Likes
241
Location
Columbia, SC
Why not try one of these?: https://xilica.com/products/xd/

They are made for speaker processing, but no reason you can't just hook up 4 AES digital inputs into the 8x8 model (or use two 4x8 models). They sound fantastic for the money. I use three of them in my theater to tri-amp 5 speakers and time align/level match 5 subs.

The specs aren't as good as the Benchmark DAC3 I have (SNR unweighted is 115dB versus 126dB), but they have a lot of versatility and include a 40-bit DSP that can even do FIR filtering. Tri-amped the sound is way better than any AVR or home theater processor could ever be anyways.

The Emotiva XMC-1 they replaced has these specs (and this is supposed to be one of the best processors you can get for under $10k):
Digital Input to Balanced Analog Output (HDMI PCM):
  • THD: < 0.00025% @ 1 kHz.
  • THD: < 0.002% (20 Hz to 20 kHz).
  • S/N ratio (all main channels): > 110 dB (A weighted).
  • S/N ratio (subwoofer channels) > 100 dB (A weighted).
You absolutely need a DSP (not just a DAC) once you get more than 2 channels if you want things to sound right. No reason software can't do this for you though. If I had more money I'd look into software DSP + 10 Benchmark DAC3's though :D

How about you make an 8 ch remote volume controlled dac that can either work with a streamer or even better make it Roon Ready for Ethernet input. All XLR output too!
 

HifiGuy

Member
Joined
Jun 25, 2018
Messages
20
Likes
4
Why not try one of these?: https://xilica.com/products/xd/

They are made for speaker processing, but no reason you can't just hook up 4 AES digital inputs into the 8x8 model (or use two 4x8 models). They sound fantastic for the money. I use three of them in my theater to tri-amp 5 speakers and time align/level match 5 subs.

The specs aren't as good as the Benchmark DAC3 I have (SNR unweighted is 115dB versus 126dB), but they have a lot of versatility and include a 40-bit DSP that can even do FIR filtering. Tri-amped the sound is way better than any AVR or home theater processor could ever be anyways.

The Emotiva XMC-1 they replaced has these specs (and this is supposed to be one of the best processors you can get for under $10k):
Digital Input to Balanced Analog Output (HDMI PCM):
  • THD: < 0.00025% @ 1 kHz.
  • THD: < 0.002% (20 Hz to 20 kHz).
  • S/N ratio (all main channels): > 110 dB (A weighted).
  • S/N ratio (subwoofer channels) > 100 dB (A weighted).
You absolutely need a DSP (not just a DAC) once you get more than 2 channels if you want things to sound right. No reason software can't do this for you though. If I had more money I'd look into software DSP + 10 Benchmark DAC3's though :D
It seems you are offtopic
 

John_Siau

Active Member
Technical Expert
Audio Company
Joined
Jul 17, 2018
Messages
184
Likes
1,407
Location
Syracuse, NY USA
LinearityDAC3_UbalancedandBalanced.JPG

The above is a linearity measurement on a Benchmark DAC3. This plot includes the unbalanced outputs (green trace - left, red trace - right) and the balanced outputs (yellow trace = left, magenta trace = right). The blue horizontal lines represent a linearity deviation of +/- 0.1 bit (the deviation used by Amirm in his tests). The cyan horizontal lines represent a linearity deviation of +/- 0.5 bit (+/- 3.01 dB). The raw data is plotted in the 4 diagonal curves. The upper pair of curves are the balanced outputs. The balanced outputs are 16 dB hotter than the unbalanced outputs (+24 dBu at 0 dBFS vs. +8.2 dBu at 0 dBFS).

At the bottom of these diagonal curves, we reach the noise floor of our measurements. This noise floor is a function of the output noise of the output, the input noise of the analyser, and the bandwidth of the bandpass filter in the measurement system. A narrower bandpass filter allows us to probe deeper into the noise to measure the linearity at lower levels. The lower operating levels of the RCA outputs make the measurements more difficult in comparison to measuring the balanced outputs. This can make it look like the XLR outputs have better linearity, but this is not the case. Inside the DAC3, both sets of outputs are derived from the same analog output of a differential amplifier that follows the ES9028PRO D/A converter. This means that the apparent differences in linearity shown in Amirm's linearity measurements of the DAC3 are a physical impossibility. This does not mean that his measurements were wrong, it just means that noise was interfering with his measurements. The solution is to use a narrower bandpass filter.

There are 4 different bandpass filters that can be used in the AP2522 that I used to make this measurement. Similar options are available in the AP2722 and the APx555.

1) Analog bandpass filter in the 'analog analyzer'
2) Digital bandpass filter in the 'DSP audio analyzer'
3) Digital bandpass filter in the 'Harmonic Distortion Analyzer' using the 'Fundamental Amplitude' measurement
4) FFT analysis using the 'FFT spectrum analyzer'

Note: I do not have an APx555 but I have the other two models. Amirm and I both have AP2522 analyzers, so I chose to use this box rather than the higher performance AP2722.

This above list is sorted by widest to narrowest filter. The best low-level linearity measurements can be made with the FFT spectrum analyzer but this is very slow and very cumbersome. There is also no automatic way to make a traditional linearity plot after the FFT measurements have been done on an Audio Precision test station. I have done very low-level high-precision FFT linearity measurements, but it can take hours to complete the tests. What these FFT measurement show is that most sigma-delta converters have virtually perfect linearity. For this reason, linearity measurements tend to be completely useless when measuring sigma delta converters. The deviations at the bottom of the curve are almost always due to the fact that the curve hits the noise measurement limit of the test and have absolutely nothing to do with the actual linearity of the converter. This can be proven by increasing the discrimination of the test by narrowing the bandpass filter so that accurate measurements can be made further down into the noise. At the end of the day, these efforts tend to prove that most sigma-delta converters have virtually-perfect linearity.

For the test above, I used method 3 ('Fundamental Amplitude' meter in the 'Harmonic Distortion Analyzer'). This method allows us to probe deep into the noise floor to examine the linearity. As I said above, the FFT technique (option 4) will allow a deeper analysis, but the results are not automatically plotted into a linearity curve.

On the DAC3, the unbalanced outputs operate at a signal level that is 16 dB lower than the balanced outputs. This means that it is much harder to measure the linearity of the unbalanced outputs. In terms of bits, 16 dB is about 2.6 bits. If our measurement shows that the 'linearity' is 2.6 bits 'better' on the XLR outputs, this is an indication that the measurement is being limited by the capabilities of our test equipment. In the test above, you can see that the 16 dB separation between the balanced and unbalanced curves is reduced to about 12 dB where at the bottom of the curves where we hit the noise limitations of our measurement. This means that the noise limitations will make the unbalanced outputs look about 4 dB (0.7 bits) worse than the unbalanced. We are almost at the point where our test equipment is not a limitation when comparing the XLR and RCA outputs.

So lets talk about the results:

Using the XLR outputs, we hit the noise limitations of our measurement at -150 dB relative to 24 dBu (right-hand scale) which is -150 dBFS (equivalent to about -25 bits). This means that the 'linearity' curve will reach a 1 dB deviation about 12 dB (2 bits) above this point (and this is clearly visible as a 1 dB deviation at -23 bits). This bend in the XLR 'linearity' curve starting at -19 bits and reaching 1 dB at -23 bits is due to noise and does not have anything to do with linearity.

Using the RCA outputs, we hit the noise limitations of our measurements at about -162 dB relative to 24 dBu which is -146 dB relative to the 8 dBu output of the RCA outputs at 0 dBFS. This means that we hit the noise limits of out measurement at -146 dBFS on the unbalanced outputs. As stated above, our measurements are degraded by 4 dB when measuring the unbalanced outputs. Doing the same calculations that we did on the XLR outputs, we would expect the 1 dB deviation to occur at a level that is 4 dB higher (about 0.7 bits higher). So the RCA outputs should reach a 1 dB deviation at -22.3 bits instead of the -23 bits that we measured with the XLR outputs). If we examine the linearity plots for the RCA outputs, we can see that we reach a 1 dB deviation at about -22 bits. This is the expected result. Again, the deviation is due to noise and does not have anything to do with linearity.

In the context of sigma-delta converters, 'linearity' measurements are simply a reflection of the SNR of the output being measured and the SNR of the detector in the analyzer. It is much better to measure the SNR of the output directly than it is to determine noise performance through the very convoluted use of a linearity plot.

The sole purpose of the linearity plot should be to verify that there are no linearity errors above the point at which noise begins to corrupt the measurement. From the tests above, we can see that the linearity is perfect above the point at which noise corrupts the measurement. Once we are within 24 dB above the noise floor of the measurements, (-150 dBFS for the XLR outputs) and (-146 dBFS for the RCA outputs) the linearity deviation is meaningless and will vary according to the noise discrimination of the test equipment . For this reason it is also meaningless to say that an output has 'x bits' of resolution on the basis of a linearity measurement.

Bottom line:

Use linearity measurements to linearity above the noise floor of the measurement, and then use a simple noise meter to determine the SNR of the output. The SNR can be expressed in dB and in bits of resolution measured over the entire 20 kHz bandwidth. This is the true resolution in terms of bits (as long as no linearity deviations are discovered at levels above the noise floor of the linearity test).

Sorry for the length of this post!

Edit: changed 'Once we are 12 dB above the noise floor of the measurements ...' to read '... 24 dB'. At 12 dB above the noise floor, the noise causes a +/- 1 dB deviation in the linearity curve. At 24 dB above the noise floor, the noise causes about a +/- 0.1 dB deviation in the linearity curve. We have to stay out of this noise-contaminated region when evaluating linearity. Improved test equipment allows us to push the noise contaminated region lower thereby extending our ability to look at the linearity. But once the noise contaminates the measurement, it is no longer a measurement of linearity.
 
Last edited:

mindbomb

Active Member
Joined
Nov 11, 2017
Messages
284
Likes
175
@John_Siau

A lot of companies who are using ESS dacs have a dramatic rise in smpte imd at around -25dbfs, however the benchmark dac 3 does not have this issue. Can you share any insight into what those companies are doing wrong?
 
Last edited:

John_Siau

Active Member
Technical Expert
Audio Company
Joined
Jul 17, 2018
Messages
184
Likes
1,407
Location
Syracuse, NY USA
@John_Siau

A lot of companies who are using ESS dacs have a dramatic rise in smpte imd at around -25dbfs, however the benchmark dac 3 does not have this issue. Can you share any insight into what those companies are doing wrong?
Absolutely!

This IMD (and THD) is a direct result of omitting a differential amplifier after the output of the chip. Like most high-quality D/A converters, the ESS chip outputs are balanced. Many designers think it is OK to connect these directly to the outputs on the back of the unit. Typically the balanced connection from the chip gets buffered and connected to pins 2 and 3 of the XLR jack. Often, the RCA output is directly connected to pin 2 of the XLR jack and consequently, the RCA output only sees one side of the converter's balanced output. If you take the XLR output and connect it to the balanced input on an Audio Precision test station, the IMD will not show up. But if you connect the RCA output to the same analyzer, the IMD is present.

What is happening??

All D/A converter chips produce significant common-mode distortion on the balanced outputs. This unwanted common-mode distortion is easily removed with a differential amplifier. If it is not removed, the result is IMD.

The AP test stations have transformer-coupled inputs that provide excellent common mode rejection. This is equivalent to the function of a differential amplifier. The transformer-coupled input on the analyzer removes the common-mode distortion at the output of the XLR and everything looks good, but it really isn't. Most XLR inputs do not have the common-mode rejection ratio that is provided by the inputs on the AP test station. More importantly, the RCA outputs on these boxes have high IMD and there is no way to remove it downstream.

How to eliminate the IMD:

The solution is to place a well-trimmed differential amplifier at the balanced outputs of the D/A chip. This removes the IMD while producing an unbalanced output that can be buffered and sent to the RCA outputs. It can also be re-balanced and sent to the XLR outputs. The is the correct way to provided balanced XLR outputs from a D/A converter. The transition from balanced to unbalanced and back to balanced is necessary. This is what removes the common-mode distortion.

We often provide a balanced pair of differential amplifiers so that the we have a balanced output from the differential amplifier pair. This technique is used in our new HPA4 line/headphone amplifier.
 
Top Bottom