Listening Tests - Benchmark DAC3 versus Topping D90LE

[MattHooper]

Personally -maybe I am blessed because I consistently picked the right gear- I haven't heard huge "obvious sonic differences" in many years. I hear slight ones, and then the hard part is to establish a personal preference between choices that are very close in performance.

And that just applies to my "music shrine system", which is where I may care about the ultimate 5% in SQ. i can equally enjoy my favorite music on the $500 system i have in my little nature cabin.

I should qualify perhaps - at least in my case "obvious" could be replaced with "distinct." In other words, the differences I seemed to hear in the AC cables were in the big picture quite subtle...but they seemed distinct, in terms of always identifiable. (Until I blind tested).

 

[Mr. Widget]

However - at the end of the day - we are talking (generally) minscule differences, that even if audible are so tiny we have to listen carefully while switching from one to the other. Sometimes we even have to be trained on what cues to listen for.

After all that, do we reallly care? I just want people to stop claiming that if people spend megabucks on stuff that makes zero or close to zero difference that they'll achieve audio nirvana.

Well, then there is that!

I currently have two systems that bring me audio nirvana... both systems would be considered stupid expensive in many people's eyes, but they each bring me great joy every time I power one of them up. At the end of the day getting to listen to our favorite music or watching exceptional images with sound that brings us joy is what this hobby is about.

Would I enjoy a $500 system as much? No. Did I enjoy a $500 system when I was 16? Yes.

 

[IAtaman]

While I respect measurements and use them myself professionally and as an enthusiast, I haven't see the conclusive correlation between human auditory perception and the measured performance of DACs and other electronics that measure well.

I am not even particularly concerned with 99th percentile human auditory perception. Whenever I ask to see the results of such a test, the ASR response is about proving a negative or some similar dismissive response.

I would like to see a test similar to Harman's work with loudspeakers on the the thresholds of audibility with various electronics. Unfortunately it is costly to run such a test and the industry so far has found it more profitable to sell IEC power cables and $10,000 DACs.

I hear you. It is good to keep an open mind and I try to do that myself as well.

I was also a bit annoyed with dismissive ASR responses when I became a bit more active here. But then again I imagined myself as a physics expert on an internet forum, trying to deal with an endless supply of skeptics that claim maybe there are 4 spatial dimensions and not 3, and if all the data was indeed conclusive, and whether enough research was conducted in this matter etc etc. I imagine I'd be a lot less kind after 5 years, so I am not complaining anymore.

And you know what, maybe there are indeed 4 spatial dimensions, there is something we have not observed that will change all our theories about space and time, who knows, maybe there are 11. What we know for sure however, is that it's not going to be the suspicion of an semi-knowledgable enthusiast that will be the end of the Standard Model.

All analog digital and digital analog conversion is supported by math theorems. If one would like to shake the world of established knowledge, a good way of doing that might be to go through the math and find a hole somewhere maybe or add to it. But expecting anecdotal evidence to change the course of an engineering field would be unlikely and maybe somewhat naive in my opinion.

On the topic of conclusiveness of the evidence, what convinced me was to get away from all this relatively complicated math of one-bit DACs. At its simplest, DAC is actually a very straightforward device. You want to convert a number to a voltage, so you can take the binary digits of that number, connect them to an op-amp via resistors that correctly factor for their significance to sum all those voltages up and voila - you have a DAC. You can model this on a computer with ideal components and you will find it works fine. Except the real world is not ideal, resistors vary, op-amps are not perfect, electrons are slippery etc. and all these imperfections introduce non-linearity. But we can measure that non linearity and can gauge how far we are away from mathematically ideal. That non-linearity is so low for well designed and built devices that measure transparent, I believe it is exceedingly unlikely for them to sound different. But if someone offers a mechanism as to why that might not be the case, I am all ears.

 

[thecheapseats]

<snip> Did I enjoy a $500 system when I was 16? Yes. <snip>

and that's the comment that i'll remember from this read... what got the ball rolling to begin with... thx man...

 

[Andreas007]

I appreciate the work done.
But to conclude anything from that result is ridiculous.
I‘m working in R&D of a major chip company and you would immediately embarras yourself if you would present a result with such a low confidence and simultaneously claim that you are on to something.

 

[RichB]

I appreciate the work done.
But to conclude anything from that result is ridiculous.
I‘m working in R&D of a major chip company and you would immediately embarras yourself if you would present a result with such a low confidence and simultaneously claim that you are on to something.

For the record, I nor any other on this thread have made this claim so I am not sure who this is directed at.

- Rich

 

[RichB]

I have updated the results table throughout this thread to use the simplified Correct/Incorrect table because the two-column table was confusing.
The original table remains as thumbnail to preserve continuity with subsequent posts.

- Rich

 

[Mr. Widget]

On the topic of conclusiveness of the evidence, what convinced me was to get away from all this relatively complicated math of one-bit DACs. At its simplest, DAC is actually a very straightforward device. You want to convert a number to a voltage, so you can take the binary digits of that number, connect them to an op-amp via resistors that correctly factor for their significance to sum all those voltages up and voila - you have a DAC. You can model this on a computer with ideal components and you will find it works fine. Except the real world is not ideal, resistors vary, op-amps are not perfect, electrons are slippery etc. and all these imperfections introduce non-linearity. But we can measure that non linearity and can gauge how far we are away from mathematically ideal. That non-linearity is so low for well designed and built devices that measure transparent, I believe it is exceedingly unlikely for them to sound different. But if someone offers a mechanism as to why that might not be the case, I am all ears.

With no experimental evidence and only thinking this through, it has been my long standing "gut" feeling that virtually all DAC chips should sound alike. However that is not the same as saying that all DACs are alike or may sound alike. The finished product with their power supplies and analog circuits etc. are what I am curious about.

From personal sighted and uncontrolled "pre ASR" experience, I have believed that DACs as components not chips have steadily improved since their introduction until some point say 10 years ago or so. I am perfectly willing to accept that this may be incorrect, but determining if this can be proved/disproved by controlled listening tests is what I am interested in.

 

[Purité Audio]

To be truly convinced you need to perform unsighted level matched comparisons for yourself.
Keith

 

[SIY]

Not sure what the hoo hah is about. Blind and level-matched, there was no significant identification.

Exactly what would be expected, dog bites man.

 

[Dj7675]

Disclaimer:
This post describes my subjective impressions of these DACs, level matched with instant switching.
It is not a double-blinded multi-run statistical analysis of anything. I know what those are, and this is not that.
Even if sighted tests are not your thing, the setup is fun. IMO, of course.

Note: Most images a thumbnails, so click to enlarge.

Why?
After endless futzing with Windows 11 and Roon, I got tired of noticeable differences in playback, which I blame on Windows/NUC.
I could never get Windows 11 to start Roon automatically and got tired of trying so I decided to ditch Windows for a Roon Rock.
I bought a NUC 12 I5-1240P (for headroom if I decide to DSP) and loaded Roon Rock, stuck it in the closet next to the network gear.
The BIOS is set to boot on loss. Problem solved. This in turn, led me to play with Raspberry Pi (RAAT) endpoints.

Then, I started watching videos, read the thread on ASR on I2S, and thought it might be fun to compare USB to I2S connectivity.
ASR SINAD DAC had the Topping D90 on top and it had every connection (I2S, AES, USB, SPDIF), so I bought one from Amazon.
I2S has to wait for a NUC parts so I set about to compare these two DACs.

The Setup
The Benchmark DAC3 and Topping D90LE are connected to the ARX RS-1 automatic switch using a Wi-Fi enabled Shelly1 relay.
The Shelly 1 web page power button remotely switches the XLR output from each DAC with slight pop, but no detectable delay.

The gear:
- Benchmark DAC3 set to HT modes (Fixed volume) and 4 Volt output
- Topping D90LE set to XLR out, filter 5 (linear phase fast) DAC mode (Fixed volume)
- Benchmark AHB2 driving Revel M126be monitors
- Raspberry Pi 3B+ and Pi 4B with DietPi and Roon RAAT (minimal configuration)
- Pangea USB cable and Monosaudio 4N $15 USB cable (quite nice for the money)
https://www.amazon.com/gp/product/B08H8SHKNV/ref=ppx_yo_dt_b_asin_title_o01_s00?ie=UTF8&psc=1

Fast Switching
- ARX RS-1 configured with a 12-volt trigger to select the XLR input
- Shelly 1 relay with Wi-Fi and a web app to select the input
- Roon DSP volume control, level matching, and grouped zones for simultaneous playback

View attachment 268915 View attachment 268916

Level Matching
The levels were matched using the Roon DSP volume control and measuring the voltage (using a FLUKE 87V true sinewave multimeter) while playing a 1 kHz 0dBFS tone into the M126bes.
The target voltage is 2.83 volts, commonly used since at 8 Ohms this is 1 watt. This was also the listening level used.

D90 2.807 volts: View attachment 268913 DAC3 2.825: View attachment 268918
The DAC3 has a slightly more output of 0.018 volts.

I am no EE, but I supposed that voltage is linear to dB, so I measured the DAC3 adjusting the volume down 1 db.
The 1 dB difference reduced the voltage by 0.244, so (if this calculation makes sense) the volume difference is 0.07 dB by voltage, well below the commonly recommended 0.1 dB level match.
View attachment 268917
The iPad ran Roon to select tracks and the iPhone displayed the Shelly1 web page to switch inputs at the press of the web power button.
View attachment 268922

Is there an Audible Difference
Let me say for the record, that if I sat a family member down and switched DACs they would not know the switch happened.
If I walked into the room, I probably would not know which DAC was playing.

Still, with instantaneous switching, I do detect some differences.
The D90LE adds a bit to female vocals that result in a slightly more pronounced sibilants.
On some tracks with a great deal of ambience, the D90 seems to a more expansive sound, where the DAC3 sounds a hair more damped.
There are times with the emphasis does not strike me as quite right but, it could easily be more accurate.

The DAC3 seems to be a bit better defined dynamics where the instruments have a bit more impact.

Tracks used for vocals and ambience:

- I See Fire - Jasmine Thompson
- California Dreaming - Diana Krall
- Tall Trees in Georgia = Eva Cassidy

Tracks used for detail and impact:

- Undiscovered First - Feist
- Santa Monica Dream - Angus & Julia Stone
- La Bell Dame Sans Regrets - Sting

So, I do think there are very subtle detectable differences.

Amazon D90LE order debacle
I bought this from Amazon from the KGUSS Store but it appears to be used and not new as advertised.

- It has the wrong remote that does not appear as the pictures and does not work with the D90LE (see image below).
- The settings were not factory, it was in DAC mode and to RCA only.
- The packaging looked disheveled.
- There was not protective plastic on the display (I don't know if there should be, but most products at this price range have it).

Here is a picture of the remote (that clearly is not the one for the D90).
View attachment 268914

The D90LE is going back to Amazon.

EDIT:

I performed a blind test, full post here:

Listening Tests - Benchmark DAC3 versus Topping D90LE

Disclaimer: This post describes my subjective impressions of these DACs, level matched with instant switching. It is not a double-blinded multi-run statistical analysis of anything. I know what those are, and this is not that. :) Even if sighted tests are not your thing, the setup is fun. IMO...

www.audiosciencereview.com

The results: I correctly identified the DAC3 from the D90LE 2/3rds of the 15 attempts.

Edit 7Mar2023: Here is the simplified version of the results, the old version was causing confusion. A thumbnail of the original presentation is provided below.
View attachment 270037

Old table:
View attachment 270039


I think this is meaningful, others may not.

Rich

What I appreciate most about this is a good faith attempt to set something up and test. A few will take the time to attempt to do this in the best way they can. Appreciate you taking the time to do this and posting your results.

 

[IAtaman]

From personal sighted and uncontrolled "pre ASR" experience, I have believed that DACs as components not chips have steadily improved since their introduction until some point say 10 years ago or so. I am perfectly willing to accept that this may be incorrect, but determining if this can be proved/disproved by controlled listening tests is what I am interested in.

That is fine. As said above, I do appreciate the good faith put into these tests as well. It is jut I am not sure how successful listening tests would be to reach a conclusion one way or another when the much less understood and non-linear device in the test is the measuring equipment, that is, your own perception.

 

[Blumlein 88]

To be truly convinced you need to perform unsighted level matched comparisons for yourself.
Keith

Yep, that is an excellent experience.

 

[RichB]

Here is an expanded chart with the human factor

This is for fun, but this is a pretty description of emotions during the test.
It is possible, that the pattern is the result of rushing and fatigue.

If I do this again, I may take short breaks between each run.

- Rich

 

[RichB]

Here are the ASR measurements of DAC3 and the D90SE (which is the almost the same DAC without MQA).

DAC 3 - Maximum SINAD 1kHz to 10 kHz is 104 dB (THD 0.0006)

DS90SE SINAD 1Khz to 10 kHz is 94 dB (THD 0.002)

Filter 7 is very well behaved.
I was using the DS90LE using the linear phase fast filter 5 (I think), not measured here.

- Rich

 

[lordhumungous]

The question i have is the Benchmark really worth paying 1600 more for. I love Benchmark products though.

 

[RichB]

The question i have is the Benchmark really worth paying 1600 more for. I love Benchmark products though.

Only you can make that decision.
For me, the build quality, well implemented reconstruction filter, high gain match for the AHB2, and trigger for the amp are enough.

- Rich

 

[lordhumungous]

Only you can make that decision.
For me, the build quality, well implemented reconstruction filter, high gain match for the AHB2, and trigger for the amp are enough.

- Rich

Well when I can buy 3 Topping D90LE’s for the price of the Benchmark Dac 3 ……. Hmmm
I have the AHB2 and the one Topping LA90 so I could go either way.