• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Listening Tests - Benchmark DAC3 versus Topping D90LE

RichB

Major Contributor
Forum Donor
Joined
May 24, 2019
Messages
1,946
Likes
2,611
Location
Massachusetts
Disclaimer:
This post describes my subjective impressions of these DACs, level matched with instant switching.
It is not a double-blinded multi-run statistical analysis of anything. I know what those are, and this is not that. :)
Even if sighted tests are not your thing, the setup is fun. IMO, of course.

Note: Most images a thumbnails, so click to enlarge.

Why?
After endless futzing with Windows 11 and Roon, I got tired of noticeable differences in playback, which I blame on Windows/NUC.
I could never get Windows 11 to start Roon automatically and got tired of trying so I decided to ditch Windows for a Roon Rock.
I bought a NUC 12 I5-1240P (for headroom if I decide to DSP) and loaded Roon Rock, stuck it in the closet next to the network gear.
The BIOS is set to boot on loss. Problem solved. This in turn, led me to play with Raspberry Pi (RAAT) endpoints.

Then, I started watching videos, read the thread on ASR on I2S, and thought it might be fun to compare USB to I2S connectivity.
ASR SINAD DAC had the Topping D90 on top and it had every connection (I2S, AES, USB, SPDIF), so I bought one from Amazon.
I2S has to wait for a NUC parts so I set about to compare these two DACs.

The Setup
The Benchmark DAC3 and Topping D90LE are connected to the ARX RS-1 automatic switch using a Wi-Fi enabled Shelly1 relay.
The Shelly 1 web page power button remotely switches the XLR output from each DAC with slight pop, but no detectable delay.

The gear:
- Benchmark DAC3 set to HT modes (Fixed volume) and 4 Volt output
- Topping D90LE set to XLR out, filter 5 (linear phase fast) DAC mode (Fixed volume)
- Benchmark AHB2 driving Revel M126be monitors
- Raspberry Pi 3B+ and Pi 4B with DietPi and Roon RAAT (minimal configuration)
- Pangea USB cable and Monosaudio 4N $15 USB cable (quite nice for the money)
https://www.amazon.com/gp/product/B08H8SHKNV/ref=ppx_yo_dt_b_asin_title_o01_s00?ie=UTF8&psc=1

Fast Switching
- ARX RS-1 configured with a 12-volt trigger to select the XLR input
- Shelly 1 relay with Wi-Fi and a web app to select the input
- Roon DSP volume control, level matching, and grouped zones for simultaneous playback

DAC2vD90Rig.jpg DAC2vD90RigM26.jpg

Level Matching
The levels were matched using the Roon DSP volume control and measuring the voltage (using a FLUKE 87V true sinewave multimeter) while playing a 1 kHz 0dBFS tone into the M126bes.
The target voltage is 2.83 volts, commonly used since at 8 Ohms this is 1 watt. This was also the listening level used.

D90 2.807 volts: D90VoltageDSP-14.jpg DAC3 2.825: DAC3VoltageDSP-12.jpg
The DAC3 has a slightly more output of 0.018 volts.

I am no EE, but I supposed that voltage is linear to dB, so I measured the DAC3 adjusting the volume down 1 db.
The 1 dB difference reduced the voltage by 0.244, so (if this calculation makes sense) the volume difference is 0.07 dB by voltage, well below the commonly recommended 0.1 dB level match.
DAC3 and D90LE Volume-Voltage Levels.jpg

The iPad ran Roon to select tracks and the iPhone displayed the Shelly1 web page to switch inputs at the press of the web power button.
Shelly1.jpg

Is there an Audible Difference
Let me say for the record, that if I sat a family member down and switched DACs they would not know the switch happened.
If I walked into the room, I probably would not know which DAC was playing.

Still, with instantaneous switching, I do detect some differences.
The D90LE adds a bit to female vocals that result in a slightly more pronounced sibilants.
On some tracks with a great deal of ambience, the D90 seems to a more expansive sound, where the DAC3 sounds a hair more damped.
There are times with the emphasis does not strike me as quite right but, it could easily be more accurate.

The DAC3 seems to be a bit better defined dynamics where the instruments have a bit more impact.

Tracks used for vocals and ambience:

- I See Fire - Jasmine Thompson
- California Dreaming - Diana Krall
- Tall Trees in Georgia = Eva Cassidy

Tracks used for detail and impact:

- Undiscovered First - Feist
- Santa Monica Dream - Angus & Julia Stone
- La Bell Dame Sans Regrets - Sting

So, I do think there are very subtle detectable differences.

Amazon D90LE order debacle
I bought this from Amazon from the KGUSS Store but it appears to be used and not new as advertised.

- It has the wrong remote that does not appear as the pictures and does not work with the D90LE (see image below).
- The settings were not factory, it was in DAC mode and to RCA only.
- The packaging looked disheveled.
- There was not protective plastic on the display (I don't know if there should be, but most products at this price range have it).

Here is a picture of the remote (that clearly is not the one for the D90).
D90WrongRemote.jpg

The D90LE is going back to Amazon.

EDIT:

I performed a blind test, full post here:

The results: I correctly identified the DAC3 from the D90LE 2/3rds of the 15 attempts.

Edit 7Mar2023: Here is the simplified version of the results, the old version was causing confusion. A thumbnail of the original presentation is provided below.
DAC3 v D90LE blind test Simplified.jpg


Old table:
DAC3 v D90LE blind test.jpg


I think this is meaningful, others may not.

Rich
 
Last edited:

Mr. Widget

Major Contributor
Forum Donor
Joined
Oct 11, 2022
Messages
1,146
Likes
1,697
Location
SF Bay Area
Is there an Audible Difference
Let me say for the record, that if I sat a family member down and switched DACs they would not know the switch happened.
If I walked into the room, I probably would not know which DAC was playing.

Still, with instantaneous switching, I do detect some differences.
First off, thank you for posting. This is interesting. I have done this sort of comparison over the years and found similar results with DACs and amplifiers.

Since joing ASR last fall and having discussed this type of "difference" with others here on ASR I must agree that an unsighted comparison is a must.

Is there anyway you could impose on a friend to perform the switching without your knowledge of which DAC is being used? I would find the results of that test very intriguing.

I have been meaning to perform such a test myself, but looking at my rather long list of obligations and of other interests, it will likely be a long time before I am able to get to this.
 

jae

Major Contributor
Joined
Dec 2, 2019
Messages
1,208
Likes
1,508
Why not measure both with a microphone? The differences you say you picked up on should be easily identifiable like this.
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,521
Likes
37,050
I've got a Focusrite interface. I've always thought it had a bit of a sound signature. A bit less air, a slightly opaque sound, and a barely fluffy less impactful bass compared to other DACs or interfaces.

I've recorded it with three different good ADCs, and then compared in Foobar ABX to a couple other DACs. I've had someone switch it in a way I didn't know which was which without an ADC involved. Never have I gotten any results that even hinted I could tell it different from the others. Yet even now I think I can hear it.:oops:
 
Last edited:

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,521
Likes
37,050
Why not measure both with a microphone? The differences you say you picked up on should be easily identifiable like this.
Most likely you could hear a difference due to varying noise levels. Even if there is no other difference. You'd be surprised what a microphone picks up you normally don't notice.
 
OP
RichB

RichB

Major Contributor
Forum Donor
Joined
May 24, 2019
Messages
1,946
Likes
2,611
Location
Massachusetts
First off, thank you for posting. This is interesting. I have done this sort of comparison over the years and found similar results with DACs and amplifiers.

Since joing ASR last fall and having discussed this type of "difference" with others here on ASR I must agree that an unsighted comparison is a must.

Is there anyway you could impose on a friend to perform the switching without your knowledge of which DAC is being used? I would find the results of that test very intriguing.

I have been meaning to perform such a test myself, but looking at my rather long list of obligations and of other interests, it will likely be a long time before I am able to get to this.

Good point, I will enlist my daughter to be the record keeper and switcher for a blinded comparison this weekend.
Monday, the D90ES goes back to Amazon.

- Rich
 
OP
RichB

RichB

Major Contributor
Forum Donor
Joined
May 24, 2019
Messages
1,946
Likes
2,611
Location
Massachusetts
I've got a Focusrite interface. I've always thought it had a bit of a sound signature. A bit less air, a slightly opaque sound, and a barely fluffy less impactful bass compared to other DACs or interfaces.

I've recorded it with three different good ADCs, and then compared in Foobar ABX to a couple other DACs. I've had someone switch it in a way I didn't know which was which without an ADC involved. Never have I gotten any results that even hinted I could tell it different from the others. Yet even now I think I can hear it.:oops:

I rule that outcome out, but I'll report the results after blind testing this weekend.

- Rich
 

pma

Major Contributor
Joined
Feb 23, 2019
Messages
4,591
Likes
10,727
Location
Prague
Yet even now I think I can hear it.:oops:
And maybe you really do. It is extremely difficult to setup a blind test properly, in case of tiny differences. They may be erased in the test. On the other hand, the difference is probably unimportant.
 
OP
RichB

RichB

Major Contributor
Forum Donor
Joined
May 24, 2019
Messages
1,946
Likes
2,611
Location
Massachusetts
Why not measure both with a microphone? The differences you say you picked up on should be easily identifiable like this.

Perhaps, this would be a steady state measurement of sweep, pink noise, that is not dynamic measurement of content.

If there was way to gather a high-resolution of voltage measurement at the input to the amplifier for each DAC.
These measurements could be compared with software that could also scale to align the curves.
IMO this would be conclusive.

- Rich
 
OP
RichB

RichB

Major Contributor
Forum Donor
Joined
May 24, 2019
Messages
1,946
Likes
2,611
Location
Massachusetts
I completed the one-and-only blind test that for these DACs.
Honestly, I find this stressful and though I found a track that I felt I could identify, it was quite difficult.


My daughter was behind me so I could not see here and randomized the starting point for each run.
She was recording the Shelly1 state on or off, she had no idea what this means.
This makes it blind, though since she had no idea what the significance of on/off state, it was in a sense, double-blind.

She would switch upon request and I selected the playback and decision point.
I tended to select Off because it seemed easier to detect that switch direction.

The cases where I was most certain where when I switched during the vocals and not a quieter interlude.
I performed better when I leaned in closer to the speakers and focused on the vocal intro section of Jasmine Thompson's I See Fire.
Attempts during the louder passages with vocal and instruments were hopeless (for me).

The results: I correctly identified the DAC3 from the D90LE 2/3rds of the 15 attempts.

Edit 7Mar2023: Here is the simplified version of the results, the old version was causing confusion. A thumbnail of the original presentation is provided below.

DAC3 v D90LE blind test Simplified.jpg




DAC3 v D90LE blind test.jpg
I think this is meaningful, others may not.
I think I might be able improve the test setup, but I plan to box up the D90LE today.

- Rich
 
Last edited:

Mr. Widget

Major Contributor
Forum Donor
Joined
Oct 11, 2022
Messages
1,146
Likes
1,697
Location
SF Bay Area
Thank you!

As you said, I think this IS meaningful, and yes others may not.

I totally get the stressful comment, the times I have been involved in double blind tests, it is strangely stressful.
 

MattHooper

Master Contributor
Forum Donor
Joined
Jan 27, 2019
Messages
7,194
Likes
11,808
Thanks for the heads up on this RichB!

Very interesting indeed.

It reminds me of my own blind tests I did in the late 90's between a Meridian CDP I owned, a Sony CDP and a Meitner DAC (which had volume control).
I was told they shouldn't sound different (if 'properly designed') which totally made sense to me. Except the sonic differences I seemed to hear, especially the Meridian vs the others, seemed absolutely distinct!

So I did two sets of blind tests (different days), including with level matched at the speaker terminals (volt or multi-metered) and was able to ace the blind tests identifying the units, no problem. In fact the specific tonal difference I keyed on with the Meridian was so distinct I even repeated the test where I went OUTSIDE the listening room, down the hallway a bit and just yelled to my helper when to switch between DACs, and again easily discerned the Meridian player!

Ironically, given that I had that experience, these days I don't think much at all about DAC sonic differences. I bought a Benchmark DAC1 because I knew it measured well and used it for many years, until I needed some added functionality and replaced it with a Benchmark DAC2L. Stereophile's measurements confirmed what I would have expected - excellent engineering and measurements. And...that's all I need. I don't see a need to care about DACs again.

If I want to care about sonic differences I'll look to vinyl playback - cartridges etc.

But your test suggests that perhaps there is still some differences detectable. I wonder what would account for detectable sonic differences between DACs today (to the degree they exist). Would it come more from the analog engineering or the choices made for the digital engineering?
 
OP
RichB

RichB

Major Contributor
Forum Donor
Joined
May 24, 2019
Messages
1,946
Likes
2,611
Location
Massachusetts
Thanks for the heads up on this RichB!

Very interesting indeed.

It reminds me of my own blind tests I did in the late 90's between a Meridian CDP I owned, a Sony CDP and a Meitner DAC (which had volume control).
I was told they shouldn't sound different (if 'properly designed') which totally made sense to me. Except the sonic differences I seemed to hear, especially the Meridian vs the others, seemed absolutely distinct!

So I did two sets of blind tests (different days), including with level matched at the speaker terminals (volt or multi-metered) and was able to ace the blind tests identifying the units, no problem. In fact the specific tonal difference I keyed on with the Meridian was so distinct I even repeated the test where I went OUTSIDE the listening room, down the hallway a bit and just yelled to my helper when to switch between DACs, and again easily discerned the Meridian player!

Ironically, given that I had that experience, these days I don't think much at all about DAC sonic differences. I bought a Benchmark DAC1 because I knew it measured well and used it for many years, until I needed some added functionality and replaced it with a Benchmark DAC2L. Stereophile's measurements confirmed what I would have expected - excellent engineering and measurements. And...that's all I need. I don't see a need to care about DACs again.

If I want to care about sonic differences I'll look to vinyl playback - cartridges etc.

But your test suggests that perhaps there is still some differences detectable. I wonder what would account for detectable sonic differences between DACs today (to the degree they exist). Would it come more from the analog engineering or the choices made for the digital engineering?

If I had to guess, I'd say the choice of output stage post DAC and the digital filtering are adding subtle differences.
Benchmark's elimination of inter-sample overs shows good engineering of an issue that occurs in source material.

There are number of tracks where I felt the D90 was emphasizing the decay (echo) but complex and vocal clarity was better on the DAC3.
It is not possible to determine which is correct, so I guess it comes down to trust.

As far as viny, I recognize that there are older albums that definitely sound far better than the current digital releases that are virtually unlistenable.
If I bought a turntable, I'd get one with a USB interface to rip and manually tag tracks. My friend sent me some vinyl rips, and they sound very good.

I was first exposed to Roon when Oppo added support to the UPD-205. Roon has been far more reliable than J River so bought the $500 lifetime membership (now $829, yikes).
Roon has transformed my listening experience. All my CDs are ripped, and I have added a Qobuz subscription.
The concept of reaching for media has become foreign to me. Also, Roon radio does a good job of selecting the next track and I found new music that I love.

- Rich
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,521
Likes
37,050
Did you match levels differently this time?
 

MattHooper

Master Contributor
Forum Donor
Joined
Jan 27, 2019
Messages
7,194
Likes
11,808
If I had to guess, I'd say the choice of output stage post DAC and the digital filtering are adding subtle differences.
Benchmark's elimination of inter-sample overs shows good engineering of an issue that occurs in source material.

There are number of tracks where I felt the D90 was emphasizing the decay (echo) but complex and vocal clarity was better on the DAC3.
It is not possible to determine which is correct, so I guess it comes down to trust.

As far as viny, I recognize that there are older albums that definitely sound far better than the current digital releases that are virtually unlistenable.
If I bought a turntable, I'd get one with a USB interface to rip and manually tag tracks. My friend sent me some vinyl rips, and they sound very good.

I was first exposed to Roon when Oppo added support to the UPD-205. Roon has been far more reliable than J River so bought the $500 lifetime membership (now $829, yikes).
Roon has transformed my listening experience. All my CDs are ripped, and I have added a Qobuz subscription.
The concept of reaching for media has become foreign to me. Also, Roon radio does a good job of selecting the next track and I found new music that I love.

- Rich

I've been considering trying Roon, only because I've found a few foibles in the existing UI for my Bluesound NODE.

One issue is that the NODE clearly wasn't designed for streaming from a local drive vs from a streaming service. Which shocked me because that was my main reason for getting one. I expect Roon will do a better job of organizing/playing my existing library.

Also, and this seems to be a "bug" that drives me crazy but doesn't seem to be mentioned by others: I can't stand that the Bluesound IOS controller app doesn't show any "playing now" indication beside a track that has been selected. On every other interface I've used, if there is a list of songs, either playlist or album track list, and I select a track, some indicator appears beside the track telling me that's the one playing - e.g. a "play" icon appears or an active meter bouncing.
Nothing on the Bluesound! Which makes using it a bit of a chore for me. It seems that when you select a track in Roon you do get a "playing" icon beside the selected track.
 

antcollinet

Master Contributor
Joined
Sep 4, 2021
Messages
7,408
Likes
12,291
Location
UK/Cheshire
Thank you!

As you said, I think this IS meaningful, and yes others may not.

How many times do you think you might get it right (on average) if you were purely guessing. What do you think the possible test to test variation on that might be?

Also - the listener response was always "off" which suggests you were not hearing any difference at all.
 
OP
RichB

RichB

Major Contributor
Forum Donor
Joined
May 24, 2019
Messages
1,946
Likes
2,611
Location
Massachusetts
Did you match levels differently this time?

No. I left everything unchanged since measuring.
Listening levels approximately 1 watt.

- Rich
 
OP
RichB

RichB

Major Contributor
Forum Donor
Joined
May 24, 2019
Messages
1,946
Likes
2,611
Location
Massachusetts
The blind test is very meaningful - it proved you could not tell the difference between the two DACs - as expected.

I suppose we find the answer we are looking for.

- Rich
 
Top Bottom