• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Listening Tests - Benchmark DAC3 versus Topping D90LE

antcollinet

Master Contributor
Joined
Sep 4, 2021
Messages
7,408
Likes
12,291
Location
UK/Cheshire
Agreed and plan on doing it when the queue of more important things is gone through.


No argument with any of that.

I have no beef with your logic, but I have not yet seen the conclusive test. Harman's speaker tests are probably not going to convince some "flat earthers" that scientifically designed loudspeakers are preferred by more listeners than magically designed speakers, but I do accept their test results and testing methods. I would like to see this sort of test of these measurably "transparent" electronics before claiming certainty.

To me there is still reasonable doubt.
You can't have a test that proves differences cannot be heard. That is trying to prove a negative.

All you can prove is that the peeople involved with the test can't hear it.

What is requried is a test that proves a difference CAN be heard. That can be done by just one person. So far - nothing conclusive. But don't be surprised (if one comes up) that person is going to be "grilled" because the result is SO unlikely. It is far more likely to be due to faulty test set up (Getting this right is hard), than to actually show a genuine difference.

And we are not talking about speakers - they are clearly differentiable. Amps - the jury is still out for me, especially when close to limits, though differences - if any - are likely to be of little consequence. But Dacs are pretty much a done deal.
 

ex audiophile

Addicted to Fun and Learning
Forum Donor
Joined
Jan 28, 2017
Messages
635
Likes
805
You can't have a test that proves differences cannot be heard. That is trying to prove a negative.

All you can prove is that the peeople involved with the test can't hear it.

What is requried is a test that proves a difference CAN be heard. That can be done by just one person. So far - nothing conclusive. But don't be surprised (if one comes up) that person is going to be "grilled" because the result is SO unlikely. It is far more likely to be due to faulty test set up (Getting this right is hard), than to actually show a genuine difference.

And we are not talking about speakers - they are clearly differentiable. Amps - the jury is still out for me, especially when close to limits, though differences - if any - are likely to be of little consequence. But Dacs are pretty much a done deal.
Agree completely except that I wonder about speakers. Have you seen the Andrew Robinson video where he goes to the Harman facility and neither he nor any of the other participants could reliably distinguish speakers of quite different price/quality, including the Revel Salon 2? I can find it if you wish.
I don't mean to get into whether AR is a credible reviewer or not, it's just that he couldn't tell speakers apart when blinded. Perhaps the "distinguishability" of speakers is not as great as we think.
 

Mr. Widget

Major Contributor
Forum Donor
Joined
Oct 11, 2022
Messages
1,146
Likes
1,697
Location
SF Bay Area
You can't have a test that proves differences cannot be heard. That is trying to prove a negative.

All you can prove is that the people involved with the test can't hear it.
And that will satisfy me if the sample size is large enough... 50-100 people.

And we are not talking about speakers - they are clearly differentiable. Amps - the jury is still out for me, especially when close to limits, though differences - if any - are likely to be of little consequence. But Dacs are pretty much a done deal.
You may or may not be surprised to hear... it sounds like you and I are on the same page here.
 
OP
RichB

RichB

Major Contributor
Forum Donor
Joined
May 24, 2019
Messages
1,946
Likes
2,611
Location
Massachusetts
And that will satisfy me if the sample size is large enough... 50-100 people.


You may or may not be surprised to hear... it sounds like you and I are on the same page here.
Harman evaluates hearing and trains listeners who participate in speaker evaluation.

Here is an older video from Jon Iverson that I agree with:


At 25 minutes, Jon discusses his observations concerning DACs.

- Rich
 
Last edited:
OP
RichB

RichB

Major Contributor
Forum Donor
Joined
May 24, 2019
Messages
1,946
Likes
2,611
Location
Massachusetts
Here is a thought experiment.
Let's say that I truthfully could select the DAC3 with high probability, say 14 out 15 times (though I could not).

Could we conclude that these DACs sound differently?
I'd say no, not without further investigation.

We would have to look at difference in the environment.
Here other possibilities for detectability:

- The cables are different or different lengths; one cable is perhaps faulty or has different impedance
- The ARX RS-1 is creating an interaction
- The volume difference while low .7 dB could account for the difference
- One both of these DAC samples are not performing up to specification
- These Raspberry PIs introducing some errors or noise that exhibits performance issues
- Reconstruction Filter Implementation - these definitely differ in these DACS

So, even a determinative result does not necessarily implacate the DAC.

It may also be worthwhile to consider the aspects for ASR DAC testing.

- Output impedance -- does this differ substantially from users at home?
- Reconstruction Filter Implementation
- Power and other environmental issues
- Frequency based distortion levels and characteristics

It seems possible that some DACs are more immune to impurities found in a particular home verse the AP measuring environment.

- Rich
 

okok

Senior Member
Joined
Oct 31, 2020
Messages
377
Likes
161
must be a lie or the human ears too sharp or the output level not equally done
 

MattHooper

Master Contributor
Forum Donor
Joined
Jan 27, 2019
Messages
7,194
Likes
11,806
You can't have a test that proves differences cannot be heard. That is trying to prove a negative.

All you can prove is that the peeople involved with the test can't hear it.

What is requried is a test that proves a difference CAN be heard. That can be done by just one person. So far - nothing conclusive. But don't be surprised (if one comes up) that person is going to be "grilled" because the result is SO unlikely. It is far more likely to be due to faulty test set up (Getting this right is hard), than to actually show a genuine difference.

And we are not talking about speakers - they are clearly differentiable. Amps - the jury is still out for me, especially when close to limits, though differences - if any - are likely to be of little consequence. But Dacs are pretty much a done deal.

I hear you!

A problem for us "regular Joes" - those of us without formal training in electronics - is that there are many edge cases where we can see knowledgeable people disagreeing. Everything from the sonic importance of X or Y in speaker design, amp design, DACs etc. So it's not always easy to just point to one monolithic opinion in every case to say "See, This Is Settled!"

So individuals are still left to sort of triangulate the best they can on what experts say, along with their own experience in forming one's direction or view. (Which is also what engineers have to do...)

For example, in my case: Tube rolling on my CJ amp. I seem to hear really distinct differences with certain sets of tube types vs others, and really enjoy this effect. I've seen people with electronics knowledge point out why this is totally plausible (even expected), while others will say often tube rolling shouldn't change the sound audibly at all. And of course the whole thing is ripe for the influence of bias.

So...what do I do? In my case, I go with what I seem to percieve, and keep tube rolling for the fun of it. But I do so with a built in caveat "I Could Be Imagining Things" and also would never make any strong claim that anyone else has to take seriously, based on my purely anecdotal experience.
 

antcollinet

Master Contributor
Joined
Sep 4, 2021
Messages
7,408
Likes
12,291
Location
UK/Cheshire
Here is a thought experiment.
Let's say that I truthfully could select the DAC3 with high probability, say 14 out 15 times (though I could not).

Could we conclude that these DACs sound differently?
I'd say no, not without further investigation.

We would have to look at difference in the environment.
Here other possibilities for detectability:

- The cables are different or different lengths; one cable is perhaps faulty or has different impedance
- The ARX RS-1 is creating an interaction
- The volume difference while low .7 dB could account for the difference
- One both of these DAC samples are not performing up to specification
- These Raspberry PIs introducing some errors or noise that exhibits performance issues
- Reconstruction Filter Implementation - these definitely differ in these DACS

So, even a determinative result does not necessarily implacate the DAC.

It may also be worthwhile to consider the aspects for ASR DAC testing.

- Output impedance -- does this differ substantially from users at home?
- Reconstruction Filter Implementation
- Power and other environmental issues
- Frequency based distortion levels and characteristics

It seems possible that some DACs are more immune to impurities found in a particular home verse the AP measuring environment.

- Rich
They are all valid points.

However - at the end of the day - we are talking (generally) minscule differences, that even if audible are so tiny we have to listen carefully while switching from one to the other. Sometimes we even have to be trained on what cues to listen for.

After all that, do we reallly care? I just want people to stop claiming that if people spend megabucks on stuff that makes zero or close to zero difference that they'll achieve audio nirvana.

Until I got the MiniDSP, my DAC was an old external soundblaster that "only" achieves just under -90dB THD+N. I think I spent about £70 on it. It was fine. I'm pretty certain I'd not hear a difference between that and a multi thousand pound Chord DAC, and even if I could, I'd not care which one I listened through.
 
Last edited:

antcollinet

Master Contributor
Joined
Sep 4, 2021
Messages
7,408
Likes
12,291
Location
UK/Cheshire
I hear you!

A problem for us "regular Joes" - those of us without formal training in electronics - is that there are many edge cases where we can see knowledgeable people disagreeing. Everything from the sonic importance of X or Y in speaker design, amp design, DACs etc. So it's not always easy to just point to one monolithic opinion in every case to say "See, This Is Settled!"

So individuals are still left to sort of triangulate the best they can on what experts say, along with their own experience in forming one's direction or view. (Which is also what engineers have to do...)

For example, in my case: Tube rolling on my CJ amp. I seem to hear really distinct differences with certain sets of tube types vs others, and really enjoy this effect. I've seen people with electronics knowledge point out why this is totally plausible (even expected), while others will say often tube rolling shouldn't change the sound audibly at all. And of course the whole thing is ripe for the influence of bias.

So...what do I do? In my case, I go with what I seem to percieve, and keep tube rolling for the fun of it. But I do so with a built in caveat "I Could Be Imagining Things" and also would never make any strong claim that anyone else has to take seriously, based on my purely anecdotal experience.
It's all any of us can do.

But here is the really good thing. If we find we are listening, and we don't like the sound of the system - we know it is fixable, and probably without buying new kit. We just have to approach it in a systematic and engineering based way. Measurements if possible and take advantage of the experts here to help us.
 

BDWoody

Chief Cat Herder
Moderator
Forum Donor
Joined
Jan 9, 2019
Messages
6,948
Likes
22,625
Location
Mid-Atlantic, USA. (Maryland)
While I respect measurements and use them myself professionally and as an enthusiast, I haven't see the conclusive correlation between human auditory perception and the measured performance of DACs and other electronics that measure well.

I am not even particularly concerned with 99th percentile human auditory perception. Whenever I ask to see the results of such a test, the ASR response is about proving a negative or some similar dismissive response.


Have you seen this thread?


This is going to be more of a preponderance of the evidence thing, because proving inaudibility is a bit of a Russell's teapot.
 
Last edited:

Sokel

Master Contributor
Joined
Sep 8, 2021
Messages
5,830
Likes
5,758
Have you seen this thread?


This is going to be more of a preponderance of the evidence thing, because proving inaudibility is a bit of a Russell's teapot.
I think I have asked about this before but never got an answer:
Can we be pointed to a valid research about those specific limits?
The thread seems to take them as granted but with no references.
 

theREALdotnet

Major Contributor
Joined
Mar 11, 2022
Messages
1,181
Likes
2,036
I think the result is very much in line with the prediction made by measurements of the two DACs and it‘s also a testament to the quality of your switching setup.

The result shows that both DACs sound the same. Specifically, they both sound like DAC “off”. The fact that this was the correct answer 67% of the time is simply due to the fact that DAC “off” was playing 67% of the time. Like a broken clock being correct twice a day :)

Commonly, during tests like this, people try hard to discern differences and imagine them if there aren’t any. That’s where the guessing comes in. But you steadfastly determined that the two DACs do in fact sound like one and the same device.
 

theREALdotnet

Major Contributor
Joined
Mar 11, 2022
Messages
1,181
Likes
2,036
Since you have the test setup in place, you might consider conducting a different type of test: play the same musical passage twice, once through each DAC (without knowing which DAC is playing). Do this with 10 different passages. The test administrator would flip a coin each time to determine which DAC plays first, and you note down which version you prefer.
 

restorer-john

Grand Contributor
Joined
Mar 1, 2018
Messages
12,579
Likes
38,274
Location
Gold Coast, Queensland, Australia
It seems possible that some DACs are more immune to impurities found in a particular home verse the AP measuring environment.

Some D/As have characteristics that enable easy identification in blind tests, for instance mild DC offsets that cause a tiny 'tick' when the relays select that input. Or the latency of the D/A may be audible against another D/A with an identical data stream, with instant switching. Accurate level matching is difficult if the only controls are coarse digital steps.

Channel balance is another giveaway, especially with classical music where tiny imbalances will shift the image of individual instruments enough to be a consistent indicator of which D/A you are listening to.

I applaud @RichB for his own personal exploration of his own D/A differences and relaying his impressions. At the end of the day, it really doesn't matter what anyone thinks about the validity or lack thereof of his tests. They were for his own benefit, not anyone else's. But, as always, it's much easier for the noisy peanut gallery to decry his methods and outcomes than actually get off their own #sses and do their own comparisons.
 

pablolie

Major Contributor
Forum Donor
Joined
Jul 8, 2021
Messages
2,022
Likes
3,318
Location
bay area, ca
To me at least, arguing about *perceived* SQ is pointless. We are all entitled to have personal preferences for whatever reason - but trying to see any vestige of "scientific proof" in listening tests that are conducted in ways that simply help establish such personal preference discredits the exercise if the intent of the test is to "provide proof X is better".

On the other hand, I wish tests weren't limited to "strict linearity is best", because that provides zero service to those who prefer a certain sound signature. It should be possible to publish a scale that says along the lines of "0-10 scale when it comes to (a) analytical nature (b) pleasing harmonics.. etc.

As I have said before, personally I prefer a more analytical nature myself (without obsessing about it being the sharpest possible accuracy at the moment). Analytical enough to be able to tell apart compressed from FLAC with certain reference tracks I use for such silly exercises), and allows me to easily tell apart good from lousy recordings. But I'll never obsess about that last 1dB in SINAD etc... But that said: a friend of mine is basically Chopin obsessed, and he has a system that is (a) quite upscale and (b) quite colored... but I admit I love listening to Chopin and other piano works when I visit his place (which is basically weekly) on his system So I absolutely accept personal preference can triumph over perfection in measurements. But again - I think it would be useful to quantify such preference based on the nature of harmonics etc.

To me, neither of the two products mentioned by the OP seem to deviate from the engineering approach to deliver on as much linearity as possible. So I'd have to be convinced there's a difference in how they sound. That said, all that matters is that the OP established their preference, and we should all let them enjoy it to the fullest. Lucky those who have settled on their perfect preference in audiophile land... :)
 

MattHooper

Master Contributor
Forum Donor
Joined
Jan 27, 2019
Messages
7,194
Likes
11,806
I really enjoy reading about people's blind tests - even just the attempts can be entertaining or even educational (e.g. the discussions they bring forth).

It's amazing how triggering blind tests are for the more subjective-oriented audiophile crowd. In any cable thread on the "other forums" if I bring up my own blind tests (e.g. of Shunyata AC cables - negative for differences) many immediately hallucinate that I have made an argument like "I could not reliably detect differences with high end AC cables" to "THEREFORE I've shown there are never sonic differences between AC cables!"

But that of course is never the claim. The skepticism I have regarding audiophile AC cables doesn't just come from my own blind test, but from the well known technically controversial claims made for such cables. As for the implications of my personal blind tests, I have to explain over and over that I take two main implications:

1. Insofar as I seem to have controlled for sighted bias, the test suggests I can not reliably detect an audible difference between the cables.

And I can use that as a basis in deciding what I'm willing to pay for (in this case, I'm not willing to pay a lot of money for cables that don't make any notible improvement to the sound of my system. Someone else may choose otherwise).

2. It is yet another lesson learned with regards to the power of sighted bias. When you have experienced "obvious sonic differences" disappear once you don't know which gear you are hearing, it's a personal lesson on the power of sighted bias. It is reasonable to take such lessons to claims other people are making. So if another audiophile is staking his position on "what I know I heard" between two cables, my blind test reminds me of the unreliability of this type of report. I've been there, I've had a similar perception, but I also know how misleading that perception can be, so I'll wait for stronger evidence than another audiophile's impression he/she heard a difference.

Always open to the proposition cables make audible differences. Just...show me some reliable evidence, that's all.
 

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,521
Likes
37,050
I think the result is very much in line with the prediction made by measurements of the two DACs and it‘s also a testament to the quality of your switching setup.

The result shows that both DACs sound the same. Specifically, they both sound like DAC “off”. The fact that this was the correct answer 67% of the time is simply due to the fact that DAC “off” was playing 67% of the time. Like a broken clock being correct twice a day :)

Commonly, during tests like this, people try hard to discern differences and imagine them if there aren’t any. That’s where the guessing comes in. But you steadfastly determined that the two DACs do in fact sound like one and the same device.
You misunderstood the procedure he used. I did too at first because of the chart presented. He listened to each dac for each trial. Then selected which he thought was the Dac3. So he heard both each time and chose correctly 10 of 15.

He did NOT just listen once and choose the Dac3 all 15 times.
 

Mr. Widget

Major Contributor
Forum Donor
Joined
Oct 11, 2022
Messages
1,146
Likes
1,697
Location
SF Bay Area
Have you seen this thread?
I just gave it a quick once over. very interesting and useful, but it doesn't satisfy my question. Can we mere mortals actually hear a difference in a blind test or not. The preponderance of evidence shown here is we can not. But the evidence is not based on a large sample size of actual listening tests.

Short of a large sample size test, I'll have to wait until I can set aside the time to create my own test. (I am hoping in the meantime someone comes forth with a large scale well thought out and executed test with a definitive conclusion one way or the other.)
 

pablolie

Major Contributor
Forum Donor
Joined
Jul 8, 2021
Messages
2,022
Likes
3,318
Location
bay area, ca
... "obvious sonic differences" disappear once you don't know which gear you are hearing, it's a personal lesson on the power of sighted bias. ...
Personally -maybe I am blessed because I consistently picked the right gear- I haven't heard huge "obvious sonic differences" in many years. I hear slight ones, and then the hard part is to establish a personal preference between choices that are very close in performance.

And that just applies to my "music shrine system", which is where I may care about the ultimate 5% in SQ. i can equally enjoy my favorite music on the $500 system i have in my little nature cabin.
 
Top Bottom