My blind DAC comparison: SMSL SU-8 vs Xduoo XD-05 vs Shanling M5s

[velasfloyd]

Hi everyone! I have been reading this forum from some time ago, thanks everyone for all the knowledge you have written here. This is my first post, I want to share my own testing with my equipment.

In order to know if a DAC really matters (for me) I wanted to blind test the devices I have. I know that we can easily lie to ourselves if we know what we are hearing, so I decided that an instant switch was the best way to test.

- My procedure:

I have a Burson Soloist SL as headphone amplifier. A valuable thing in this headamp is that it has a line in selector (2 different line ins). this is a button with no memory, in other words the only way I can know what line is selected is to watch a little led on the front.
So I connected two devices to the burson and with the line in selector button I can make an instant switch between them without knowing which is which.

I use Beyerdynamic's Amiron Home for all tests

I listened Dark side of the moon in all the tests. Because is a good record and most important I almost know it from memory because Pink Floyd is by far my favorite band.

Im not a golden ear and I cannot talk in this sublime audio terms like soundstage air space bla bla bla... Im just a normal person in my 30ties who enjoys music and just want the best audio possible...

Results:

1 - SMSL SU-8 vs XDuoo XD-05 line out

From the first time I have my SU-8 I noted a great improvement in the sound. This was my first time testing if this improvement was real or just the known placebo effect. I connected one line to the SU-8 and the other to the line-out output of the XD-05. This line out output has fixed volume so I have to change the volume of SU-8 to set them at the same level.

I could barely hear a different sound, SU-8 being better. From all the guesses I did I had a 100% accuracy detecting the best one and it was consistently the SU-8. But this is just a very little difference. I think there is no way I could perceive this difference if it wouldn't be for an instant switch. I cant describe exactly the difference.... I just found it better.... may be like a better definition of sound.

This is what I expected (due to the price of every equipment) and I was happy that I could hear a difference! (super small tho)

2- SMSL SU-8 vs Shanling M5S line out mode

The shanling M5s dap has a line out mode. This is nothing more than a convenience to instantly change the volume to the maximum. it is not a "real line out" because the signal pass through the amp part (if this matters can be discussed elsewhere). If I could hear a difference between XD-05 and SU-8, being XD-05 a real line out... the result of this second test should be obvious...

It was not!! short history: I couldn't hear any difference!! I made a lot of guesses like "I think this is sounding better"... and in some cases were SU-8 and in others M5S... that means I could not be consistent with my guesses and due to that.... no difference!!

how this is possible? no difference between a well evaluated dedicated desktop dac and a midfi dap's fake line out???

this was not what I wanted honestly... but this was my result.

3- SMSL SU-8 vs Xduoo XD-05 phone out

I thought that may be Shanling M5S was super good and so if XD-05 line out was worst, then the XD-05 phone out should be even worst...
I put the xduoo at the max volume, set an equal level on SU-8 (and both connected to Burson Soloist) and then........ SORPRISE!! I could not hear any difference again.... same history, lot of guesses without any consistence!!!


CONCLUSIONS

1.- may be, as stated a lot of times in this forum, DAC does not matter too much. even with instant blind switch the differences, if ever, are rather small (of course my testing is super limited)
2- I dont understand why phone out of xduoo xd-05 sounded better than line-out when used as a line out..... as the phone out uses the amp part of the device "should" sound worse..... may be the burson amp only care about the energy that come into it (as the phone out has more volume than line out in this xd-05 case)

well friends. Sorry if this testing sounded too boring for you. I just wanted to share it with you and may you can add something to it. I will continue testing devices as it is a hobbie after all.

 

[Nango]

Thanks for sharing your findings.

 

[solderdude]

Hi everyone! ...

Don't know whether to congratulate you or feel for you sorry as you now belong to the bunch of deaf idiots who cannot even hear all the obvious differences.
Let this be a lesson to you... never test blind and level matched as you can loose some special abilities.

 

[SIY]

Chapeau! for putting your perceptions to the test. Most people are too lazy or too nervous to even try it.

Question: how did you do the level matching?

 

[velasfloyd]

Edit: I used Beyerdynamic's Amiron Home for all my tests. It is my favorite headphone and I love it. It is my only Hifi headphone as well haha

Don't know whether to congratulate you or feel for you sorry as you now belong to the bunch of deaf idiots who cannot even hear all the obvious differences.
Let this be a lesson to you... never test blind and level matched as you can loose some special abilities.

haha. I would prefer to actually hear differences if I buy another and more expensive stuff. It is not fun to find that your endGame could be a 100 dac + 100 amp hehehe.
I believe, this is how I feel, that we never lose the hope of finding another equipment that actually gives you the "wuooooo" effect again. There is always this question "but if I spend in another device 200$ more expensive that the one I have.... may be I will hear the difference?"
This was a dac test. I will do another one with amps in the future.

Chapeau! for putting your perceptions to the test. Most people are too lazy or too nervous to even try it.

Question: how did you do the level matching?

before the test I just care by simple hearing to have the 2 signals at the same volume. I know it is not perfect but at least should be quite random.

 

[majingotan]

Why not play with your brain memory for fun? This time, play 3-5 full music tracks level matched between the two DACs of course before switching then rely on your memory to see if you notice that "wuooooo" effect again

 

[Gabs]

Your Amiron Home may not manage to resolve the difference ! (I'm just kidding ;-) )

 

[digicidal]

My guess would be that the level matching wasn't close enough in the first test... and that had levels been closer you wouldn't have been able to determine the difference in test #1 either. The only other possibility that I could guess at in #3 might be that the phones output had enough extra noise to essentially "cover" whatever you were picking up on in test #1 but missing in #3. I'd say that's a very slim possibility in comparison to the level being just enough off in the first test however.

Great experience regardless... and you should look at it as already achieving your goals for a minimum of expense - as opposed to hoping for even better. Now if you want to replace the gear, you can worry more about features and aesthetics... which can cost you dearly depending on what they are in your case.

 

[Gedeon]

It's really a bitter pill to swallow realizing that this "hobby" can't evolve anymore around renewing electronics each few years, just because "new and clear improvements" have been reached out.

I guess that many of us have got a reasonably good enough setup which can't be easily improved, in ways we are able to perceive, just replacing/adding electronic boxes.

Maybe through speakers, maybe through active dsp filters, maybe room treatments,...

It's, more or less, like a final station in a journey. Time to plan the next one, but with pretty different goals.

 

[digicidal]

I've found spending more time actually listening to music can make swallowing that bitter pill far more palatable.

 

[L5730]

I am still in two minds and am trying to be diplomatic to this whole debate.
I haven't sampled enough equipment to make wider sweeping generalisations, so my experiences are limited in scope.

We have definitely heard differences between CD players when we have compared through the same amplifier and speakers. But I have some problems with our testing methods. (a) we didn't measure them, so there could be real measurable differences (b) there was no level matching performed (c) it was a sighted test (d) there was a brand bias against one.

I want to avoid coming across as trying to pick holes in your post, but I would just like to ask some questions and make some notes.

Pink Floyd's The Dark Side of the Moon is well known album for sure. It's quite dense dynamically and the silence is carpet bombed with organ drones. It's very much a production and will suffer from some high frequency loss and other distortions due to early tape bouncing. It just isn't 'audiophile' quality, despite this band being touted as just that, HiFi purchaser music.
Which version were you listening to? There are at least 14 different sounding versions on regular 16 bit 44.1 kHz CD!
The 2003 SACD hybrid is very bright and 'overly clear' (too much so) and is also dynamically limited for loudness.
My offer for arguments against using this as test material are apparent, but if it's the same music through different systems it really shouldn't matter so much. It just make make subtle differences a little harder to hear. I'm very open to that idea being challenged.

I really want to go into an expensive HiFi shop, armed with a multimeter or picoscope and some test music and have a listen. Sighted bias being an argument, but the fact I ain't buying might reduce that a bit. Level match and compare. I think a lot of CD players, streamers, DACs are all going to sound VERY similar once level matched. If they don't, the differences might be obvious once measured. I doubt they'd let me do that - but they should be doing this themselves!

Technology generally allows for the same or better performance in a smaller size and more energy efficient package. Valves > transistors (when not driven into distortion for artistic reason) being an example.
I am blown away by the performance of little boxes powered by 5v USB power. If we went into the 1980's and looked at the audio interfaces then, or analogue gear, the modern stuff is crazy how much better and smaller it is.

 

[Gedeon]

We are all humans. And we, usually, love to think we are gifted/special/talented, and even more when talking about something we love or we like.

Double ABX/Blind tests are usually really eye opener for many of us about "real" differences in gear when listening music.

But I'm also believe you can train yourself to catch minor differences it's totally possible. Through serious training, experience and a great (really great) amount of hours doing critical listening. But certainly, not the way I enjoy listening music. You can concentrate/inmerse a lot and enjoy. But doing in a clinical/analytic way, kills most of the enjoyment, it's exhausting (for me at least). I just think that, so serious/analytic/demanding approach isn't for me.

 

[digicidal]

Double ABX/Blind tests are usually really eye opener for many of us about "real" differences in gear when listening music.

But I'm also believe you can train yourself to catch minor differences it's totally possible. Through serious training, experience and a great (really great) amount of hours doing critical listening.

It's provable (at least to a large extent) - the Harman listener tests showed this, i.e. that trained listeners regardless of other factors were more able to differentiate variables. The only issue is that all of that, unless still properly blinded and level-matched can still easily be misled. Of course, as bad as our ears are at qualitative and quantitative analysis of distortion and noise - our brain is even worse (in the sense that it far too easily filters it out in most cases). Great as an evolutionary survival mechanism... not so great as an analytical tool for audio gear.

Totally agree with you about the "clinical analysis mode" of listening being tiring and not nearly as enjoyable as hearing potentially inferior SQ while relaxing with no express purpose outside of recreation. Plus it can lead to questioning even previously known values... which can lead directly to significant and unnecessary reductions in liquidity.

 

[JJB70]

We are all humans. And we, usually, love to think we are gifted/special/talented, and even more when talking about something we love or we like.

Double ABX/Blind tests are usually really eye opener for many of us about "real" differences in gear when listening music.

But I'm also believe you can train yourself to catch minor differences it's totally possible. Through serious training, experience and a great (really great) amount of hours doing critical listening. But certainly, not the way I enjoy listening music. You can concentrate/inmerse a lot and enjoy. But doing in a clinical/analytic way, kills most of the enjoyment, it's exhausting (for me at least). I just think that, so serious/analytic/demanding approach isn't for me.

Completely agree. The question is not just about whether their are discernible differences between equipment but whether such differences make a difference to enjoyment of music. If discerning differences is an exhausting process and requires such concentration by trained listeners I would consider them to be irrelevant in terms of using audio equipment to listen to music.

 

[L5730]

Well, yes enjoying listening to music and actually listen for the sake of comparison and analysis are two different games entirely.
I rarely buy things and not test them for the intended purpose with a scrutinising view.

I suppose however, it'd be like buying a car. Do we buy a family 5-door and then try to fit our family in and luggage just for the sake of proving to ourselves that it fits? Do we buy a sporty little number then take it to a disused air strip or track or rolling road and give it the beans to test it? I guess most usual vehicle purchases have fuel economy as one of the priorities, and I guess we do roughly assess if we are getting the fuel consumption we expect. If we upgrade to a more powerful vehicle, I guess we give it a bit of reserved thrash about, seeing how it accelerates on the dual carriage way or motorway. Can't open the taps and test top speed as it's way to easy to get picked up in the UK - leaving aside the arguments about driving at speed.

When we buy audio equipment or visual equipment, I think most of us tend to compare and contrast against what we already have. Is the noise floor quieter/are the blacks deeper? Is the contrast better/are the dynamics better demonstrated?

This is all off topic slightly though.
In my opinion everyone should do as the OP has done, test and analyse. Once satisfied, then it's about enjoying the music. If not satisfied > DSP to taste

 

[Blur]

DAC choice always matters. Form factor, connectors, EMI rejection, and especially output voltage nearing 2v RMS all are things to consider when purchasing either a new DAC or upgrading. I have tested many DAC's against each other and usually the biggest difference is that darn output voltage. The low output voltage DAC's make the amp suffer and thus audio fidelity to the headphones.

Amp's on the other hand have impedance issues, current / voltage output, and output power to all consider.

 

[velasfloyd]

I am still in two minds and am trying to be diplomatic to this whole debate.
I haven't sampled enough equipment to make wider sweeping generalisations, so my experiences are limited in scope.

We have definitely heard differences between CD players when we have compared through the same amplifier and speakers. But I have some problems with our testing methods. (a) we didn't measure them, so there could be real measurable differences (b) there was no level matching performed (c) it was a sighted test (d) there was a brand bias against one.

I want to avoid coming across as trying to pick holes in your post, but I would just like to ask some questions and make some notes.

Pink Floyd's The Dark Side of the Moon is well known album for sure. It's quite dense dynamically and the silence is carpet bombed with organ drones. It's very much a production and will suffer from some high frequency loss and other distortions due to early tape bouncing. It just isn't 'audiophile' quality, despite this band being touted as just that, HiFi purchaser music.
Which version were you listening to? There are at least 14 different sounding versions on regular 16 bit 44.1 kHz CD!
The 2003 SACD hybrid is very bright and 'overly clear' (too much so) and is also dynamically limited for loudness.
My offer for arguments against using this as test material are apparent, but if it's the same music through different systems it really shouldn't matter so much. It just make make subtle differences a little harder to hear. I'm very open to that idea being challenged.

I really want to go into an expensive HiFi shop, armed with a multimeter or picoscope and some test music and have a listen. Sighted bias being an argument, but the fact I ain't buying might reduce that a bit. Level match and compare. I think a lot of CD players, streamers, DACs are all going to sound VERY similar once level matched. If they don't, the differences might be obvious once measured. I doubt they'd let me do that - but they should be doing this themselves!

Technology generally allows for the same or better performance in a smaller size and more energy efficient package. Valves > transistors (when not driven into distortion for artistic reason) being an example.
I am blown away by the performance of little boxes powered by 5v USB power. If we went into the 1980's and looked at the audio interfaces then, or analogue gear, the modern stuff is crazy how much better and smaller it is.

Thats the reason I mainly only read this forum now. It is based on measurements and not reviewers and user's golden magical ears. What I wanted to test myself, in a simple way and not scientifically, is that even if some dacs measure different this would matter for me or not.

I have been enjoying my equipment for some month or years, but never until now put them into tests, because as someone wrote, it is a whole different listening mode.

I can say that, at least with Pink floyd, because I am listening to them almost everyday for the last 20 years, I can hear the difference between different cd's versions. In the past, with "normal" equipment (old walkman or mp3 in the computer, and 20$ headphones) every version sounded the same. One of the moments of most happiness in my life was the first time I listened my CD's through my new and first expensive headphone: The Amiron. It was like a new world for me. So I decided the "stupid" idea of repeating CD's for Pink Floyd. Let me explain myself:

I always found funny to see on the cds something like: "remastered under the band supervision" "newly remaster" etc. Because I thought it was some kind of false thing to sell more.... they all sounded the same to me always. But when I listened "obscured by clouds 2011 remaster cd"... wow!! it was different!!... I previously had a version of the year 2000, probably with some 90ty remaster, and that was the one I knew almost from memory...... for me it is kind of similar when you have a favourite movie that you saw 100 times in vhs, and then thanks to the internet or dvd or bluray you can see the cut scenes.... its like you can hear other details and stuff like that.

That effect made me re-purchase the Pink Floyd cds collection... neverending history, I want to have them all now!

in the specific case of Dark side of the moon, I have the 2011 remaster and the 1992 remaster (I think), and I prefer 1992 one, it is just better for me. (again, I only can hear the difference thanks to amiron). Most of the 2011 remasters sound better for me, but there are some exceptions: the wall, dark side of the moon.... so I did my tests with this version. Anyway, I do believe that as long as I use the same version for all the tests, this doest not matter really.

sorry my long texts and thanks everyone for reading and answer, great community here

 

[Gedeon]

DAC choice always matters. Form factor, connectors, EMI rejection, and especially output voltage nearing 2v RMS all are things to consider when purchasing either a new DAC or upgrading. I have tested many DAC's against each other and usually the biggest difference is that darn output voltage. The low output voltage DAC's make the amp suffer and thus audio fidelity to the headphones.

Amp's on the other hand have impedance issues, current / voltage output, and output power to all consider.

No one is saying the opposite, but most honest double blind/ABX tests with "competent enough" devices (which now, thanks to this website by example, can be found for way less money and way more "fidelity" than, let's say, 10 years ago) show no differences for almost everyone. That's the point.

Just waiting, some day, a brave team composed by honest and independent engineers/reviewers, willing to bite the bullet and able to design and perform objective review of "speakers" to get reliable measurements about their performing (like rtings is trying to do with headphones). We can't forget that speakers technology principles and solutions, hasn't really evolved "a lot" in last, ... 20-30 years ??

 

[garbulky]

Thats the reason I mainly only read this forum now. It is based on measurements and not reviewers and user's golden magical ears. What I wanted to test myself, in a simple way and not scientifically, is that even if some dacs measure different this would matter for me or not.

I have been enjoying my equipment for some month or years, but never until now put them into tests, because as someone wrote, it is a whole different listening mode.

I can say that, at least with Pink floyd, because I am listening to them almost everyday for the last 20 years, I can hear the difference between different cd's versions. In the past, with "normal" equipment (old walkman or mp3 in the computer, and 20$ headphones) every version sounded the same. One of the moments of most happiness in my life was the first time I listened my CD's through my new and first expensive headphone: The Amiron. It was like a new world for me. So I decided the "stupid" idea of repeating CD's for Pink Floyd. Let me explain myself:

I always found funny to see on the cds something like: "remastered under the band supervision" "newly remaster" etc. Because I thought it was some kind of false thing to sell more.... they all sounded the same to me always. But when I listened "obscured by clouds 2011 remaster cd"... wow!! it was different!!... I previously had a version of the year 2000, probably with some 90ty remaster, and that was the one I knew almost from memory...... for me it is kind of similar when you have a favourite movie that you saw 100 times in vhs, and then thanks to the internet or dvd or bluray you can see the cut scenes.... its like you can hear other details and stuff like that.

That effect made me re-purchase the Pink Floyd cds collection... neverending history, I want to have them all now!

in the specific case of Dark side of the moon, I have the 2011 remaster and the 1992 remaster (I think), and I prefer 1992 one, it is just better for me. (again, I only can hear the difference thanks to amiron). Most of the 2011 remasters sound better for me, but there are some exceptions: the wall, dark side of the moon.... so I did my tests with this version. Anyway, I do believe that as long as I use the same version for all the tests, this doest not matter really.

sorry my long texts and thanks everyone for reading and answer, great community here

Now that you've taken the blind test and seemiungly found no or extremely little difference.... try using the one you previously thought was worse for some time. Now you can see if the worse one still sounds the same as the one you thought was better (before the blind test).

 

[solderdude]

Now that you've taken the blind test and seemiungly found no or extremely little difference.... try using the one you previously thought was worse for some time. Now you can see if the worse one still sounds the same as the one you thought was better (before the blind test).

That's rather pointless as he knows what DAC he is listening to.
The trick is to not knowing what one listens to, even in long duration tests.