Wiim Ultra vs Wiim Ultra + External Dac (Blind Test Results)

[Mr.A]

I am not defending the scientific validity of the test (I made it clear from the beginning that this was not the purpose); I am simply pointing out the ridiculous positions and arguments that the ‘most prominent armchair scientists of our century’ have put forward to attack a casual comparison
because they didn’t like the results.

I’m left with a positive and entertaining experience, and still to decide based on that whether to keep the DAC or purchase another one (or others) to continue comparing and having fun.

 

[Purité Audio]

Hardly ridiculous to insist that your comparison was conducted properly, there is already enough anecdote.
Keith

 

[kyuu]

I am not defending the scientific validity of the test (I made it clear from the beginning that this was not the purpose); I am simply pointing out the ridiculous positions and arguments that the ‘most prominent armchair scientists of our century’ have put forward to attack a casual comparison because they didn’t like the results.

It's not a matter of "not liking" the results. It's a matter of the results being invalid because proper controls weren't in place. If you just want to do that sort of thing for your own amusement, that's fine. No one is going to care in that case.

But if you then try to pretend that the results can be used to state that there actually was an audible difference between the two tested components, that's when you're going to get people riled.

 

[antcollinet]

Using the ad hominem fallacy signals a lack of objective grounds to refute a given conclusion.

Let alone the fact that previously you conveniently ignored responding to what the explanation would be for the 50% probability that the WiiM’s volume level was higher, yet listeners still preferred the Topping DAC 72% of the time.

The problem is - once you have inaccurate level matching you create an audible difference. Then it is down to preference. The expectation is that the slightly louder one will sound better but that isn't mandatory for every individual. It is just a confounder, you've created a difference that has nothing to do with the performance of the devices you are comparing - this then just mixes into all the other possibilities.

And are you saying you readjusted the level matching with every test? That's just another way to introduce confounders. Slightly different (and insufficiently accurate) level matching each time. And for another thing that would make it invalid to mix all the different people into one 72% result, because each test has a different test setup. Each test should then only stand on its own - in which case almost none of them give even close to a significant result.

 

[Mr.A]

Have you watched it? Because he wasn't comparing DACs, he was comparing filters.

Yes, and the conclusion he proposes (which could be extrapolated to the comparison of DACs) is that supposedly he can detect differences at theoretically inaudible levels, above 20kHz.

However, I didn’t share it for that reason but rather to demonstrate that those who have relentlessly criticized my homemade comparison would refute even that test, where theoretically everything is objectively controlled according to their scientific standards.

 

[Mr.A]

It's not a matter of "not liking" the results. It's a matter of the results being invalid because proper controls weren't in place. If you just want to do that sort of thing for your own amusement, that's fine. No one is going to care in that case.

But if you then try to pretend that the results can be used to state that there actually was an audible difference between the two tested components, that's when you're going to get people riled.

The only ones who got riled up were the most prominent armchair scientists of our century, who are incapable of reading something before criticizing it, because from the very first post, it was made clear that the purpose of the test was not to propose any scientifically valid conclusion.

 

[Purité Audio]

Reconstruction filters are measurably different and if you are young enough you may be able to discern a difference.
Keith

 

[Overseas]

I used about 7 DACs. 6 of them 'sound' the same, 1 not. I do not know exactly the measurements of them all, but they were surely close enough. Of course, no scientific testing...which makes me wonder how come the 6 sounded the same. Go figure.

 

[BDWoody]

I am simply pointing out the ridiculous positions and arguments that the ‘most prominent armchair scientists of our century’ have put forward to attack a casual comparison
because they didn’t like the results.

They didn't like your methodology. The results aren't meaningful, and go against a large body of knowledge that supports that position.

You are now directly insulting the membership (with quite a few actual scientists around, including in this thread trying in vain to help you), and you have no idea how out of your depth you are, or who you are talking to.

Either speak with more respect and in good faith, or you'll be moving along.

Maybe it's you who doesn't like being told your endeavor had no value beyond your own entertainment, and couldn't let yourself imagine that your results were meaningless?

I'm going to give you the rest of the week off to do more reading should you choose before you further beclown yourself.

 

[Sir Sanders Zingmore]

[

Ridiculous and nonsensical.

You demand perfect scientific evidence for propositions you don’t seem to like, yet with complete ease, you assume and take as valid baseless propositions, pulled out of thin air, that conveniently support your position, like the Topping had the higher volume 75% of the time.

You really ought to try a bit harder to understand why *your* use of statistics/probability is incorrect.
Think about it this way.
Given that 75% of people (or whatever it was) preferred the topping, what is the probability that the topping was the louder DAC (hint, it’s not 50%)

 

[kyuu]

Given that 75% of people (or whatever it was) preferred the topping, what is the probability that the topping was the louder DAC (hint, it’s not 50%)

And/or that the person doing the switching was inadvertently giving hints that the participants were responding to, since they were not blinded.

 

[JeremyFife]

For the sake of hypothesizing, let’s assume that the volume levels of both DACs were not perfectly balanced.

In that case, there would be a 50% probability that the WiiM’s volume was higher.

In that scenario (50% probable) what would be the explanation for the listeners preferring the Topping DAC?

You can't do that - you can't fix an uncontrolled result where biased preference is in play with retrospective maths. That's unfortunate as many experimental results could be improved otherwise.

Your study was fun and informative. It's reinforced for me, at least, the importance of strong controls and just how difficult it is to do that in real life.
I applaud you for giving it a go, and I'll be fascinated if you repeat your study with improved controls (if you have the stamina )

Don't be put out by some negative comments, don't mis-read some very useful comments that are genuinely intended to help you get it right next time.

Enjoy the music

 

[antcollinet]

[

You really ought to try a bit harder to understand why *your* use of statistics/probability is incorrect.
Think about it this way.
Given that 75% of people (or whatever it was) preferred the topping, what is the probability that the topping was the louder DAC (hint, it’s not 50%)

Here is another hint. Just because the probability of heads in a coin toss is 50%, it doesn't mean if you toss a coin 10 times, you'll get 5 heads and 5 tails. In fact the probability of exactly 5 heads and 5 tails is only 25%. And that is even if the method of level matching didn't bias the probability up or down from 50% (for example, granularity of volume steps between the two devices)

 

[Matt_Holland]

The constructive feedback for the OP has been excellent in detail, but at times lacking in tact. I’ve observed this so many times from school days, university and in my professional life, there is a personality trait common to highly intelligent, scientifically minded people whereby they can come across as extremely arrogant/patronising/intolerant to those they interact with. I can feel it in some of the responses in this thread.

It seems the OP has taken offence to the tone of some of the feedback and has unfortunately reacted badly and has undermined his case.

The breadth of knowledge, intellect and experience on ASR is incredible. If we want to endear ourselves to the lay-person who may be of the more subjective/hifi world we may, on occasion, need to take a more empathetic approach to communications.

I hope this comment causes no offence. I’m just disappointed at how this thread has developed (the OP being told off like a naughty boy and suspended from school). I really hope the OP does return and does redo the test with improved controls and we can all learn from the experience.

 

[Old_School_Brad]

The constructive feedback for the OP has been excellent in detail, but at times lacking in tact. I’ve observed this so many times from school days, university and in my professional life, there is a personality trait common to highly intelligent, scientifically minded people whereby they can come across as extremely arrogant/patronising/intolerant to those they interact with. I can feel it in some of the responses in this thread.

It seems the OP has taken offence to the tone of some of the feedback and has unfortunately reacted badly and has undermined his case.

The breadth of knowledge, intellect and experience on ASR is incredible. If we want to endear ourselves to the lay-person who may be of the more subjective/hifi world we may, on occasion, need to take a more empathetic approach to communications.

I hope this comment causes no offence. I’m just disappointed at how this thread has developed (the OP being told off like a naughty boy and suspended from school). I really hope the OP does return and does redo the test with improved controls and we can all learn from the experience.

I've noticed that as well, but I don't think the responsibility rests entirely on one side. In my view, people could benefit a lot from recognizing when someone has more expertise on a subject and just listening, even if the person comes across as arrogant or patronizing. Most of the time, it's not intended that way - it's simply that they want to stay on topic and make their point efficiently. It's rarely meant to be insulting. I think too many people are overly sensitive and take things personally when they shouldn't.

 

[ivanj]

Yes, and the conclusion he proposes (which could be extrapolated to the comparison of DACs) is that supposedly he can detect differences at theoretically inaudible levels, above 20kHz.

However, I didn’t share it for that reason but rather to demonstrate that those who have relentlessly criticized my homemade comparison would refute even that test, where theoretically everything is objectively controlled according to their scientific standards.

They only contradicted your test, they have not refuted it.

 

[GXAlan]

One of the criticisms/weaknesses of the group here is extreme bias that any sort of detectable difference in electronics is chalked up to poor controls or poor test conditions.

There was a whole thread where someone heard the difference between disc and file playback and the OP was criticized needlessly when reviewing the user manual pointed out that disc playback utilizes an upsampling/remastering algorithm and file playback via USB did not get upsampled.

We should be trying to figure out what possible scenarios could lead to these differences to be heard.

Here, if we look at the OP’s amplifier, it’s got a pretty low input impedance.

If the system uses capacitors for DC protection, and if the WiiM Ultra is going to the unbalanced TRS jack with only 4.7 kOhm of input impedance, that could lead to bass roll off compared to the D90 III which is running true balanced…

 

[solderdude]

After completing the test, I’m now faced with this dilemma: should I keep the Topping D90 III DAC, or return it and try a different DAC with a different technology (like the Denafrips Ares II) for another round of testing, purely for the fun of ‘chasing the perfect audio’?

When you are happy with the results and feel the test was valid you could keep the DAC.
When you have the funds and your friends will still be friends when you put them through the same wringer time and time again to 'very subtle and only noticeable in critical listening sessions' then by all means why stop here ?

Chasing the perfect audio, however IMHO, should not consist of picking DACs but picking recordings and above all transducers (in their circumstances).
A much bigger but more rewarding bees-nest when chasing perfect audio as the differences between competent DAC's are so small. Note... what's competent to person A may not be that for person B and not all DACs (or filters) are created equal.

I appreciate the effort even if it could be done a bit more scientific. This is REALLY hard to do 'correctly' b.t.w.

 

[Matt_Holland]

I've noticed that as well, but I don't think the responsibility rests entirely on one side. In my view, people could benefit a lot from recognizing when someone has more expertise on a subject and just listening, even if the person comes across as arrogant or patronizing. Most of the time, it's not intended that way - it's simply that they want to stay on topic and make their point efficiently. It's rarely meant to be insulting. I think too many people are overly sensitive and take things personally when they shouldn't.

I agree the responsibility should be on both sides and did point out the unfortunate way that the OP reacted.

On the whole ASR is a place of friendliness, learning and support, but this thread is not the best example that.

 

[davmol]

...
Me: On the same day, I did the test and only picked the Topping DAC 5 out of 10 times.
...
After completing the test, I’m now faced with this dilemma: should I keep the Topping D90 III DAC, or return it and try a different DAC with a different technology (like the Denafrips Ares II) for another round of testing, purely for the fun of ‘chasing the perfect audio’?

Going back to the original dilemma, given that you personally did not detect any difference at all, and it is an expensive DAC, I would return it. Still might be fun to try some other types of DAC, along with the improved controls, as long as you enjoy doing it.