Wiim Ultra vs Wiim Ultra + External Dac (Blind Test Results)

[JustJones]

Ridiculous and nonsensical.

You demand perfect scientific evidence for propositions you don’t seem to like, yet with complete ease, you assume and take as valid baseless propositions, pulled out of thin air, that conveniently support your position, like the Topping had the higher volume 75% of the time.

Ok, in a real test the volumes would be set with a multimeter so if the volume has 50% chance it's higher or lower with one device then no conclusion can be drawn as to preference. Your conclusion of 72% isn't worth a bucket of warm spit. Adios.

 

[BDWoody]

DAC’s Blind ABX Testing

Ok, if this is the direction you are choosing, this is going to go downhill quickly.

If you're here to try to convince us that your test was more than a fun time, rather than learning how to make it something useful to anyone, you are going to find a lot of frustration.

You really need to stop defending the validity of your test or your conclusions from it, and start figuring out how to make your test double blind, how to find and set the filters, and get a multimeter if you don't have one.

Did you read about Clever Hans where I linked it earlier? The fact that your test didn't blind the switcher to what was being switched in and out by itself is enough to be a problem. Combined with a level matching process that wasn't good enough, and you have a faulty test.

Please reconsider the hill you are defending. You've got lots of very well credentialed people trying to help in good faith. Let them.

 

[SIY]

Still waiting for an explanation as to what the reasoning would be for the 50% probability that the WiiM’s volume level was higher, yet listeners still preferred the Topping DAC 72% of the time.

Misuse of statistics.

Now go work on your controls.

 

[antcollinet]

DAC’s Blind ABX Testing

Have you watched it? Because he wasn't comparing DACs, he was comparing filters.

 

[Mr.A]

I am not defending the scientific validity of the test (I made it clear from the beginning that this was not the purpose); I am simply pointing out the ridiculous positions and arguments that the ‘most prominent armchair scientists of our century’ have put forward to attack a casual comparison
because they didn’t like the results.

I’m left with a positive and entertaining experience, and still to decide based on that whether to keep the DAC or purchase another one (or others) to continue comparing and having fun.

 

[Purité Audio]

Hardly ridiculous to insist that your comparison was conducted properly, there is already enough anecdote.
Keith

 

[kyuu]

I am not defending the scientific validity of the test (I made it clear from the beginning that this was not the purpose); I am simply pointing out the ridiculous positions and arguments that the ‘most prominent armchair scientists of our century’ have put forward to attack a casual comparison because they didn’t like the results.

It's not a matter of "not liking" the results. It's a matter of the results being invalid because proper controls weren't in place. If you just want to do that sort of thing for your own amusement, that's fine. No one is going to care in that case.

But if you then try to pretend that the results can be used to state that there actually was an audible difference between the two tested components, that's when you're going to get people riled.

 

[antcollinet]

Using the ad hominem fallacy signals a lack of objective grounds to refute a given conclusion.

Let alone the fact that previously you conveniently ignored responding to what the explanation would be for the 50% probability that the WiiM’s volume level was higher, yet listeners still preferred the Topping DAC 72% of the time.

The problem is - once you have inaccurate level matching you create an audible difference. Then it is down to preference. The expectation is that the slightly louder one will sound better but that isn't mandatory for every individual. It is just a confounder, you've created a difference that has nothing to do with the performance of the devices you are comparing - this then just mixes into all the other possibilities.

And are you saying you readjusted the level matching with every test? That's just another way to introduce confounders. Slightly different (and insufficiently accurate) level matching each time. And for another thing that would make it invalid to mix all the different people into one 72% result, because each test has a different test setup. Each test should then only stand on its own - in which case almost none of them give even close to a significant result.

 

[Mr.A]

Have you watched it? Because he wasn't comparing DACs, he was comparing filters.

Yes, and the conclusion he proposes (which could be extrapolated to the comparison of DACs) is that supposedly he can detect differences at theoretically inaudible levels, above 20kHz.

However, I didn’t share it for that reason but rather to demonstrate that those who have relentlessly criticized my homemade comparison would refute even that test, where theoretically everything is objectively controlled according to their scientific standards.

 

[Mr.A]

It's not a matter of "not liking" the results. It's a matter of the results being invalid because proper controls weren't in place. If you just want to do that sort of thing for your own amusement, that's fine. No one is going to care in that case.

But if you then try to pretend that the results can be used to state that there actually was an audible difference between the two tested components, that's when you're going to get people riled.

The only ones who got riled up were the most prominent armchair scientists of our century, who are incapable of reading something before criticizing it, because from the very first post, it was made clear that the purpose of the test was not to propose any scientifically valid conclusion.

 

[Purité Audio]

Reconstruction filters are measurably different and if you are young enough you may be able to discern a difference.
Keith

 

[Overseas]

I used about 7 DACs. 6 of them 'sound' the same, 1 not. I do not know exactly the measurements of them all, but they were surely close enough. Of course, no scientific testing...which makes me wonder how come the 6 sounded the same. Go figure.

 

[BDWoody]

I am simply pointing out the ridiculous positions and arguments that the ‘most prominent armchair scientists of our century’ have put forward to attack a casual comparison
because they didn’t like the results.

They didn't like your methodology. The results aren't meaningful, and go against a large body of knowledge that supports that position.

You are now directly insulting the membership (with quite a few actual scientists around, including in this thread trying in vain to help you), and you have no idea how out of your depth you are, or who you are talking to.

Either speak with more respect and in good faith, or you'll be moving along.

Maybe it's you who doesn't like being told your endeavor had no value beyond your own entertainment, and couldn't let yourself imagine that your results were meaningless?

I'm going to give you the rest of the week off to do more reading should you choose before you further beclown yourself.

 

[Sir Sanders Zingmore]

[

Ridiculous and nonsensical.

You demand perfect scientific evidence for propositions you don’t seem to like, yet with complete ease, you assume and take as valid baseless propositions, pulled out of thin air, that conveniently support your position, like the Topping had the higher volume 75% of the time.

You really ought to try a bit harder to understand why *your* use of statistics/probability is incorrect.
Think about it this way.
Given that 75% of people (or whatever it was) preferred the topping, what is the probability that the topping was the louder DAC (hint, it’s not 50%)

 

[kyuu]

Given that 75% of people (or whatever it was) preferred the topping, what is the probability that the topping was the louder DAC (hint, it’s not 50%)

And/or that the person doing the switching was inadvertently giving hints that the participants were responding to, since they were not blinded.

 

[JeremyFife]

For the sake of hypothesizing, let’s assume that the volume levels of both DACs were not perfectly balanced.

In that case, there would be a 50% probability that the WiiM’s volume was higher.

In that scenario (50% probable) what would be the explanation for the listeners preferring the Topping DAC?

You can't do that - you can't fix an uncontrolled result where biased preference is in play with retrospective maths. That's unfortunate as many experimental results could be improved otherwise.

Your study was fun and informative. It's reinforced for me, at least, the importance of strong controls and just how difficult it is to do that in real life.
I applaud you for giving it a go, and I'll be fascinated if you repeat your study with improved controls (if you have the stamina )

Don't be put out by some negative comments, don't mis-read some very useful comments that are genuinely intended to help you get it right next time.

Enjoy the music

 

[antcollinet]

[

You really ought to try a bit harder to understand why *your* use of statistics/probability is incorrect.
Think about it this way.
Given that 75% of people (or whatever it was) preferred the topping, what is the probability that the topping was the louder DAC (hint, it’s not 50%)

Here is another hint. Just because the probability of heads in a coin toss is 50%, it doesn't mean if you toss a coin 10 times, you'll get 5 heads and 5 tails. In fact the probability of exactly 5 heads and 5 tails is only 25%. And that is even if the method of level matching didn't bias the probability up or down from 50% (for example, granularity of volume steps between the two devices)

 

[Matt_Holland]

The constructive feedback for the OP has been excellent in detail, but at times lacking in tact. I’ve observed this so many times from school days, university and in my professional life, there is a personality trait common to highly intelligent, scientifically minded people whereby they can come across as extremely arrogant/patronising/intolerant to those they interact with. I can feel it in some of the responses in this thread.

It seems the OP has taken offence to the tone of some of the feedback and has unfortunately reacted badly and has undermined his case.

The breadth of knowledge, intellect and experience on ASR is incredible. If we want to endear ourselves to the lay-person who may be of the more subjective/hifi world we may, on occasion, need to take a more empathetic approach to communications.

I hope this comment causes no offence. I’m just disappointed at how this thread has developed (the OP being told off like a naughty boy and suspended from school). I really hope the OP does return and does redo the test with improved controls and we can all learn from the experience.

 

[Old_School_Brad]

The constructive feedback for the OP has been excellent in detail, but at times lacking in tact. I’ve observed this so many times from school days, university and in my professional life, there is a personality trait common to highly intelligent, scientifically minded people whereby they can come across as extremely arrogant/patronising/intolerant to those they interact with. I can feel it in some of the responses in this thread.

It seems the OP has taken offence to the tone of some of the feedback and has unfortunately reacted badly and has undermined his case.

The breadth of knowledge, intellect and experience on ASR is incredible. If we want to endear ourselves to the lay-person who may be of the more subjective/hifi world we may, on occasion, need to take a more empathetic approach to communications.

I hope this comment causes no offence. I’m just disappointed at how this thread has developed (the OP being told off like a naughty boy and suspended from school). I really hope the OP does return and does redo the test with improved controls and we can all learn from the experience.

I've noticed that as well, but I don't think the responsibility rests entirely on one side. In my view, people could benefit a lot from recognizing when someone has more expertise on a subject and just listening, even if the person comes across as arrogant or patronizing. Most of the time, it's not intended that way - it's simply that they want to stay on topic and make their point efficiently. It's rarely meant to be insulting. I think too many people are overly sensitive and take things personally when they shouldn't.

 

[ivanj]

Yes, and the conclusion he proposes (which could be extrapolated to the comparison of DACs) is that supposedly he can detect differences at theoretically inaudible levels, above 20kHz.

However, I didn’t share it for that reason but rather to demonstrate that those who have relentlessly criticized my homemade comparison would refute even that test, where theoretically everything is objectively controlled according to their scientific standards.

They only contradicted your test, they have not refuted it.