Wiim Ultra vs Wiim Ultra + External Dac (Blind Test Results)

[Mr.A]

Interesting that you use the funktion ones,two of the best pa’s I’ve heard used them,the last in a really dingy underground vaulted warehouse which had zero rights to sound so good,4 stacks of resolution with all the bass cabs etc,ears didn’t even hurt the next day.

Funktion Ones are amazing, no doubt! It's impressive how they can make even difficult spaces sound great. I'm really happy with my setup —just two speakers and a sub, but the clarity and detail are incredible!

 

[Mr.A]

And even though this was not fully scientific, even then this is not a statistically relevant percentage. It could easily be reached by random chance - a coin toss. The statistics of this stuff is counter intuitive.

Further - as I understand from your test description, each track was only judged once for each listener. It doesn't seem possible to make that statistically relevant.

Your objective is having fun - that is fine. But you are not discovering anything of any value.

I appreciate your perspective, and you're right that our test wasn't fully scientific or controlled. It was more about having fun and exploring the differences we could perceive in a casual setting. That said, while the information gathered may not hold up to strict statistical analysis, it’s still relevant to me and my experience. If it’s not relevant to you, that's fine. I'm not trying to convince you or anyone else, just sharing.

 

[Mr.A]

It's interesting how some people get upset or try to lecture you just for sharing a personal experience. In the end, I'm not trying to convince anyone or impose an opinion, just sharing what I've experienced. Everyone is free to draw their own conclusions, or not.

 

[amirm]

Very nice effort. More than many try to do.

 

[Mr.A]

Very nice effort. More than many try to do.

Thank you! I appreciate the kind words. I'm just trying to share my experiences and learn from others in the community. Engaging in positive and constructive discussions about audio gear is rewarding, as it not only helps gain new insights but also deepens the appreciation for the hobby.

 

[antcollinet]

I have the miniDSP as well... it doesn't offer the "Loudness" feature. It's just straight EQ. I prefer the RME over my MiniDSP by a HUGE margin.

Yeah - but that wasn't my point.

 

[Old_School_Brad]

And even though this was not fully scientific, even then this is not a statistically relevant percentage. It could easily be reached by random chance - a coin toss. The statistics of this stuff is counter intuitive.

Further - as I understand from your test description, each track was only judged once for each listener. It doesn't seem possible to make that statistically relevant.

Your objective is having fun - that is fine. But you are not discovering anything of any value.

It depends. 72 % alone isn’t necessarily statistically relevant without further context like sample size and repeatability. However, stating that it could be reached by random chance would be wrong - if the sample size is large enough it would indicate a real difference. -And the coin toss would approach 50 %.

In this case particular case 72 % doesn't indicate much.

Very nice effort. More than many try to do.

I agree.
It's commendable to even consider blind testing in this context. Instead of harsh criticism, we should encourage and gently suggest improvements if the goal is to obtain meaningful data.

 

[Mike_Trueman]

Nice test. You should record it and put in on youtube. So the WiiM is also a DAC. Looks like the Topping DAC sounds better in the top end then.

 

[yys310]

In this case particular case 72 % doesn't indicate much.

7/10 is very different from 70/100.

 

[JeremyFife]

Thanks for the suggestions! We definitely had a great time with some good music, and I appreciate the tips on how to make the test more meaningful. The idea of tightening up the process with level matching and improving the blinding sounds like a solid next step. We’re thinking about trying an AB/X test in the future to focus on identifying differences rather than preferences. If my friends are up for it, I’d love to give it another shot and see what we discover.

Just +1 thanks for taking the time to test like this, and sharing. Really appreciated, and interesting

 

[Overseas]

7/10 is very different from 70/100.

Sure.
Also very different from 3/10.

 

[yys310]

Sure.
Also very different from 3/10.

Guess I didn't make the point clear enough.

A:In a ABX test, the attendee got 43 correct out of 60. How likely it is really a different between A and B

ChatGPT:
In an ABX test, the goal is to determine if the attendee can reliably distinguish between two stimuli, A and B. To assess how likely it is that the attendee's correct guesses are due to a real difference between A and B (rather than just chance), we can perform a binomial test or calculate the p-value for the result.

In this case:

• The number of trials n=60
• The number of correct answers k=43
• Under the null hypothesis (assuming no difference between A and B), the probability of a correct answer in each trial is p=0.5, since it would be just a random guess.

The p-value will help us understand how likely it is that a person could have gotten 43 or more correct answers purely by guessing.
Let me calculate this for you.
The p-value is approximately 0.00053. This indicates that the probability of achieving 43 or more correct answers purely by chance (under the assumption that there's no difference between A and B) is very low—about 0.05%.
Since this p-value is well below the common significance threshold of 0.05, it suggests that it is very likely there is a real difference between A and B in the test, and the attendee is not simply guessing.

So it's not that 72% isn't representative enough. You need to deny the whole deny the whole experiment.

 

[BDWoody]

Nice test. You should record it and put in on youtube. So the WiiM is also a DAC. Looks like the Topping DAC sounds better in the top end then.

That's the thing... We can't really draw conclusions as the controls weren't tight enough.

It looks like it was a fun test, but not with enough rigor to be more than that.

That's the 'danger' with this kind of post, especially here at ASR, which is why you see so many pointing out what it is missing.

On one hand, we don't want to discourage the effort made or squelch the enthusiasm, but we also don't want to elevate it to something it isn't.

I'm hoping the encouragement to add some meaningful controls will lead to more tests that aren't likely to be as easily dismissed.

People willing to do anything like this are rare, and the OP has indicated an interest in learning more and moving forward, so let's see what the next test offers.

I also hope the OP (and anyone else trying to set up a listening test) will use the members here to help with the details of the next one, as it's not trivial to set up a test with enough rigor to be taken as anything more than anecdote.

That said, it is also really the only way to discover for yourself just how hard it is to tell these things apart.

For me, my audio life is divided into before and after I did some properly controlled testing. It really is worth going through the hassle at least once.

 

[Willem]

For me, my audio life is divided into before and after I did some properly controlled testing

For me that happened sometime in the late 1970s, when Quad's Peter Walker offered me a listening test of his three amplifier designs up to then. This was when audio pundits claimed that his early tube amplifier sounded best, and his most recent 405 current dumping amplifier worst. That irritated him enormously, so he commissioned James Moir to do a proper controlled blind listening test. The result was published in a scientific paper and demonstrated that there was no audible difference between the three designs when used within their power limitations. Peter also put together a demo system that he would occasionally use to offer people a chance to hear for themselves. Well, I thought I could hear differences, but Peter cheerfully showed me that I had been guessing randomly and was completely unable to reliably distinguish the amplifiers. Lesson learned. The only proviso that I later learned to appreciate is that of the power limitation. Underpowered amplifiers can indeed sound different.

 

[JeremyFife]

James Moir's test https://quadrevisie.nl/quadinfo/pdf/a29.pdf

Edit for context: Controlled listening test of three Quad amplifiers (valve and SS) carried out in 1978 ... guess what the conclusion was

 

[Avraham]

I recently got into audiophilia and posted a thread (

https://www.reddit.com/r/audiophile/comments/1fogb7d/wiim_ultra_external_dac

) asking whether adding an external DAC to my Funktion One Compact System (Wiim Ultra - Subwoofer SB10A - F81.2 Speakers) could improve it, especially the soundstage.

Most of the responses said that an external DAC would likely have little to no impact on the system.

However, given the debate it sparked, I decided to order a Topping D90 III Sabre DAC to test for myself and form an informed opinion.

The day it arrived, I set up a blind A/B test for my girlfriend using a Nobsound MC103 Pro splitter between two sources:

1: Wiim Ultra (RCA) → (XLR) Nobsound MC103 Pro → (XLR) SB10A Subwoofer → F81.2 Speakers.

2: Wiim Ultra (Optical) → Topping D90 III DAC (Optical) → (XLR) Nobsound MC103 Pro → (XLR) SB10A Subwoofer → F81.2 Speakers.

Before starting, I verified the system was calibrated using pink noise (I had to raise the left channel by +0.5 dB) and matched the output volume from both sources at 82 dB using a sound level meter.

The test consisted of playing 10 tracks from TIDAL (switching sources A/B during the song) and asking the listener to choose which one sounded better.

Here are the results:

Girlfriend: Chose the Topping DAC 7 out of 10 times as the better-sounding option.

Songs where she picked the Topping DAC: September (Earth, Wind & Fire), Sirius (Alan Parsons Project), Don’t Answer Me (Alan Parsons Project), Bubbles (Horikawa), Money (Pink Floyd), Money for Nothing (Dire Straits), Valerie (Amy Winehouse).

Songs where she didn’t: In the Air Tonight (Phil Collins), Dreams (Fleetwood Mac), Hotel California (Eagles).

Me: On the same day, I did the test and only picked the Topping DAC 5 out of 10 times.

Intrigued by the results (thinking my girlfriend’s picks might have been luck), I invited four friends (two men and two women) to a “DAC Party” a few days later for more testing.

Of the four friends, one is a sound engineer and professional musician (whose song is Grammy-nominated this year), and the other three are casual music listeners.

Professional Musician: Picked the Topping DAC 9 out of 10 times as the better-sounding option.

Songs where he picked the Topping DAC: Giorgio Moroder (Daft Punk), Bubbles (Horikawa), Rosanna (Earth, Wind & Fire), Money for Nothing (Dire Straits), Valerie (Amy Winehouse), Poem of Chinese Drum (Hok Man Yim), Hotel California (Eagles), Tea in the Sahara (The Police), Dreams (Fleetwood Mac).

The only song he picked incorrectly: Vivaldi: The Four Seasons.

Casual Listener 1: Chose the Topping DAC 8 out of 10 times.

Incorrect choices: Valerie (Amy Winehouse), Hotel California (Eagles).

Casual Listener 2: Chose the Topping DAC 7 out of 10 times.

Incorrect choices: Dreams (Fleetwood Mac), Poem of Chinese Drum (Hok Man Yim), Vivaldi: The Four Seasons.

Casual Listener 3: Chose the Topping DAC 7 out of 10 times.

Incorrect choices: Valerie (Amy Winehouse), Hotel California (Eagles), Money for Nothing (Dire Straits).

Overall Statistic: Out of 60 total comparisons across all listeners, the Topping D90 III DAC was chosen 43 times “correctly” as the better-sounding option, which translates to 72% of the time.

The reasons each listener gave for their choices were varied, but the general consensus was that the Topping D90 III Sabre DAC seemed slightly clearer and had more precise bass.

Despite the results, the overall opinion was that the differences between the two sources are very subtle and only noticeable in critical listening sessions—probably inaudible for most people in casual listening environments.

This test was done for fun, and the results are shared as an anecdote, not as scientific evidence of any conclusion.

After completing the test, I’m now faced with this dilemma: should I keep the Topping D90 III DAC, or return it and try a different DAC with a different technology (like the Denafrips Ares II) for another round of testing, purely for the fun of ‘chasing the perfect audio’?

Looking forward for questions and comments.

Get a Benchmark DAC3 HGC, and you'll never have to test again!

 

[Matt_Holland]

Underpowered amplifiers can indeed sound different.

This is a very interesting point. I’ve always believed I can hear subtle differences between audio components, except competent DACs/CD players which to me all sound incredibly similar. However I’ve always heard (at least believed to have heard) differences between amplifiers. I wonder if some of these differences were real and could be attributed to some of them being underpowered.

 

[Petrushka]

This is a very interesting point. I’ve always believed I can hear subtle differences between audio components, except competent DACs/CD players which to me all sound incredibly similar. However I’ve always heard (at least believed to have heard) differences between amplifiers. I wonder if some of these differences were real and could be attributed to some of them being underpowered.

A slight difference in volume will be heard as a difference in quality. Often repeated, but seldom believed.

 

[Matt_Holland]

A slight difference in volume will be heard as a difference in quality. Often repeated, but seldom believed.

My observations were always built up over extended listening rather than any controlled tests. Often products that I had preconceived thoughts on how they would sound performed very differently.

Anyway, I know that what I’ve heard is not actual differences due to biases etc.

My point really relates to amplifiers and whether the differences that people claim to hear could in some cases be down to some amps being underpowered.

 

[Petrushka]

My observations were always built up over extended listening rather than any controlled tests. Often products that I had preconceived thoughts on how they would sound performed very differently.

Anyway, I know that what I’ve heard is not actual differences due to biases etc.

My point really relates to amplifiers and whether the differences that people claim to hear could in some cases be down to some amps being underpowered.

Underpowered means distorting due to clipping.