• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

"Do digital audio players sound different playing 16/44.1 music?" - Listener Results.

Theo

Active Member
Joined
Mar 31, 2018
Messages
288
Likes
180
this maybe considered as the conclusion :
"Despite the fact that a number of respondents could not hear a difference, for those who felt they could, it was certainly interesting to show that the data wasn't all just random. Even if the objective superiority of the Oppo UDP-205 and its ESS ES9038Pro "flagship" DAC could not be demonstrated to a significant degree in the listening test, the obvious objective and in turn subjective limitations of the computer motherboard were heard by a significant number."
 

edechamps

Addicted to Fun and Learning
Forum Donor
Joined
Nov 21, 2018
Messages
910
Likes
3,620
Location
London, United Kingdom
Good grief. This thing goes on and on and on. I gave up after the first page. Is there a summary of it?

The TL;DR is: there is no evidence that respondents could hear any difference between Apple iPhone 6, Oppo UDP-205 and Sony SCD-CE775. There is evidence that they preferred these over ASRock Z77 Extreme4, most likely because of the noise floor from the noisy PC.

If you ask me, these results are extremely unsurprising.
 

Tks

Major Contributor
Joined
Apr 1, 2019
Messages
3,221
Likes
5,494
The TL;DR is: there is no evidence that respondents could hear any difference between Apple iPhone 6, Oppo UDP-205 and Sony SCD-CE775. There is evidence that they preferred these over ASRock Z77 Extreme4, most likely because of the noise floor from the noisy PC.

If you ask me, these results are extremely unsurprising.

Except the voodoo crowd will exagerate this testing to prove golden ears are definitely out there.
 

RayDunzl

Grand Contributor
Central Scrutinizer
Joined
Mar 9, 2016
Messages
13,202
Likes
16,982
Location
Riverview FL
Last edited:

bennetng

Major Contributor
Joined
Nov 15, 2017
Messages
1,634
Likes
1,692
In order to avoid oversimplification about his experiment, Archimago said:

"As you can see in last week's comments, it is quite possible that my nVidia GTX 1080 GPU card is a major source of the poor noise floor."

He was referring to my comment:
https://archimago.blogspot.com/2019...howComment=1557941747224#c3450090488343936775

The important things:
1. GPU card is not a required component in computer audio.
2. Archimago's ALC898 is better than my ALC892 in paper specs.
3. Digital filters in DACs are not as important as some people imagined, but low noise and high DNR within 20kHz are important:
https://archimago.blogspot.com/2019...showComment=1558250881181#c645515325341019845
 

edechamps

Addicted to Fun and Learning
Forum Donor
Joined
Nov 21, 2018
Messages
910
Likes
3,620
Location
London, United Kingdom
So did he test to see if people can tell the difference before asking for preference?

No, but I don't think that's necessary in that case, because the test is done blind and a statistical significance test is done on the preference rankings. If people couldn't tell the difference, one would see random preferences with no clear pattern and a high p-value, and that's indeed what we see (except for the motherboard audio). IANAS (I am Not A Statistician), but the approach seems fair to me.

To me, the biggest problem, from a study design perspective, is this:

Isn't that interesting? For each device, the largest number of "votes" was in the order of presentation! Many respondents, specifically the ones who thought there was "no noticeable difference" simply voted "A-B-C-D" to create this pattern. In fact, for those who thought there was "no noticeable difference", the "A-B-C-D" pattern of response from best to worst accounted for almost 80% of those votes. This is basically "noise" that needs to be filtered out if we are to hopefully understand the true preferences of those who felt they could hear a difference.

I don't know how Archimago "filtered out" that noise. It would have been best to design a protocol where the files are randomized for each respondent so that we don't have this problem.
 

Nemo

Active Member
Forum Donor
Joined
May 9, 2019
Messages
103
Likes
112
Location
Spain
I took part in this test, and for me the worst was clearly the iPhone, followed by the motherboard at some distance. I couldn’t quite decide reliably weather the Oppo or the “veteran” Sony player were the better (more accurate) player.

It was enlightening to hear different presentations in a completely unbiased environment and

1) realize the smaller-than-expected degree of difference and,
2) how difficult was to try to identify the most accurate presentation
 

JJB70

Major Contributor
Forum Donor
Joined
Aug 17, 2018
Messages
2,905
Likes
6,148
Location
Singapore
I think that this is another excellent piece of work by Archimago and makes for very interesting reading. Most of it sort of confirms what I would expect but it is nevertheless useful to see studies like this.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,389
Location
Seattle Area
I think any value in the test is lost in poorly legible graphs (as presented -- I know you can click on them for better view), and too long of an article. If it lost me, for sure it is going to lose others. No subjectivist is going to have the patience to go through all that is there to change their views. And any objectivist will have a hard time finding self-enclosed bit-sized data to quote in front of others.

I am not discounting the effort. Our goal in doing such work is to teach and inform. If these aspects are poorly done, then the end value is not there.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,389
Location
Seattle Area
Here is an example of a miss: why on earth start the article with how old respondents were?

Respondent%2BAge.png


This should be back-up/appendix type data.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,389
Location
Seattle Area
No, but I don't think that's necessary in that case, because the test is done blind and a statistical significance test is done on the preference rankings. If people couldn't tell the difference, one would see random preferences with no clear pattern and a high p-value, and that's indeed what we see (except for the motherboard audio). IANAS (I am Not A Statistician), but the approach seems fair to me.
Since these are take home tests, i.e. people listen with different gear and conditions, then the combination is not as statistically valid. You have more than one variable than the source file itself.
 

Tks

Major Contributor
Joined
Apr 1, 2019
Messages
3,221
Likes
5,494
I think any value in the test is lost in poorly legible graphs (as presented -- I know you can click on them for better view), and too long of an article. If it lost me, for sure it is going to lose others. No subjectivist is going to have the patience to go through all that is there to change their views. And any objectivist will have a hard time finding self-enclosed bit-sized data to quote in front of others.

I am not discounting the effort. Our goal in doing such work is to teach and inform. If these aspects are poorly done, then the end value is not there.

To be honest, I didn't mind the length in the slightest.
 

mitchco

Addicted to Fun and Learning
Audio Company
Joined
May 24, 2016
Messages
640
Likes
2,397
Archimago has done a tremendous job of slicing and dicing the data with very interesting points. He should be commended, not only on this effort, but his blog in general as one of the very few (in the world) that goes a long way to dispel digital audio myths. Other than ASR, there are very few sites that provide real measurements and in Arch's case the addition of listening tests so folks can hear with their own ears any differences...

@amirm you are in a unique position for not only measuring gear, and gathering a library of measurements, but if you recorded the output of these devices with music as well so people can actually hear any comparative sonic differences would be a worthwhile endeavour. Sure it is not easy, (I have done it more than a few times myself and know the effort well), but a "relative" comparison would be useful for many folks. It would also point out that for most folks any artefacts measuring below -72 dBFS (or 12 bits of resolution) can be correlated to likely being inaudible while listening over speakers or headphones at reference level (i.e. 77 to 83 dB SPL). This would help put the measurements in perspective to correlate what we hear and what is deemed transparent.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,368
Likes
234,389
Location
Seattle Area
@amirm you are in a unique position for not only measuring gear, and gathering a library of measurements, but if you recorded the output of these devices with music as well so people can actually hear any comparative sonic differences would be a worthwhile endeavour. Sure it is not easy, (I have done it more than a few times myself and know the effort well), but a "relative" comparison would be useful for many folks.
This has been suggested a number of times before and in every case, I have stubbornly refused to go there. This time is no exception. :) Honestly, I can't keep up with life and reviews right now as it is. Uploading files and dealing with level differences, ground loops, etc. would add huge amount of work. For a single project it may be reasonable but for general reviews, I need to leave it to others who have the patience and time I don't have.
 

GGroch

Major Contributor
Forum Donor
Joined
Apr 7, 2018
Messages
1,059
Likes
2,049
Location
Denver, Colorado
@amirm[/USER] you are in a unique position for not only measuring gear, and gathering a library of measurements, but if you recorded the output of these devices with music as well so people can actually hear any comparative sonic differences would be a worthwhile endeavour.....

The primary value of Archimago's test to me is that it was blind. I expect the results would be totally different if the listeners knew the products they were assessing.

If each of Amir's product measurements were followed with the recorded output, I am certain I would hear the measured defects found...whether they were actually audible or not. Sadly, I hear what I expect to hear.
 
Last edited:

mitchco

Addicted to Fun and Learning
Audio Company
Joined
May 24, 2016
Messages
640
Likes
2,397
@GGroch I understand what you are saying and agree. However, one can take two known files and ABX them using Foobar for example. So effectively blinded and one can determine if there is an audible difference, and if there is, what is your preference...
 
Top Bottom