• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required as is 20 years of participation in forums (not all true). Come here to have fun, be ready to be teased and not take online life too seriously. We now measure and review equipment for free! Click here for details.

"Do digital audio players sound different playing 16/44.1 music?" - Listener Results.

Theo

Active Member
Joined
Mar 31, 2018
Messages
247
Likes
139
#3
this maybe considered as the conclusion :
"Despite the fact that a number of respondents could not hear a difference, for those who felt they could, it was certainly interesting to show that the data wasn't all just random. Even if the objective superiority of the Oppo UDP-205 and its ESS ES9038Pro "flagship" DAC could not be demonstrated to a significant degree in the listening test, the obvious objective and in turn subjective limitations of the computer motherboard were heard by a significant number."
 

edechamps

Active Member
Joined
Nov 21, 2018
Messages
200
Likes
518
Location
London, United Kingdom
#4
Good grief. This thing goes on and on and on. I gave up after the first page. Is there a summary of it?
The TL;DR is: there is no evidence that respondents could hear any difference between Apple iPhone 6, Oppo UDP-205 and Sony SCD-CE775. There is evidence that they preferred these over ASRock Z77 Extreme4, most likely because of the noise floor from the noisy PC.

If you ask me, these results are extremely unsurprising.
 

Tks

Senior Member
Joined
Apr 1, 2019
Messages
382
Likes
328
Location
NYC
#5
The TL;DR is: there is no evidence that respondents could hear any difference between Apple iPhone 6, Oppo UDP-205 and Sony SCD-CE775. There is evidence that they preferred these over ASRock Z77 Extreme4, most likely because of the noise floor from the noisy PC.

If you ask me, these results are extremely unsurprising.
Except the voodoo crowd will exagerate this testing to prove golden ears are definitely out there.
 

RayDunzl

Major Contributor
Central Scrutinizer
Joined
Mar 9, 2016
Messages
7,012
Likes
3,270
Location
Riverview, FL
#7
Last edited:

bennetng

Addicted to Fun and Learning
Joined
Nov 15, 2017
Messages
564
Likes
429
#8
In order to avoid oversimplification about his experiment, Archimago said:

"As you can see in last week's comments, it is quite possible that my nVidia GTX 1080 GPU card is a major source of the poor noise floor."

He was referring to my comment:
https://archimago.blogspot.com/2019...howComment=1557941747224#c3450090488343936775

The important things:
1. GPU card is not a required component in computer audio.
2. Archimago's ALC898 is better than my ALC892 in paper specs.
3. Digital filters in DACs are not as important as some people imagined, but low noise and high DNR within 20kHz are important:
https://archimago.blogspot.com/2019...showComment=1558250881181#c645515325341019845
 

edechamps

Active Member
Joined
Nov 21, 2018
Messages
200
Likes
518
Location
London, United Kingdom
#9
So did he test to see if people can tell the difference before asking for preference?
No, but I don't think that's necessary in that case, because the test is done blind and a statistical significance test is done on the preference rankings. If people couldn't tell the difference, one would see random preferences with no clear pattern and a high p-value, and that's indeed what we see (except for the motherboard audio). IANAS (I am Not A Statistician), but the approach seems fair to me.

To me, the biggest problem, from a study design perspective, is this:

Isn't that interesting? For each device, the largest number of "votes" was in the order of presentation! Many respondents, specifically the ones who thought there was "no noticeable difference" simply voted "A-B-C-D" to create this pattern. In fact, for those who thought there was "no noticeable difference", the "A-B-C-D" pattern of response from best to worst accounted for almost 80% of those votes. This is basically "noise" that needs to be filtered out if we are to hopefully understand the true preferences of those who felt they could hear a difference.
I don't know how Archimago "filtered out" that noise. It would have been best to design a protocol where the files are randomized for each respondent so that we don't have this problem.
 

Nemo

Member
Patreon Donor
Joined
May 9, 2019
Messages
15
Likes
13
Location
Spain
#11
I took part in this test, and for me the worst was clearly the iPhone, followed by the motherboard at some distance. I couldn’t quite decide reliably weather the Oppo or the “veteran” Sony player were the better (more accurate) player.

It was enlightening to hear different presentations in a completely unbiased environment and

1) realize the smaller-than-expected degree of difference and,
2) how difficult was to try to identify the most accurate presentation
 

JJB70

Major Contributor
Patreon Donor
Joined
Aug 17, 2018
Messages
1,304
Likes
2,349
Location
Milton Keynes, England
#12
I think that this is another excellent piece of work by Archimago and makes for very interesting reading. Most of it sort of confirms what I would expect but it is nevertheless useful to see studies like this.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
20,060
Likes
23,155
Location
Seattle Area
#13
I think any value in the test is lost in poorly legible graphs (as presented -- I know you can click on them for better view), and too long of an article. If it lost me, for sure it is going to lose others. No subjectivist is going to have the patience to go through all that is there to change their views. And any objectivist will have a hard time finding self-enclosed bit-sized data to quote in front of others.

I am not discounting the effort. Our goal in doing such work is to teach and inform. If these aspects are poorly done, then the end value is not there.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
20,060
Likes
23,155
Location
Seattle Area
#15
No, but I don't think that's necessary in that case, because the test is done blind and a statistical significance test is done on the preference rankings. If people couldn't tell the difference, one would see random preferences with no clear pattern and a high p-value, and that's indeed what we see (except for the motherboard audio). IANAS (I am Not A Statistician), but the approach seems fair to me.
Since these are take home tests, i.e. people listen with different gear and conditions, then the combination is not as statistically valid. You have more than one variable than the source file itself.
 

Tks

Senior Member
Joined
Apr 1, 2019
Messages
382
Likes
328
Location
NYC
#16
I think any value in the test is lost in poorly legible graphs (as presented -- I know you can click on them for better view), and too long of an article. If it lost me, for sure it is going to lose others. No subjectivist is going to have the patience to go through all that is there to change their views. And any objectivist will have a hard time finding self-enclosed bit-sized data to quote in front of others.

I am not discounting the effort. Our goal in doing such work is to teach and inform. If these aspects are poorly done, then the end value is not there.
To be honest, I didn't mind the length in the slightest.
 

mitchco

Active Member
Joined
May 24, 2016
Messages
223
Likes
541
#17
Archimago has done a tremendous job of slicing and dicing the data with very interesting points. He should be commended, not only on this effort, but his blog in general as one of the very few (in the world) that goes a long way to dispel digital audio myths. Other than ASR, there are very few sites that provide real measurements and in Arch's case the addition of listening tests so folks can hear with their own ears any differences...

@amirm you are in a unique position for not only measuring gear, and gathering a library of measurements, but if you recorded the output of these devices with music as well so people can actually hear any comparative sonic differences would be a worthwhile endeavour. Sure it is not easy, (I have done it more than a few times myself and know the effort well), but a "relative" comparison would be useful for many folks. It would also point out that for most folks any artefacts measuring below -72 dBFS (or 12 bits of resolution) can be correlated to likely being inaudible while listening over speakers or headphones at reference level (i.e. 77 to 83 dB SPL). This would help put the measurements in perspective to correlate what we hear and what is deemed transparent.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
20,060
Likes
23,155
Location
Seattle Area
#18
@amirm you are in a unique position for not only measuring gear, and gathering a library of measurements, but if you recorded the output of these devices with music as well so people can actually hear any comparative sonic differences would be a worthwhile endeavour. Sure it is not easy, (I have done it more than a few times myself and know the effort well), but a "relative" comparison would be useful for many folks.
This has been suggested a number of times before and in every case, I have stubbornly refused to go there. This time is no exception. :) Honestly, I can't keep up with life and reviews right now as it is. Uploading files and dealing with level differences, ground loops, etc. would add huge amount of work. For a single project it may be reasonable but for general reviews, I need to leave it to others who have the patience and time I don't have.
 

GGroch

Senior Member
Patreon Donor
Joined
Apr 7, 2018
Messages
392
Likes
476
Location
Denver, Colorado
#19
@amirm[/USER] you are in a unique position for not only measuring gear, and gathering a library of measurements, but if you recorded the output of these devices with music as well so people can actually hear any comparative sonic differences would be a worthwhile endeavour.....
The primary value of Archimago's test to me is that it was blind. I expect the results would be totally different if the listeners knew the products they were assessing.

If each of Amir's product measurements were followed with the recorded output, I am certain I would hear the measured defects found...whether they were actually audible or not. Sadly, I hear what I expect to hear.
 
Last edited:
Top Bottom