• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required as is 20 years of participation in forums (not all true). Come here to have fun, be ready to be teased and not take online life too seriously. We now measure and review equipment for free! Click here for details.

Measured Differences Between Software Audio Players

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
28,163
Likes
72,430
Location
Seattle Area
#1
In another thread a discussion of measured differences between audio players broke out. I thought it should have its own thread.

I have documented the same in one of my articles: http://www.audiosciencereview.com/f...puter-activity-can-impact-dac-performance.22/

Here is the Windows Media Player output of the DAC:



And the same file being played by Media Player Classic:



The difference was traced to hard disk activity bleeding out of the computer into DAC and then its analog output.

This however did NOT make a case for using different audio players. The problem was poor DAC implementation by Shiit, providing so little isolation between its USB input and analog DAC output. While I did not test that, my son had observed that he could "hear" his computer activity while playing games through this DAC.

This brings us to a problem with the argument with using "audiophile players." Namely the fact that it is the high-end audio users that resort to them. Yet the same user is likely to have a high-end DAC with proper isolation/async USB input, in no need of using a different audio player.

Using alternative audio players may forces you often to use more convoluted audio paths and/or less usable players. So their value needs to be there before I would jump on them.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
28,163
Likes
72,430
Location
Seattle Area
Thread Starter #2
As I mentioned, this was posted in another thread as proof of jitter differences between two different audio players:

-----
post by "bibo01,

I quote this, but my reply is for anyone involved.
We at Italian NextHardware.com have been measuring and discussing differences in media players. We have spent over 80 pages at doing that (I can give link if required) and, although the thread went astray, our experience and tests are not over.
Let me tell you what happened: one of our technician users was surprised to hear in his system differences between media players - Foobar, Daphile and WTF - all playing bit-perfect material.
After making sure that those differences actually existed through controlled blind testing with various people involved, we started to measure looking for differences. Testing protocols were applied and various people were involved to confirm results. All software players were playing on the same PC, captures were repeated at least three times under differences circumstances.
To make story short, Foobar (and its PC and OS) produced more jitter than WTF, which somehow influenced the DAC USB receiver, even though this was powered separately from PC and galvanic isolated (supposedly).
The following is a brief demonstration of results:
Foobar:


WTF
:


Testing instruments was recently certified:


All this is to say that with the right gear, expertise and time in hand it is possible to measure transport differences, hardware and/or software, but that most of the times it has relevance to the place it was measured only.

PS: I must admit that without Blizzard/Mivera this thread has no sense anymore and its latest discussion can be taken elsewhere.
--------------------

Bib provided a link to a thread that explains what was tested but it is 52 pages and with google translation, it is painful to read. So I don't know what is being tested.

From what I can see, this is time domain output of some signal. The scope is capturing all the paths the trace took and providing the differences in shaded green. Unlike my measurements, they are not in frequency domain where we can see the spectrum of variations. This is important because the only jitter influence that we care about is in audible band. Variations say at 1 Mhz is of no import to us because we don't hear them. The time domain information does not give us this critical data that can be used with psychoacoustics to inform us of audibility of said distortions.

Of course if this is showing changes in a digital signal then it is not useful at all as we care about what comes out of the DAC, not what goes into it.

And a bit on such measurements using an oscilloscope. The certification above may give one a false sense of reliance on its data. In general, scopes do not make good audio measurement instruments. These digital scopes digitize the input signal using very high-speed Analog to Digital Converters (ADCs). They typically run at > 1 Ghz -- orders of magnitude faster than we need for audio -- and as a compromise have very reduced dynamic range. The ADC may just be 10 to 12 bits. The scope used in this testing is no exception. It is a 20 year old (long obsolete) scope. I could not find its ADC resolution but I am pretty confident it is not even 16 bits let alone 24 bits that are used in audio instruments. So calibrated or not, it is not an instrument of choice for making DAC measurements.

So Bib, would you please provide background info on what signal is being measured here and how you determined it indicates audible differences?
 

Sal1950

Major Contributor
The Chicago Crusher
Forum Donor
Joined
Mar 1, 2016
Messages
7,731
Likes
6,237
Location
Central Fl
#5
As I mentioned, this was posted in another thread as proof of jitter differences between two different audio players:


Testing instruments was recently certified:



So Bib, would you please provide background info on what signal is being measured here and how you determined it indicates audible differences?
Is that OS Windows XP?
 

bibo01

Active Member
Joined
Mar 10, 2016
Messages
105
Likes
11
#6
...
So Bib, would you please provide background info on what signal is being measured here and how you determined it indicates audible differences?
Testing has been carried out according to your concept of resolution (thus with performance very similar to an AP2700). The test with oscilloscope was added to see if even with a simple triangle wave at 1 kHz (DAC DUT does not reproduce square waves) would also show asymmetries in the waveform. That is exactly what happens (using the same signal) sampling at 24bit and analyzing the signal with software oscilloscopes.
In simple terms, it was added another analysis to the numerous previous analysis.

...For further info please look in this thread on CA as I am responding to questions on the whole procedure.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
28,163
Likes
72,430
Location
Seattle Area
Thread Starter #7
There are 55 pages in that thread. I can't understand why if the discussion is posted here, you can't give us specific answers here rather a link to such a long thread elsewhere. I am game for reading external papers but not such huge threads. At least give a summary here or a link to summary post there. If you are not interested in doing so that is fine. We can move on and leave your argument unproven.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
28,163
Likes
72,430
Location
Seattle Area
Thread Starter #8
To make story short, Foobar (and its PC and OS) produced more jitter than WTF, which somehow influenced the DAC USB receiver, even though this was powered separately from PC and galvanic isolated (supposedly).
Testing has been carried out according to your concept of resolution (thus with performance very similar to an AP2700). The test with oscilloscope was added to see if even with a simple triangle wave at 1 kHz (DAC DUT does not reproduce square waves) would also show asymmetries in the waveform.
Working with what you have provided, the scope graphs are incapable of differentiating between jitter, reference modulation and noise. This is why we perform the spectrum analysis. Jitter will be symmetrical and frequency dependent allowing us to identify it as such. So not sure how you arrived at them being jitter in this display.

Do you have a measurement that shows jitter and its spectrum?
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
28,163
Likes
72,430
Location
Seattle Area
Thread Starter #9
Testing has been carried out according to your concept of resolution (thus with performance very similar to an AP2700). The test with oscilloscope was added to see if even with a simple triangle wave at 1 kHz (DAC DUT does not reproduce square waves) would also show asymmetries in the waveform. That is exactly what happens (using the same signal) sampling at 24bit and analyzing the signal with software oscilloscopes.
Sorry, no. Here is the data sheet for the Keysight (Agilent) scope used: http://literature.cdn.keysight.com/litweb/pdf/5980-2397EN.pdf?id=1000070291:epsg:dow

"Resolution2 Real Time 8 bits (0.4% of full scale), 12 bits or greater with sufficient averaging"

No way is that equiv. to 24 bit converters used in Audio Precision or any other audio analyzer.

Dither is added before digitizing the signal. Combined with such low bit resolution, much of what you show in your graphs could be due to the scope's highly limited vertical resolution (compared to Audio Precision analyzer).

I use Audio Precision analyzer not because it is the greatest thing, but because it is a known quantity. When you use some other instrument not meant for this purpose you need to first qualify and quantify its capabilities. It is improper to throw out some graphs from it that most people don't understand and then throw a certificate of calibration to say, it must be good enough for this purpose.

The calibration in this case is totally unnecessary anyway because you are just performing AB comparisons. The difference would show up regardless of whether the absolute values are right or wrong.

As I said in the outset, it is pretty possible to show differences with the players using the "right" DAC which is sensitive to such things. I like to see tests with high-end DACs that people routinely use that demonstrates such vulnerability. Otherwise if you just want to run with a talking point, use my graphs as they make the point hell of a lot better than your measurements. :)

From your read of the post on CA forum, this is the DAC you used: http://jlsounds.com/ak4490.html



This is even more lower-end product than what I tested in my OP. Buy, borrow or steal an MSB DAC and show such differences and then it will be material to routine customer who thinks they should resort to these players. Not a DIY, $50 board.
 

Blumlein 88

Major Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
9,600
Likes
12,784
#10
There are 55 pages in that thread. I can't understand why if the discussion is posted here, you can't give us specific answers here rather a link to such a long thread elsewhere. I am game for reading external papers but not such huge threads. At least give a summary here or a link to summary post there. If you are not interested in doing so that is fine. We can move on and leave your argument unproven.
Post #1346 has some useful info from Bibo01 in that CA thread he linked. Post #1350 has links to some of the more pertinent discussion at the Nexthardware forums in Italian. Use the Chrome browser for those and it will offer to translate each page for you. Though the translation by google is surprisingly crude.
 

Blumlein 88

Major Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
9,600
Likes
12,784
#11
Okay I decided to copy and paste from CA so hopefully bibo01 doesn't object. So here is his explanation there.


I will explain the whole procedure.

The DAC model is a JLSounds with AK4490, reclock upgrade to low noise xo and three separate power supplies.

Numerous acquisitions were made (to check the repeatability of the phenomenon) directly from DAC analog outputs.

Initially the signal used (as stimulus) was a multi-tone 10 octaves and 120 sinusoids (bandwidth from 16 Hz to 16324 Hz so Audio Diffmaker operates a correct alignment). For each octave there are 12 semitones, with a crest factor of about 4.
This signal, as opposed to a piece of music, tends to increase the phenomena related to non-linear changes (distortions) so one can see them better.

All acquisitions made by the two players in question (Foobar and WTF) produced a repeatable and consistent difference through a NTD analysis.
The evidenced phenomenon is a mix of delay and jitter (as waveform symmetry is not only shifted but also deformed).
The digitized signal (with sampling at 16bit and 24bit) was viewed through the oscilloscope software "Visual Analyzer", clearly visible in the original 3D on Nexthardware.com.

These analyses would have been enough already because the NTD showed clear and repeatable differences on all acquired signals.

To facilitate testing you can get hold of a high-performance A/D acquisition system (it must guarantee a very high repeatability over time) and acquire this signal for at least 3 times on each player; then eventually you could email them to me so I can collaborate with you in search of any differences.

In this analysis I also wanted to add another test using a symmetrical signal such as a triangular 1 kHz wave.
This signal was created with 32bit/192kHz resolution, it was reproduced by both Foobar and WTF, acquired from the analog DAC outputs and digitized.
Observing the difference through Visual Analyzer, even with this signal the same phenomena detected previously with multi-tone signal occurred - that is delay + jitter.

Finally, to add another test, the acquired signals were reproduced by a D/A and analyzed by oscilloscope in real-time, windowing as seen in the graphs I previously posted and sampling at 5MSa/sec.

Of course, one adds a little error (but known and constant) due to D/A conversion, but it does not compromise the clear difference that occurs between the two signals derived from each respective player - Foobar and WTF.

It goes without saying that I can provide you with both the original signals (those used as reference) and those acquired.
 

Blumlein 88

Major Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
9,600
Likes
12,784
Joined
Jul 26, 2016
Messages
13
Likes
0
#13
Testing has been carried out according to your concept of resolution (thus with performance very similar to an AP2700). The test with oscilloscope was added to see if even with a simple triangle wave at 1 kHz (DAC DUT does not reproduce square waves) would also show asymmetries in the waveform. That is exactly what happens (using the same signal) sampling at 24bit and analyzing the signal with software oscilloscopes.
In simple terms, it was added another analysis to the numerous previous analysis.

...For further info please look in this thread on CA as I am responding to questions on the whole procedure.
Not even close to Amir's tests, just junk science in comparison.
 
Joined
Jul 26, 2016
Messages
13
Likes
0
#14
Sorry, no. Here is the data sheet for the Keysight (Agilent) scope used: http://literature.cdn.keysight.com/litweb/pdf/5980-2397EN.pdf?id=1000070291:epsg:dow

"Resolution2 Real Time 8 bits (0.4% of full scale), 12 bits or greater with sufficient averaging"

No way is that equiv. to 24 bit converters used in Audio Precision or any other audio analyzer.

Dither is added before digitizing the signal. Combined with such low bit resolution, much of what you show in your graphs could be due to the scope's highly limited vertical resolution (compared to Audio Precision analyzer).

I use Audio Precision analyzer not because it is the greatest thing, but because it is a known quantity. When you use some other instrument not meant for this purpose you need to first qualify and quantify its capabilities. It is improper to throw out some graphs from it that most people don't understand and then throw a certificate of calibration to say, it must be good enough for this purpose.

The calibration in this case is totally unnecessary anyway because you are just performing AB comparisons. The difference would show up regardless of whether the absolute values are right or wrong.

As I said in the outset, it is pretty possible to show differences with the players using the "right" DAC which is sensitive to such things. I like to see tests with high-end DACs that people routinely use that demonstrates such vulnerability. Otherwise if you just want to run with a talking point, use my graphs as they make the point hell of a lot better than your measurements. :)

From your read of the post on CA forum, this is the DAC you used: http://jlsounds.com/ak4490.html



This is even more lower-end product than what I tested in my OP. Buy, borrow or steal an MSB DAC and show such differences and then it will be material to routine customer who thinks they should resort to these players. Not a DIY, $50 board.
This must be junk compared to your equipment.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
28,163
Likes
72,430
Location
Seattle Area
Thread Starter #15
This must be junk compared to your equipment.
It is not that it is "junk." It is just designed for a different use than analyzing low order bits of audio signal. As it says, it is limited to 0.4% accuracy which as we all know, is well above distortion of many audio products.

On the other hand, it samples at 1 Gigasmples/second so it can analyze signals to 500 Mhz. My audio analyzer only goes up to 260 Ksamples/sec and is only good to 130 Khz. It has far more bandwidth than my analyzer but as a trade off, far less dynamic range.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
28,163
Likes
72,430
Location
Seattle Area
Thread Starter #16
Post #1346 has some useful info from Bibo01 in that CA thread he linked. Post #1350 has links to some of the more pertinent discussion at the Nexthardware forums in Italian. Use the Chrome browser for those and it will offer to translate each page for you. Though the translation by google is surprisingly crude.
Thanks. I did eventually find that post and hence my comments about the DAC he used. :)
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
28,163
Likes
72,430
Location
Seattle Area
Thread Starter #17
I have glanced through those also. The best I can tell these are differential signals out of the players, diced different ways.

The problem with differential analysis is that it is very hard to do as their arguments back and forth shows. Analog output of the DAC is digitized with a clock that is different the second time around, together with the ADC itself acting differently. Dialing all of that out is hard.

The much bigger problem is audibility. Differential signals are psychoacoustically blind. A lot of difference for examples shows up when the signal is louder. But the louder the signal, the harder it is to hear any artifacts. Do a differential analysis with output of lossy audio codecs. You can readily hear those differences (shame they are not uploading the above differentials for us to listen). Yet when you play the music, finding those differences can be impossibly hard for non-trained listeners due to masking.

Mind you, I am in favor of finding these differences and reporting on them. To do that though, first the hardware used has to be quantified. The DAC in use is in no way appropriate since it is a random product and who knows how stable its output is. For example if it is a bare board, the PC activity could leak onto it outside of the USB input. The instrumentation has to be validated, etc.

And to repeat, it is easy to show that PC activity does bleed into some DACs as I showed in my article. Nothing is new there (except to orthodox objectivists). The key is to take high performance commercial DACs and demonstrate the same. Using scopes bought on ebay and random DIY tools on hobby board is not going to convince anyone of anything useful.
 

Blumlein 88

Major Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
9,600
Likes
12,784
#18
I have glanced through those also. The best I can tell these are differential signals out of the players, diced different ways.

The problem with differential analysis is that it is very hard to do as their arguments back and forth shows. Analog output of the DAC is digitized with a clock that is different the second time around, together with the ADC itself acting differently. Dialing all of that out is hard.

The much bigger problem is audibility. Differential signals are psychoacoustically blind. A lot of difference for examples shows up when the signal is louder. But the louder the signal, the harder it is to hear any artifacts. Do a differential analysis with output of lossy audio codecs. You can readily hear those differences (shame they are not uploading the above differentials for us to listen). Yet when you play the music, finding those differences can be impossibly hard for non-trained listeners due to masking.

Mind you, I am in favor of finding these differences and reporting on them. To do that though, first the hardware used has to be quantified. The DAC in use is in no way appropriate since it is a random product and who knows how stable its output is. For example if it is a bare board, the PC activity could leak onto it outside of the USB input. The instrumentation has to be validated, etc.

And to repeat, it is easy to show that PC activity does bleed into some DACs as I showed in my article. Nothing is new there (except to orthodox objectivists). The key is to take high performance commercial DACs and demonstrate the same. Using scopes bought on ebay and random DIY tools on hobby board is not going to convince anyone of anything useful.
Yes all this is reasonably close to my thinking of it. Difference testing is sometimes startlingly effective at finding differences and in other cases horribly misleading. I do believe bibo01 said we could send him files we recorded and they would analyze them the same way. So someone could use higher quality ADC and DAC to see if the result is repeatable.
 

bibo01

Active Member
Joined
Mar 10, 2016
Messages
105
Likes
11
#19
amir,

you always put in evidence that Audio Precision of yours which for accurate analysis in the time domain cannot compete with any oscilloscope having bandwidth over 500 kHz (the AP has limited bandwidth and it cannot trigger jittered signals accurately).
If you want to talk about "vertical" resolution, then is this toy going to be limited too?!
http://literature.cdn.keysight.com/litweb/pdf/5990-5271EN.pdf?id=1887887

We are talking about oscilloscopes born exactly for jitter analysi. They also have a vertical resolution of 8bit and 12bit on average, however, with a bandwidth of 33 GHz!
If we talk about analysis in the frequency domain, there is no doubt that the instrument to be used is the Audio Precision (or equivalent, like I do), but if we have to work in the time domain, the AP is not the right tool.

Re. DUT, the hardware is not mine - I was only asked to make some tests. I believe that JLSound is the maker of USB receiver/reclocker. The rest, I am not sure - I know it is made by a professional person.
 
OP
amirm

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
28,163
Likes
72,430
Location
Seattle Area
Thread Starter #20
Bib I don't mind you using whatever instrument. My issue is putting a measurement forward, using non-conventional means and say, "there is proof." The timing accuracy of these scopes is only in the X (time) axis, not Y. Your measurements show both. In digital systems/signals, X axis/time jitter is all that we worry about. In the case of analog output of the DAC, we care about both X and Y axis. Noise riding on top of the signal for example can only be identified if the scope has lower noise than the source which is not possible here given that the DAC is 24 bits and the scope is 8.

The AP within the audio band we care about, is performant in both X and Y axis or it would not be able to show such amazingly low amount of distortion as I measured with my ancient Mark Levinson 360S DAC but using Berkeley USB to AES converter:
Mark Levinson No 360S.PNG


Any jitter in time domain manifests itself in frequency domain as sidebands spikes. It must, must do that mathematically. Using averaging we are able to substantially reduce the noise floor further (that is how 8 bits becomes 12 in the scope used in your case). That allows us to dig so deep to find distortions here down at -123 dbFS. And importantly we can make judgement about their audibility which in this case is nil.

These points aside, did you notice that the scope you did use runs Windows XP? And it runs it right inside this sensitive instrument? If computers are full of badness, how did they manage to put a then hot and noisy computer right inside the instrument??? Answer is simple: good engineering. If anyone can hide the noise of a PC inside the instrument with ADC, why is it so hard to assume that someone can hide the noise of the incoming USB input into the DAC??? USB interface has a fraction of the signal complexity and noise of any computer.

My cell phone has incredible amount of computing horsepower in it with tons of activity, switching power supplies everywhere, no less than three radios (cell, Wifi, and Bluetooth), yet when I listen to music, I don't hear their activities despite such close proximity. How did they manage that? Answer: good engineering!

If you want examples of bad engineering being sensitive to PC activity, use my measurements. They are clear, done using Audio Precision, and distortions/noise rise up to level of audibility. And folks on CA forum cherish that brand of DAC far more than the one your friend used. But if you want to make a point here, then we need something better than what I already documented.

As I said at the outset, there is a problem with this whole line of reasoning. Average person is not going to give up the convenience of iTunes and use some other audiophile player. The target market is high-end customers with highly performant DACs with amazing engineering where cost is no barrier. Such DACs by definition are highly resistant to noise coming into them. Use whatever scope you want: show that the output of these DACs is influenced by software players and then we have something to talk about.

Many audio tweaks do no harm. This one, does a ton of harm in forcing someone to give up their favorite player with all the functionality they want, to chase another one for the cause of "better sound." Before we send them to such dire destination, we better be damn sure that there is some kind of improvement.
 
Last edited:
Top Bottom