E1DA 9038D performance according to df-metric

[Serge Smirnoff]

We’re a bit late to this party, but that’s mainly because we spent a lot of time exploring this device from various angles. We ultimately reached the same conclusion and so figured at this point we may as well share what we’ve found.

We ended up testing two of these devices. We tried tweaking the various coefficients and filters in the companion app, applying our standard 32 Ohm load vs no load, varying output power, etc. Nothing significantly changed our results.
TL;DR – We don’t want to rain on anybody’s parade, but we found the 9038D to perform worse than most other ESS-DAC devices. You can see where it sits in the hierarchy of DACs we’ve measured here (default rank is by the median waveform error):

DAP TI Database, HypetheSonics

Digital Audio Player Repository About this.

www.hypethesonics.com

One important disclaimer here. The tests we perform are looking for total waveform error from time-domain measurements. Metrics are only computed within the audible frequency range, but not all errors in that range are equally audible. In the OEM audiophile world, there’s obviously the potential claim that these ‘errors’ are either a) completely inaudible to the human ear and/or b) enhance the audio, such that what you hear is even better than what the artist and studio engineer intended. We are also aware that there are other tests that attempt to include psychoacoustic effects to strip out or minimize errors that are, by some algorithm, deemed inaudible. We do not want to get into such debates here. Our interest is simply in why the raw waveform errors from all other ESS DACs we’ve measured are so much better than those from the 9038D, and why Amir’s complex waveform test result looks so amazing when ours – for this specific ESS device - doesn’t.

Here are some example results using the 9038D’s brick-wall and linear phase slow roll-off filters:

Total waveform error (BW filter):

Magnitude degradation (BW filter):

Phase degradation (BW filter):

Total waveform error (LPSR filter):

Magnitude degradation (LPSR filter):

Phase degradation (LPSR filter):

We have more graphics for anybody who’s interested, but adjusting parameters like filters and harmonic compensation coefficients, load impedance, etc., made no appreciable change to the main (music signal) histogram error.

The impact of the anti-aliasing filter is noticeable with certain 44/16 test signals (but barely noticeable with any 96/24 signals). The main music signal (the error shown in the largest histogram at the foot of each slide) is largely unaffected by the filter as music spectra (and PSN spectra) tends to taper away approaching 20 kHz. We suspect blind listening tests with actual music tracks would not reveal differences between these two filters. Errors from sinusoids are always consistently low, so SINAD should be great; this is an awesome device for lovers of the classic Concerto in 1 kHz

Error from our white noise test signal is lower with the BW filter, due to less magnitude degradation at high frequencies. Here’s a comparison of best-case-scenario errors from white noise and program-simulation noise at 96 kHz/24-bit:

White Noise(@96kHz), Brick-Wall filter, 32 Ohm:

Total waveform DF = -51.4 dB	Magnitude DF = -49.9 dB	Phase DF = -54.9 dB

Program Simulation Noise(@96kHz), Brick-Wall filter, 32 Ohm:

Total waveform DF = -38.7 dB	Magnitude DF = -54.7 dB	Phase DF = -46.3 dB

For comparison, here are those same signals from the Shanling M0 Pro:


White Noise(@96kHz), Apodizing filter, 32 Ohm:

Total waveform DF = -69.7 dB	Magnitude DF = -63.5 dB	Phase DF = -80.2 dB

Program Simulation Noise(@96kHz), Apodizing filter, 32 Ohm:

Total waveform DF = -83.1 dB	Magnitude DF = -84.2 dB	Phase DF = -70.0 dB

The two main problems with the 9038D seem to be a phase inaccuracy, especially at the lower frequencies (below 1kHz) and jitter or some other source of time inconsistency, which is visible on the above magnitude diffrograms (vertical yellow-ish bars).

We applaud Amir for testing with a more complex waveform. His multi-tone/pseudo white noise signal certainly looks nice and complex in the time domain:

However, we suspect the amplitude peaks of those spectral components are fluctuating slightly with time and a statistical/periodic FFT showing only averaged amplitude vs frequency simply isn’t showing these errors.

 

[Jimbob54]

Wonder if @IVX would care to comment on this.

 

[IVX]

Jimbob54, sorry, I have no idea what about these pics, probably HF residual content disturbs their ADC to read 9038D's signal correctly. Actually, Amir's measurement was a lot worse than mine(THD+N -118.5db@1kHz DR 125db(A)), for that reason + habit of using APx555 unbalanced.

 

[AnalogSteph]

Speaking of rain and parade and stuff - what specific underlying properties are these metrics supposed to isolate? (That would make the difference between a metric and a good metric.) What do they do better than the ones established for 20-30 years+? Or is it just the usual jumbled mess that people with no actual background in converter technology (or even EE at all) tend to come up with, a linear combination of so many factors that you can't make heads or tails of it? (The Gearslutz diff test debacle comes to mind.) I sure can't, and I've been dabbling in digital audio for like 20 years. I mean, I do know what output impedance is, so there's that, but otherwise...

 

[Jimbob54]

Speaking of rain and parade and stuff - what specific underlying properties are these metrics supposed to isolate? (That would make the difference between a metric and a good metric.) What do they do better than the ones established for 20-30 years+? Or is it just the usual jumbled mess that people with no actual background in converter technology (or even EE at all) tend to come up with, a linear combination of so many factors that you can't make heads or tails of it? (The Gearslutz diff test debacle comes to mind.) I sure can't, and I've been dabbling in digital audio for like 20 years. I mean, I do know what output impedance is, so there's that at least, but otherwise...

I feel a little better about my ability to not even remotely comprehend now. Thanks!

 

[raif71]

My current setup of 9038d with Shanling M0 and Fiio A5 seems to share the findings that "... a) completely inaudible to the human ear and/or b) enhance the audio, such that what you hear is even better than what the artist and studio engineer intended"

So, I'm currently enjoying what I hear.

 

[solderdude]

However, we suspect the amplitude peaks of those spectral components are fluctuating slightly with time and a statistical/periodic FFT showing only averaged amplitude vs frequency simply isn’t showing these errors.

interesting ... But when measuring why would one only suspect and not prove it ? How big would the fluctuations need to be in amplitude for changes to be -50dB ?
That would mean a 0.027dB variance in level ?
That should be provable.... in which case there is some hard evidence where it actually came from.

 

[Sokel]

Jimbob54, sorry, I have no idea what about these pics, probably HF residual content disturbs their ADC to read 9038D's signal correctly. Actually, Amir's measurement was a lot worse than mine(THD+N -118.5db@1kHz DR 125db(A)), for that reason + habit of using APx555 unbalanced.

I think that these measurements are at 44.1Khz/16-bit (at least the first row) maybe the second is resampled?

 

[MaxwellsEq]

We’re a bit late to this party, but that’s mainly because we spent a lot of time exploring this device from various angles. We ultimately reached the same conclusion and so figured at this point we may as well share what we’ve found.

We ended up testing two of these devices. We tried tweaking the various coefficients and filters in the companion app, applying our standard 32 Ohm load vs no load, varying output power, etc. Nothing significantly changed our results.
TL;DR – We don’t want to rain on anybody’s parade, but we found the 9038D to perform worse than most other ESS-DAC devices. You can see where it sits in the hierarchy of DACs we’ve measured here (default rank is by the median waveform error):

DAP TI Database, HypetheSonics

Digital Audio Player Repository About this.

www.hypethesonics.com

One important disclaimer here. The tests we perform are looking for total waveform error from time-domain measurements. Metrics are only computed within the audible frequency range, but not all errors in that range are equally audible. In the OEM audiophile world, there’s obviously the potential claim that these ‘errors’ are either a) completely inaudible to the human ear and/or b) enhance the audio, such that what you hear is even better than what the artist and studio engineer intended. We are also aware that there are other tests that attempt to include psychoacoustic effects to strip out or minimize errors that are, by some algorithm, deemed inaudible. We do not want to get into such debates here. Our interest is simply in why the raw waveform errors from all other ESS DACs we’ve measured are so much better than those from the 9038D, and why Amir’s complex waveform test result looks so amazing when ours – for this specific ESS device - doesn’t.

Here are some example results using the 9038D’s brick-wall and linear phase slow roll-off filters:

Total waveform error (BW filter):
View attachment 312352

Magnitude degradation (BW filter):
View attachment 312354

Phase degradation (BW filter):
View attachment 312355


Total waveform error (LPSR filter):
View attachment 312358

Magnitude degradation (LPSR filter):
View attachment 312359

Phase degradation (LPSR filter):
View attachment 312360

We have more graphics for anybody who’s interested, but adjusting parameters like filters and harmonic compensation coefficients, load impedance, etc., made no appreciable change to the main (music signal) histogram error.

The impact of the anti-aliasing filter is noticeable with certain 44/16 test signals (but barely noticeable with any 96/24 signals). The main music signal (the error shown in the largest histogram at the foot of each slide) is largely unaffected by the filter as music spectra (and PSN spectra) tends to taper away approaching 20 kHz. We suspect blind listening tests with actual music tracks would not reveal differences between these two filters. Errors from sinusoids are always consistently low, so SINAD should be great; this is an awesome device for lovers of the classic Concerto in 1 kHz

Error from our white noise test signal is lower with the BW filter, due to less magnitude degradation at high frequencies. Here’s a comparison of best-case-scenario errors from white noise and program-simulation noise at 96 kHz/24-bit:

White Noise(@96kHz), Brick-Wall filter, 32 Ohm:

View attachment 312364	View attachment 312365	View attachment 312366
Total waveform DF = -51.4 dB	Magnitude DF = -49.9 dB	Phase DF = -54.9 dB

Program Simulation Noise(@96kHz), Brick-Wall filter, 32 Ohm:

View attachment 312367	View attachment 312368	View attachment 312369
Total waveform DF = -38.7 dB	Magnitude DF = -54.7 dB	Phase DF = -46.3 dB

For comparison, here are those same signals from the Shanling M0 Pro:


White Noise(@96kHz), Apodizing filter, 32 Ohm:

View attachment 312371	View attachment 312372	View attachment 312373
Total waveform DF = -69.7 dB	Magnitude DF = -63.5 dB	Phase DF = -80.2 dB

Program Simulation Noise(@96kHz), Apodizing filter, 32 Ohm:

View attachment 312374	View attachment 312375	View attachment 312376
Total waveform DF = -83.1 dB	Magnitude DF = -84.2 dB	Phase DF = -70.0 dB

The two main problems with the 9038D seem to be a phase inaccuracy, especially at the lower frequencies (below 1kHz) and jitter or some other source of time inconsistency, which is visible on the above magnitude diffrograms (vertical yellow-ish bars).

We applaud Amir for testing with a more complex waveform. His multi-tone/pseudo white noise signal certainly looks nice and complex in the time domain:

View attachment 312377

However, we suspect the amplitude peaks of those spectral components are fluctuating slightly with time and a statistical/periodic FFT showing only averaged amplitude vs frequency simply isn’t showing these errors.

I think I understand, but it would be helpful if you better explained your methodology. Are you arguing that most measurements in the frequency domain are the sum of thousands of "instances" in the time domain. So if you look closely at the behaviour of those instances, you can differentiate between two devices whose frequency behaviour or whose averaged-time behaviour look the same?

 

[csglinux]

probably HF residual content disturbs their ADC to read 9038D's signal correctly.

I don't think that's it. I'm happy to be corrected if I'm wrong, but I believe the issue is coming mainly from phase errors at low frequencies. I agree it's not super intuitive to just see green amorphous blobs without any labels. (Serge - perhaps a legend or axis scale would make things clearer?) I believe what Serge is showing here in each of those square boxes is time slices along the x axis and frequency (ranging from 20 Hz to 22 kHz) on the y axis and the color scale for each pixel is some kind of an RGB rainbow scale with blue = low error ranging up to red = high error. If you look at the M0 Pro white noise figures, they're pretty much blue all over. If you look at the 9038D white noise figures, they're kind of yellowy, ranging to orangy at lower frequencies (toward the bottom of the squares, especially for phase). It might(?) be that all this is totally inaudible to any human ear, but it is curious that other ESS DAC devices don't seem to show such errors.

View attachment 312394

My current setup of 9038d with Shanling M0 and Fiio A5 seems to share the findings that "... a) completely inaudible to the human ear and/or b) enhance the audio, such that what you hear is even better than what the artist and studio engineer intended"

So, I'm currently enjoying what I hear.

As a good audio science review member, if you had the inclination, time, money and somebody to help you, it might be interesting to see, if only just for yourself, stats from a blind, spl-matched A/B between the direct output of your 9038D and the direct output of the M0 Pro that Serge refers to in his post (not the original M0 in your photo).

 

[pjug]

@Serge Smirnoff can you make your files available so that people here can run them through DeltaWave or whatever might help get a handle on what might be going on?

 

[raif71]

As a good audio science review member, if you had the inclination, time, money and somebody to help you, it might be interesting to see, if only just for yourself, stats from a blind, spl-matched A/B between the direct output of your 9038D and the direct output of the M0 Pro that Serge refers to in his post (not the original M0 in your photo).

I know you mean well but there could be that some members might interpret differently what you wrote there. They might think in the lines of, Gee ... I don't even care about the stats of outputs of M0 and 9038D... am I a "bad" audio science review member?

 

[amirm]

We ended up testing two of these devices. We tried tweaking the various coefficients and filters in the companion app, applying our standard 32 Ohm load vs no load, varying output power, etc. Nothing significantly changed our results.

My DAC measurements are at high impedance. You can't compare them to loading the output with 32 ohm. They are also at -1 dBFS. Your fine print says -10 dBFS??? If so, that is another variation. I have not seen anyone measure a DAC at -10 dBFS. Or with loading down to 32 ohm.

 

[Sokel]

My DAC measurements are at high impedance. You can't compare them to loading the output with 32 ohm. They are also at -1 dBFS. Your fine print says -10 dBFS??? If so, that is another variation. I have not seen anyone measure a DAC at -10 dBFS. Or with loading down to 32 ohm.

I think they only test DAP's and headphone stuff,not DAC's.
That's a DAP.

 

[amirm]

Well, it would be wrong for DAPs just as well.

 

[solderdude]

Perhaps the line of thinking was to check how DAPs etc are used in practice (when driving headphones)... most dongles driving headphones directly will be having peaks around -10dBFS and are likely to have some headroom left to play loud.
Then again... if it really is about testing headphone outs then it should also be tested at say ... 16ohm as a lot of earphones these days are even lower than that. At least repeat the test with various loads. In that case the cable to the load will also start to matter for SE outputs (not so much for IEMs).

In any case it isn't that different from PK measurements.

 

[csglinux]

My DAC measurements are at high impedance. You can't compare them to loading the output with 32 ohm. They are also at -1 dBFS. Your fine print says -10 dBFS??? If so, that is another variation. I have not seen anyone measure a DAC at -10 dBFS. Or with loading down to 32 ohm.

Any chance you have the following conveniently lying around in your 9038D measurements folder: a) the formula for your multi-tone test signal, and b) the raw numbers for the phase angles of each mode?

It would probably require more work to extract phase variances - I guess you would need to go back and compute a series of FFTs on shorter time windows (400 ms or something to match what the OP's doing?). But this might not be necessary. The phase means on their own might already show something.

 

[staticV3]

@csglinux while not quite the same as what you asked for, here is the multitone signal that Amir uses as .wav file: https://www.audiosciencereview.com/forum/index.php?threads/sinad-measurements.4071/post-245803

If you could upload the audio file(s) that is used to build the DF Metric graphs, that would be sweet!

 

[NTK]

The ratings from the Df metric look to me like a random number generator.
For example, the Sony NWA105 got a median Df of -71.4 dB and the E1DA 9038 got -42.2 dB.

And here are Amir's measurements. Noise alone in the Sony is much worse than the E1DA. The Df numbers made no sense to me.

 

[staticV3]

Yes. Numbers need to be repeatable and reproducible to hold any water in audio.
That's why it'd be great to have the audio files that @csglinux and @Serge Smirnoff use to create their graphs, so that we can repeat their tests and compare results.