How can DAC's have a SOUND SIGNATURE if they measure as transparent?

[donrleonard]

Is it just me or are these a bunch of very poorly performing opamps

That's a fair point. Are the tests in this study relevant to modern DACs when THD figures of those modern DACs are pushed below those reported in the study? I think this is the point that @Beave and @RandomEar were ever so gently trying to make.

 

[antcollinet]

In 2018, Amir examined a peer-reviewed conference paper published in the 2011 proceedings of the Audio Engineering Society. The author is a faculty member at a well-regarded audio engineering program at McGill University in Canada.

Here is the reply I made last time that post was discussed.

Replace OP amps. Completely pointless, or not?

There are engineers (one in this thread) that would apparently disagree to some extent. Then either they are not engineers who understand the application of op amps - or they need to come forward, and explain in engineering terms what they think is happening. Perhaps they should back that up...

audiosciencereview.com

TLDR - the paper is not relevant to the typical line level use cases of op amps in DACs and Amps discussed here. Plus other problems with the paper. For example the op amps tested were not disclosed.

 

[antcollinet]

In both cases, multitone performance was noted as "great". But are they identical?

It doesn't matter if they are identical, They are both performing at levels such that the distortion is inaudible.

Once a DAC has inaudible distortion, another DAC with even lower distortion is, obviously, not "more" inaudible.


You really need to stop trying to find flaws with the tests being done, and actually try to do a proper test of your own hearing. You could start with the klippel listening tests linked by @RandomEar upthread. Then do (an accurately level matched) blind test on the devices you think you are hearing differences between.

I'm not going to waste my time further by answering what (to all intents and purposes) has now become sea-lioning.

 

[Old_School_Brad]

Here is the reply I made last time that post was discussed.

Replace OP amps. Completely pointless, or not?

There are engineers (one in this thread) that would apparently disagree to some extent. Then either they are not engineers who understand the application of op amps - or they need to come forward, and explain in engineering terms what they think is happening. Perhaps they should back that up...

audiosciencereview.com

TLDR - the paper is not relevant to the typical line level use cases of op amps in DACs and Amps discussed here. Plus other problems with the paper. For example the op amps tested were not disclosed.

Exactly! I read that paper and saw the circuit some time ago, and I was genuinely puzzled as to how anyone could draw any conclusions from the results.

Your comment perfectly captures what I thought about it at the time. -It's flawed and irrelevant.

..the audio was put through 2 40dB (x100) (distortion) gain stages plus a 9dB buffer and put through that circuit twice.

 

[Thomas Savage]

I love this thread it reminds me of my youth and old records that skipped. You'd have to nudge the arm every now and then.
Dacs are commodities
Yes, you may find one that's muffled and sounds different ( shows up in measurements)
Move on to speakers, DSP and your room.
Awaiting the next influx of , "My wife heard it from the kitchen! ".

In reality their more likely to hear their wife in nextdoors kitchen .

 

[voodooless]

Are the tests in this study relevant to modern DACs when THD figures of those modern DACs are pushed below those reported in the study?

Modern? DACs have been transparant for decades. These distortions figures must been heavily doctored to generate a positive ABX test. Almost all of these would land on the far red side of @amirm’s list.

 

[Blumlein 88]

Thanks for this suggestion, Ant. This is exactly the kind of feedback I was looking for.

This was Amir's multitone test for the DAC originally in question - the PS200.
View attachment 432175

And this was his multitone test for the Wiim Ultra that I did the most extensive (within my humble limitations) impressionistic comparison testing of.
View attachment 432179

In both cases, multitone performance was noted as "great". But are they identical?

I know even less about music theory than I do about electronics, terrifying though that may be. Can this test indicate whether even- or odd-order harmonics are present in these distortion profiles?

Something that you would do yourself well to learn is these uncontrolled listening impressions are overwhelmingly somewhere between deceptive and useless with a heavy lean toward deceptive. Taking results of such listening "tests", and then looking for a correlation of some sort is putting the cart before the horse. If you run across something surprising or unexpected the first step most definitely is to confirm it with controlled listening. If you don't do that you are most likely chasing ghosts.

 

[voodooless]

Finally, a 2019 masters thesis out of the University of Huddersfield (UK) cites these McGill studies and was able to reproduce their general findings.

No they did not. It’s a totally different study, specifically to test the audibility of line level transformers. This test has zero relevance to DACs.

 

[BDWoody]

Thanks for this suggestion, Ant. This is exactly the kind of feedback I was looking for.

This was Amir's multitone test for the DAC originally in question - the PS200.
View attachment 432175

And this was his multitone test for the Wiim Ultra that I did the most extensive (within my humble limitations) impressionistic comparison testing of.
View attachment 432179

In both cases, multitone performance was noted as "great". But are they identical?

I know even less about music theory than I do about electronics, terrifying though that may be. Can this test indicate whether even- or odd-order harmonics are present in these distortion profiles?

Check out the shoutometer I referenced earlier.

Do you have a feel for how small these numbers are?

I think you are just responding now out of reflex, rather than trying to process what is being explained. I'm going to give you one day off from this thread in hope that you do a little more reading, and maybe even a controlled listening test or two (get a multimeter... They are cheap). You are just going around and around at this point. Spend a little time climbing the learning curve then maybe we can communicate a little better.

 

[solderdude]

In all fairness it is understandable that people doubt test signals because when we look at the results from null tests (actual output voltage) compared to what the data is supposed to represent we often see that nulls don't go deeper than say -60dB (which is already very good) and don't come near the measured values of a test signal.
Sometimes differences are found reaching -30dB or even worse.
This is often coined as 'proof' that music is different from test signals.

Of course there are reasons for that:
1: the signals at various frequencies in music are actually very small and not anywhere near the max. amplitude of a test signal. So the system noise floor is closer than when test signals are used.
2: There are phase differences and ripples that will not null exactly and thus lower the 'numbers'.
3: aside from the reproduced signal there is an additional ADC stage with its own 'distortions' added.
4: the 'to compare with file' are samples and the 'inbetween sample values' the ADC puts out have been calculated by another program (algorithm)
5: there are timing differences between the clock of the DAC and used ADC.

So... best to only look at test signals if one wants to know the actual accuracy of the DAC conversion using high resolution test equipment.

 

[RandomEar]

In 2018, Amir examined a peer-reviewed conference paper published in the 2011 proceedings of the Audio Engineering Society. The author is a faculty member at a well-regarded audio engineering program at McGill University in Canada. This study demonstrated, through a double-blind AB preference test, that a statistically significant number of listeners were able to express preference for one DAC over another:

• "While the differences were slight, the op-amps in this test were capable of producing levels of distortion within their normal, unclipped operational range that were detectable by listeners. It is possible for listeners to differentiate between op-amps based on distortion characteristics alone. The ability to distinguish between the distortion characteristics of op-amps appears to be dependent on a complex interaction of the THD vs. Frequency, THD vs. Amplitude, and the harmonic spectrum of the distortion. This low-level distortion is enough to alter the character of the sound but does not necessarily reduce listener preference."

View attachment 432161

As you can see, 9 of the 24 listener-observations (some listeners tested more than once) were able to express a statistically significant (column 1 p<.05) preference for one DAC over another even though both the A and B DACs had THD+N ratios well below 0.1% (columns 2 & 3).

In a follow-up 2014 study, the author finds that

• "Due to the gain-bandwidth characteristics of operational amplifiers, their nonlinearities are frequency dependent, showing a rise in distortion at higher frequencies. Depending on the circuit and system implementations, this distortion can be significant to listener perception of sonic character and quality and is therefore relevant to models of op amp-based analog equipment."

Finally, a 2019 masters thesis out of the University of Huddersfield (UK) cites these McGill studies and was able to reproduce their general findings.

• "This thesis presents and discusses research undertaken into the detection and measurement of colouration produced by audio transformers. It is common for transformers to be subjectively described by audio professionals as ‘warm’, ‘fat’, ‘smooth’, etc. however there is little evidence to show if there is an audible difference and if there is a correlation between these levels of perceptual attributes and the performance of the device. Therefore, the research question was defined as: is there an audible difference with a transformer?
• "A review of the objective and subjective elements of the study was conducted. First the history, application and operation of the audio transformer with an aim to understand the objective measures and performance of the device with focus on the nonlinear distortion response. This also includes the design and testing of a suitable test circuit used in the measurement method. The subjective review concerns perception of distortion and testing methodologies for investigating audible differences. The conclusion of the review is that few pieces of research exist showing the relationship between device, distortion and perception
• "The testing of each transformer involved the use of a specially designed test circuit using a variety of measures including THD+N and frequency domain analysis, to provide the most information about the operation of the device. Using the same setup, the device responses to a variety of samples were recorded and implemented in a double-blind triple-stimulus with hidden reference test using trained listeners in accordance with the ITU-R BS.1116-3 recommendation. The test results were then analysed for a random distribution using a 1-tailed binomial test. The results of the analysis show a high likelihood that the bass samples were audibly different shown by the significant p-values of all 3 samples at less than 0.001. A slight correlation seems to exist with THD+N, 3rd harmonic distortion and level however with no other obvious trends, it was concluded that the distortion and therefore the audibility is programme dependent.
• "It was concluded that transformers are likely to produce a level of distortion deemed audible although the effect is considered to be programme dependent."

None of these tests "prove" that I (or anyone) can tell the difference between two operational amplifiers with otherwise similar measurements of THD. But the statistical significance of these seemingly well thought out experiments should increase our confidence in the plausibility that I--and myriad other audio professionals and enthusiasts--aren't suffering from some mass delusion when we hear qualitative differences between opamps.

Audio enthusiasts and technicians should be less-dismissive of claims that individuals can hear such qualitative differences between DACs. There may be quantifiable differences in the way low-level operational amplifier distortion generates audible even- or odd-order harmonics.

Apart from the fact that I could not gain access to the original paper and the possible shortcomings of this study (only 13 actual participants, sound routed through the opamps a total of 4 times, seemingly hand-crafted circuits), the table you show clearly demonstrates that nearly all significant results are linked to distortion - either harmonic or IMD - on the order of -45 dB. That is typically audible to trained listeners and therefore absolutely not surprising. It's also orders of magnitude away from the -90 to -110 dB I was quoting for typical DACs. Which is what we were talking about before. There is no logic in analyzing individual components - be it opamps, PSUs or controller chips - if the resulting device built from all these components measures transparent. You're diving into details while losing sight of the bigger picture.

Again: Single components are not devices. -45 dB is not -90 dB. Do the Klippel test and see/hear for yourself.

 

[oleg87]

I'll confess that I don't fully understand the ASR dashboard. But I believe Amir fully when he ooos and ahhs over low levels of harmonic distortion and signal to noise ratios. I just wonder whether there are other aspects to the quality of a sound wave that are not captured by this model---a model that puts a $9 apple dongle on par with much more elaborate DAC implementations.

lol, "elaborate" is a curious word to use. Do you know how much circuitry can fit on a chip the size of a pin head? I'd bet good money it's a great deal more sophisticated than your average goofball reinvent-the-square-wheel discrete audiophile DAC.

 

[Old_School_Brad]

lol, "elaborate" is a curious word to use. Do you know how much circuitry can fit on a chip the size of a pin head? I'd bet good money it's a great deal more sophisticated than your average goofball reinvent-the-square-wheel discrete audiophile DAC.

It's hard to accept that your Denafrips might be indistinguishable from an Apple dongle. That’s a tough realization, and admitting it feels almost impossible. Especially after you've spent a lot of money, and because somewhere along the way, someone sold you on the idea of its superior sound quality and the incredible listening experience it promises. So you went ahead and bought it. But now, someone’s suggesting that it might not sound any better than a 20 buck Apple dongle, something no bigger than a fingernail? It’s no wonder your mind goes into lock-down, caught in fight-or-flight mode.

 

[Mart68]

It's hard to accept that your Denafrips might be indistinguishable from an Apple dongle. That’s a tough realization, and admitting it feels almost impossible. Especially after you've spent a lot of money, and because somewhere along the way, someone sold you on the idea of its superior sound quality and the incredible listening experience it promises. So you went ahead and bought it. But now, someone’s suggesting that it might not sound any better than a 20 buck Apple dongle, something no bigger than a fingernail? It’s no wonder your mind goes into lock-down, caught in fight-or-flight mode.

IME that's a bridge too far for many audiophiles. When I was using a Topping E30 I had a few friends say 'Why are you using such a cheap DAC with such an expensive system?' - as though emptying money from my bank account would automatically improve sound quality. That's really the extent of their analysis. Some got E30s to see what the fuss was about - their verdict 'Really good - for the money' - but ofc not as good as one ten times the price.

 

[Purité Audio]

That’s pure marketing and supposed differentiation between price bands, the ‘best dac under £1k,’. which really annoys me.
Keith

 

[FrenchFan]

Don't agree with you

"DACs have a sound signature"

Even, some DACs have "3"

Go to Topping and look at the D90 III Sabre.

It has 3 Sound Simulations.

Normal
Lamp
Transistors

Okay, Topping adds a mass of distortion H2 and H3
for "lamp" and "transistor", I grant you that.

But to the question do DACs have a sound signature, today
we can only answer "Yes".

If I were Topping I would have made 3 DACs, one normal, one lamp
and one transistor.

Of course Lamp and transistor are much much more expensive.

 

[donrleonard]

I'm not going to waste my time further by answering what (to all intents and purposes) has now become sea-lioning.

As an educator, I spend a good part of my professional life working against the kinds of reasoning that lends itself to flat-earthers and climate deniers. I’m sorry if you mistook my inquiries as either insincere or narcissistic.

I was able to find an answer to my question in a highly informative video about the strengths and weaknesses of 1-2 dimensional audio measurements. It is not the multi-tone chart that allows us to detect even- and odd-order harmonics within a distortion profile, but rather the FFT.

The limitation of the FFT, of course, is that it can only evaluate the harmonic structure of a piece of audio equipment against a single fundamental frequency—typically 1khz.

Some friendly folks over on another forum have pointed me towards intermodulation distortion level sweeps as a way to at least attempt to model the behavior of actual music. In my line of work we call this experimental objective “internal validity.”

But even this is but an approximation. At the end of the day all models are wrong, but some models are useful.

Despite the limitations inherent in the ASR model, I’m glad that this hobby has the Brotherhood of the 1khz Test Toners. You all have forced me to ask better questions, and you offer an excellent baseline from which to make discoveries about the 4-dimensional qualities of this hobby.

 

[Old_School_Brad]

...
But even this is but an approximation. At the end of the day all models are wrong, but some models are useful.

Despite the limitations inherent in the ASR model, I’m glad that this hobby has the Brotherhood of the 1khz Test Toners. You all have forced me to ask better questions, and you offer an excellent baseline from which to make discoveries about the 4-dimensional qualities of this hobby.

That's a backhanded compliment if I've ever seen one.

And in response to @antcollinet pointing out the sealioning, no less. A bold reply, I must say.

 

[donrleonard]

Check out the shoutometer I referenced earlier.

Do you have a feel for how small these numbers are?

I think you are just responding now out of reflex, rather than trying to process what is being explained. I'm going to give you one day off from this thread in hope that you do a little more reading, and maybe even a controlled listening test or two (get a multimeter... They are cheap). You are just going around and around at this point. Spend a little time climbing the learning curve then maybe we can communicate a little better.

Thank you for those suggestions, BD Woody.

I have to say, my first experience as a member of this forum has left a bad taste in my mouth. When someone on another thread asked a question about two DACs that I happen to own, I posted my reflections. Through the deluge of mockery that ensued, I attempted to find the knowledge and experience of members and square it with my own knowledge of the hobby and the scientific method. In the end, my posts were moved to what I have come to understand is the graveyard of inquiry for anyone who claims to hear an audible difference between a $9 dongle dac and anything else. When I tried to continue my inquiries here, in the face of an even greater deluge of mockery, I was muted while others were permitted to continue their mockery.

I have come to realize that this is a well-recognized characteristic of ASR within the hobby. It’s unfortunate that such behavior is allowed to persist, as there is such a wealth of knowledge and experience here.

I will take my curiosity elsewhere, but I will surely continue to consume the fine reviews and critical thought happening here..

I wish you luck as you work to implement @amirm ’s vision of a gathering of “Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science.”

Best,

Don

 

[oleg87]

The limitation of the FFT, of course, is that it can only evaluate the harmonic structure of a piece of audio equipment against a single fundamental frequency—typically 1khz.

Some friendly folks over on another forum have pointed me towards intermodulation distortion level sweeps as a way to at least attempt to model the behavior of actual music. In my line of work we call this experimental objective “internal validity.”

But even this is but an approximation. At the end of the day all models are wrong, but some models are useful.

Despite the limitations inherent in the ASR model, I’m glad that this hobby has the Brotherhood of the 1khz Test Toners. You all have forced me to ask better questions, and you offer an excellent baseline from which to make discoveries about the 4-dimensional qualities of this hobby.

I find it unbelievably obnoxious when people who clearly have zero understanding what they're talking about presume to smugly pontificate about flaws in engineering performance metrics. Do mind the door on your way out - this forum will be perfectly fine without these sorts of impossibly asinine "contributions".