SINAD Measurements

[Rja4000]

BIO: Millennia Media

Wow!

 

[audio_tony]

In this test, we find that a $1.00 opamp with 0.001% THD sounds more pure and accurate and spatial, more true to the reference, than a $5.00 opamp with 0.0001% THD. We decide to ship the product with $1.00 opamp with poorer quantitative THD.

This statement holds no weight without knowing the final THD figures of the completed product(s) in which these opamps were tried.

It's possible to use an opamp with 0.0001% THD in a circuit that ultimately produces worse THD than the opamp itself due to various factors (layout, topology etc.)

Just as with opamp rolling, it's very easy to quickly turn a good design into a bad one due to poor opamp choice.

 

[signalpath]

This statement holds no weight without knowing the final THD figures of the completed product(s) in which these opamps were tried.

No no. The buffer box with one amplifier IS the completed product. It's a stupid-simplified argument to show that "good engineering" doesn't necessarily mean "lower THD." (someone here suggested that lower THD always shows better engineering).

In practice, we do use THD numbers in R/D. Sometimes we'll see a marked spike in THD, perhaps caused by an improper ground path or poor power supply decoupling or EMI (etc.). So, yes, THD numbers can play a valid role in designing a product. But that doesn't mean lower measured THD of a consumer product always infers better engineering. It doesn't. Not at all.

 

[Frank Dernie]

In a conversation with Amir (long ago), he shared that "yes, there are multiple ways based on psychoacoustics that we can calculate the distortion-free channel we need for full transparency" and restated that -120dB THD is required for "full transparency."

No.

I agree with this from my experience recording and listening to music. The idea that anybody would be able to hear a -120dB distortion tone whilst listening to a 0dB tone seems ridiculous to me. I have written here before that such performance may be needed for a device which can record all audible sound but has no volume/level control but I don't know of one, or see the need for such a device.
Here in my quiet listening room with the volume set to a good, IMO , listening level, 40db in the quiet bits of classical recordings 95dBish peaks I can not hear a tone at -75dB. The idea, therefore, that a distortion product at -75dB would be audible whilst listening to music is not credible to me.

Having written that I am not necessarily in agreement with you about what constitutes good engineering and am completely sceptical that a device having 0.01% distortion would sound any different to one with 0.001% if the frequency response of both was the same.

 

[StevenEleven]

Are we being fair/slightly misleading (to the less technically aware) with the SINAD measurements? Thought I would get some feedback on this.

Below is the table we are compiling for SINAD performance. However we are comparing components that have disparate output voltages. The typical output voltage for dBFS is 2 V rms and seems to be a sort of unofficial standard, however some of the DACs here are producing anything up to 20 V rms. This will obviously bring an advantage to the measurement, but not necessarily in the case of usage. Whilst it is perfectly correct to measure the full voltage SINAD, should we also perform a nominal 2 V rms measurement for fairer comparison? Thoughts?

Example
Chord Mojo Fed with 0dBFS signal. Output volume set to 2 V RMS. SINAD = 100dB

View attachment 14770

Volume set just prior to clipping at 5 V rms. SINAD = 106.6dB

View attachment 14771

As a card-carrying member of the Less Technically Aware I will tell you what I want to know in lay terms (assuming sufficiently flat frequency response): If for a person with hearing acuity toward the top end of what is within normal limits (not me, by the way, but we want to be helpful to more than just me), listening at the listening position in a relatively quiet home to the equipment, is there hiss, hum, distortion or buzz with the volume turned all the way down, or while listening where the better part of the program material is between 50-75 decibels, c weighted slow response (my one piece of much-used technical measurement gear is a decibel meter, which I keep by my listening position)? Based on my limited knowledge but extensive listening experience, this would be my pass-fail test.

The other thing that drives me batty is problems with channel balance. If there are problems with channel balance as one goes through the normally used volume range that is for me a fail, a Do Not Buy. It really bothers me. With good equipment I can do “serious” listening at fairly low volumes (peaking at 60 dB or so)—good for my ears and my nerves. So yes, I need to know if it’s working well at lower volumes, including maintaining proportionate channel balance and having a low enough level of baseline noise, much more than I need to know if it will work well at volumes louder than I will ever use.

So if the above can be translated into bright-line metrics, with I’m sure a band of grey, I would know what do put in the Do Not Buy box, irrespective of price, and move on from there.

Equipment I have had over the decades that failed the above for me included a cheap Sony receiver (hum), an expensive Yamaha receiver (volume balance), multiple headphone amps (all of the above), and a Windows PC (clipping at regular listening volumes) and a powered subwoofer (hum). Those go back in the box and back to the store. I as a consumer who is Less Technically Aware am not not going to play fun troubleshooting games with it or pretend the problems don’t exist. There‘s too much stuff out there that works just fine. There’s always been a second piece of gear that worked just fine for the same price.

At the other extreme, the one we need to keep in perspective IMHO—I am at times tempted to get an RME ADI-2 Pro but it has nothing to do with listening pleasure—it would be for the ultra-cool gadget ultra-high performance point of view, but knowing full well it surpasses in performance by orders of magnitude anything needed to enjoy recorded music to the fullest. So I get the respect for and pleasure of excellent engineering for engineering‘s sake angle, even as a layperson. But when I maintain my audio rigs that is not where I am coming from. I just don’t want any audible deficiencies in performance under normal listening conditions before getting to the transducers. That’s my pass-fail.

 

[RichB]

The required THD + N or SINAD performance presented by manufacturers often correlate well with their product offerings.
It's remarkable

I remain unconvinced that every thing that matters has been measured, therefore, some headroom is desirable.
Yes, measured performance often correlates with quality, price less so.

One thing that I love about this site, is providing real objective data for products, especially those with very limited specifications.
Many oxen are gored.

- Rich

 

[mansr]

I'd like to see tested devices divided into three categories:

- The Good: Distortion below audible limits under all circumstances. Buy with confidence.
- The Bad: Distortion possibly audible in some cases though without significantly impacting sound quality. Buy if other factors are compelling.
- The Ugly: Distortion severe enough to cause audible "colouration" in normal listening. Do not buy without listening first.

The classification criteria should probably take into account more than just THD+N at 1 kHz.

 

[signalpath]

I remain unconvinced that every thing that matters has been measured, therefore, some headroom is desirable.

Yes. The idea of "possibly beneficial performance headroom" can be valid -- especially for specs like noise, linearity, jitter, and SR-related filter performance. If I had the choice of a DAC with 27-bit linearity over today's very best DACs with 21-bit linearity, I would take the 27-bit DAC (and hope that someday the rest of the audio industry caught up with 27-bit performance - mics, ADCs, DAWs, power amps, etc.).

The same argument doesn't hold for THD.

But, hey, if anyone believes there's a better than 50:50 chance that a 0.0001% DAC will sound more accurate than a 0.001% DAC, knock yourself out. We all know that faith and beliefs are often stronger than settled science.

 

[RichB]

Yes. The idea of "possibly beneficial performance headroom" can be valid -- especially for specs like noise, linearity, jitter, and SR-related filter performance. If I had the choice of a DAC with 27-bit linearity over today's very best DACs with 21-bit linearity, I would take the 27-bit DAC (and hope that someday the rest of the audio industry caught up with 27-bit performance - mics, ADCs, DAWs, power amps, etc.).

The same argument doesn't hold for THD.

But, hey, if anyone believes there's a better than 50:50 chance that a 0.0001% DAC will sound more accurate than a 0.001% DAC, knock yourself out. We all know that faith and beliefs are often stronger than settled science.

I am more concerned with amplifier measurements than DACs.
It is irksome that manufacturers are happy to the quote 120+ dB S/N of the DAC chip, then do not publish the delivered performance.
Measurements here routinely show delivered performance that is 100 dB (or less) and perhaps worse at the preamplification level for typical listening levels. 20 dB here 20 dB there an pretty soon we are talking about real performance

- Rich

 

[amirm]

Yes, it was an AES paper years ago, maybe more than one ABX study. JJ might remember.

Well, I remember as I have read each one of them 100 times. You must be talking about the few published limited tests like Clark's:
Ten years of A/B/X Testing, http://www.aes.org/tmpFiles/elib/20200105/5549.pdf

I have done a brief digest of it here: https://www.audiosciencereview.com/...ity-and-reliability-of-abx-blind-testing.186/

His tests were extremely limited in scope and did not include trained listeners.

Can assure you, after 30 years of designing and blind AB listening to world-class audio circuits (over 50,000 channels in use at the highest levels of professional audio, film scoring, classical music, post production, archiving, etc.), THD below a certain threshold in no way correlates with better perceptual audio quality. None. In fact, in many cases, it's just the opposite (see my prior post).

I am afraid there is no such fact. I have performed countless controlled tests of devices with high distortion versus low. At best they sound the same. At worse, they sound worse. In no case have I found the distortion to have any value whatsoever.

Of course there are many people reporting what you say. Alas, they are never accompanied with documentation of proper double blind tests. You could say you have but that doesn't amount to anything. I have 20+ years of experience in this field as well and what you say doesn't hold water, unless you are testing sighted. Here is an example from clark's paper above:

I would encourage ASR to not "qualitatively rank" audio gear based on THD. Giving performance numbers is fine, but use care in making scientifically qualitative inferences that have no basis in science.

Noted but the ranking is working in getting more and more manufacturers to find sources of noise and distortion and fixing them. Often it costs nothing but better circuit layout or implementation to get there. Maybe the distortion is audible, maybe it is not. But it is certainly a good thing to build cleaner performing audio products at no cost to consumers.

I suggest that you start a new thread and document your double blind tests and help us replicate your results. Until then, THD+N can be used to set provable inaudibility thresholds that guarantee a device is transparent. You have not provided no alternative to this. If anything you are advocating unknowable quantities for such. That is not audio science.

 

[signalpath]

T

I am afraid there is no such fact. I have performed countless controlled tests of devices with high distortion versus low. At best they sound the same. At worse, they sound worse. In no case have I found the distortion to have any value whatsoever.

Amir, thanks for the passionate reply. Let me dig thru my archives. I remember papers and other ABX testing on this very topic -- the audibility of THD percentages. My recollection is that THD below 0.1% was inaudible in all blinded tests. This also correlates with our 30 years of blinded AB testing on audio circuits --- there is no correlation to THD and sonic performance below a certain THD threshold. If you believe you've found repeatable correlation below, say, 0.1%, I would love to see the AB data. I'm not trying to win an argument here. I'm just lobbying for good science. Someone else on this thread remembered some similar tests, Boston Audio Society (?). Let me dig around and see what I can find.

Yes, we've tested virtually every commercial-grade audio op-amp, and a number of video-grade op-amps under AB blinded conditions in our world-class audio room (designed by George Newburn), also via top-end headphones. Some of the best performing op-amps (THD, slew, etc.) in fact are video op-amps, and yet some of them have a pinched, subtle granularity to their top-end and overall timbre. Over the decades, we've tested scores of commercial audio op-amps all with superb THD performance (TI, BB, NJR, AD, ST, Nat, etc.) and have never found a correlation between THD % and sonic transparency, purity, timbre neutrality, dynamic stability, and so forth. The sonic differences between IC opamps is significant, and never correlates with THD trend. That's our experience, but (please don't take this the wrong way), I have no time to create a public forum to document the details of our blinded AB tests.

Some time ago, we had a conversation in which you said "-120dB THD is required for full transparency." I think you also directed me your "audibility threshold" page which said the same thing. If it's true that we cannot perceive THD artifacts below, say, 0.1% (-60dB), then we should probably have a further conversation on this "-120dB THD" postulate. But I guess the onus is on me to find the historical ABX tests on THD perception. I'm on it.

(by the way, the anecdote you presented above is NOT how we do AB testing. We use instant switching between DUT and reference, there is no "sonic memory" required. We're simply comparing two identical paths with different amplifiers, with immediate, blinded switching. If we can detect a difference in 8 of 10 trials, it is a positive. The harder part is determining if the difference is better, worse, different, or a partial mix. It's hard work, as you know.)

 

[scott wurcer]

Some of the best performing op-amps (THD, slew, etc.) in fact are video op-amps, and yet some of them have a pinched, subtle granularity to their top-end and overall timbre. Over the decades, we've tested scores of commercial audio op-amps all with superb THD performance (TI, BB, NJR, AD, ST, Nat, etc.) and have never found a correlation between THD % and sonic transparency, purity, timbre neutrality, dynamic stability, and so forth.

Unfortunately many of these phrases are ill defined audiophile hubris. I told someone once to Google "op-amp rolling" and collect data on what was said. There was zero consistency across dozens of comments, all of these so called attributes were equally present or absent from all manufacturers op-amps.

 

[amirm]

Some time ago, we had a conversation in which you said "-120dB THD is required for full transparency." I think you also directed me your "audibility threshold" page which said the same thing. If it's true that we cannot perceive THD artifacts below, say, 0.1% (-60dB), then we should probably have a further conversation on this "-120dB THD" postulate. But I guess the onus is on me to find the historical ABX tests on THD perception. I'm on it.

That is not what I have said. What I have said is that I can guarantee transparency for all people, in all systems and all content if SINAD distortion products are at or below 116 dB (I sometimes round this up to 120).

Anything below becomes shades of gray. I am sure that vast majority of audiophiles would be stomped at distortion products below -60 dB. So there is a vast gray area between 60 and 116 dB. We can say gear with SINAD of 70 dB is "likely inaudible" to majority of people. But I don't know how to guarantee that.

The above comes from assumption of 120 dB SPL maximum playback which would put threshold of hearing at -120 dB as a result.

Some of that analysis is in this paper: https://www.audiosciencereview.com/forum/index.php?threads/audibility-of-small-distortions.67/

The bar in blue determines the maximum level of distortion/noise that you can tolerate. Just add up to that how loud you listen and it determines the required channel for transparency.

Rooms can readily be this quiet. See this other paper I wrote: https://www.audiosciencereview.com/forum/index.php?threads/dynamic-range-how-quiet-is-quiet.14/

Stuart[2004] shows this graphically with respect to quantization noise, Coding for High-Resolution Audio Systems (J. AES paper):

20 bits would be 120 dB.

If you listen at lower levels, the requirement proportionally goes down.

 

[signalpath]

That is not what I have said. What I have said is that I can guarantee transparency for all people, in all systems and all content if SINAD distortion products are at or below 116 dB (I sometimes round this up to 120). Anything below becomes shades of gray.

OK, we're missing an important element in the "transparency" conversation. Just because an audio device has low noise and low THD does not mean it sounds accurate or transparent. This is a non-scientific assumption. We cannot "guarantee perceptual timbre accuracy" just because THD is below X.XX%. Am I understanding your assumption correctly? If not, please help.

This is yet another reason I suggest it is far better to take separate THD and noise measurements, rather than conflating the two into a single SINAD measurement and calling it "better / worse." That's a qualitative statement that is simply not accurate.

I'm also starting to think of an AB test we could create, with the help of some old-timers here. We could create a simple test bed for A=1 amplifiers: an "A" amp, a "B" amp, and a direct path for original source -- of course all trimmed to better than 0.02dB offset. The selected amplifiers will exhibit a range of inherent THD, from 0.0001 range (audio IC) to 0.1 range (discrete). We use immediate switching. We ask experienced listeners to determine which amplifier sounds closer to the direct path. We do this for perhaps 10 representative amplifiers for a legitimate sample size.

If we find null correlation of THD to sonic purity, we consider it settled science and ASR THD editorial is adjusted to reflect this. If we find true correlation between THD and sonic purity, we've advanced the science of audio to a new and unexpected place, and ASR keeps its format. I happen to know the outcome in advance because we've done all this for 30 years . But I think a public test would be super cool.

 

[signalpath]

Unfortunately many of these phrases are ill defined audiophile hubris."

Scott, help me out. If you ran a high-end audio company for 30 years and did AB listening comparisons on a regular basis for a living, and you shared some of your AB testing experience on-line, would that be hubris? (defined as "excessive pride or self-confidence"). Confused here.

 

[amirm]

OK, we're missing an important element in the "transparency" conversation. Just because an audio device has low noise and low THD does not mean it sounds accurate or transparent. This is a non-scientific assumption. We cannot "guarantee perceptual timbre accuracy" just because THD is below X.XX%. Am I understanding your assumption correctly? If not, please help.

Unless you mean something like frequency response being a factor, then the science absolutely supports what I said. I linked to peer reviewed journal papers including some from ex president of AES (Fielder). If they are not right, then you might as well throw out all the science.

Since you trust JJ, here is a slide from his presentations stating the same:

 

[amirm]

I'm also starting to think of an AB test we could create, with the help of some old-timers here. We could create a simple test bed for A=1 amplifiers: an "A" amp, a "B" amp, and a direct path for original source -- of course all trimmed to better than 0.02dB offset. The selected amplifiers will exhibit a range of inherent THD, from 0.0001 range (audio IC) to 0.1 range (discrete). We use immediate switching. We ask experienced listeners to determine which amplifier sounds closer to the direct path. We do this for perhaps 10 representative amplifiers for a legitimate sample size.

I thought you had done this for 30 years. If so, can you please post the protocols, what was tested, etc.

I can tell you that THD does not have a proportional correlation to audibility. That is because it doesn't consider masking. This is why I always show the FFT spectrum so that proper analysis can be done in that domain. THD points the way to whether we should or should not care about spectral analysis. If THD+N is good enough, then you are golden. If it is not, then you have work to do to determine its audibility.

 

[scott wurcer]

Scott, help me out. If you ran a high-end audio company for 30 years and did AB listening comparisons on a regular basis for a living, and you shared some of your AB testing experience on-line, would that be hubris? (defined as "excessive pride or self-confidence"). Confused here.

You keep using unscientific terms like, timbre neutrality, dynamic stability, pinched subtle granularity, what do they mean? Excessive use of language like this is IME intended to snow and intimidate people with as you say excessive self-confidence.

BTW I did this for 42yr. worked with life fellows of the IEEE and AES, have >$800M in sales of circuits where I was the sole designer and have never experienced any of these listening impressions of op-amp differences except in obviously pathological cases .

 

[signalpath]

Unless you mean something like frequency response being a factor, then the science absolutely supports what I said.

Audio science does not support the claim of "guaranteed transparency" by a SINAD of 116. Simply compare (fast-AB) various amplifiers which exhibit -116 THD and note their timbrel differences. (again, maybe I'm missing something here?)

I thought you had done this for 30 years. If so, can you please post the protocols, what was tested, etc.

I can describe in detail our AB test setup, if you like. But that's internal stuff, and I'm frankly not ready to get into detail about my company's internal processes. Nor would our test results qualify as "AES-level" AB testing -- it's just 2 or 3 people on the test. Need 20-50-100 for valid test.

Rather, let's create an open-sourced AB test for the audibility of THD. Of course, there's always the question of "what kind of THD is it?" Is it high 2nd order, low 5rd order, etc.. But that question I think will become moot at levels below 0.1%.

We might even consider creating 24/96 files that have been run through each different amplifier. Only you and I have the key. The problem with that is we've now created a massive overlay on the simple amplifier path, consisting of an ADC and myriad playback paths. And most people won't have capacity for fast A-B switching.

If you're game, I'm in.

 

[March Audio]

OK, we're missing an important element in the "transparency" conversation. Just because an audio device has low noise and low THD does not mean it sounds accurate or transparent. This is a non-scientific assumption. We cannot "guarantee perceptual timbre accuracy" just because THD is below X.XX%. Am I understanding your assumption correctly? If not, please help.

As I mentioned earlier, no-one is saying a single metric is an indication of final audible quality. This is your straw man.

However if noise and distortion are below certain levels then that particular metric is audibly transparent.