SINAD Measurements

[signalpath]

I followed until this last bit, though would like to hear how negative feedback correlates to worse sonic performance... THD is a measure of linearity, and of course SINAD includes noise, so I am not sure exactly what you are requesting we (that is, @amirm ) to use instead of THD and SINAD? The 32-tone test is also a measure of linearity, and I did not think jitter below a reasonable level correlated well as a distinguishing metric either? Confused - Don

Like many design issues in audio, "it depends."

We've tested .0001% IC opamps that sound wrong and colored, and .0001% IC opamps that sound really good (we use them). We've done this kind of blinded AB testing for 30 years. I'm not saying we're the end-all, be-all in this conversation, but I will stand by our research and say confidently that distortion numbers very often have no correlation to the AB perceptual audio performance of an amplifier.

Is massive internal IC amplifier feedback the culprit? Maybe. In some cases. I frankly can't make the unequivocal statement. I remember a conversation with an audio IC designer from Burr Brown who shared his experience with massive internal feedback vis a vis THD and he said pretty much the same thing, and that's always stuck with me.

Yes, rather than using SINAD as a qualitative indication of audio engineering, I suggest presenting SINAD/THD/IM as quantitative measurements ONLY. No editorial as to "better" or "worse" or color ranking, or "better engineered," etc.. Split it out. Would rather see a pure signal-to-noise number without pilot tone, represented broadband and unweighted, naked and exposed without THD baggage along for the ride. Such is an example of a scientifically-accurate, qualitative representation, as are jitter, freq resp, linearity, filter responses, and many others.

 

[signalpath]

Do you mean that
A) if THD is too low, sound quality begins to worsen

No.

Ultra-low THD numbers do not necessarily correlate with worse sound.

THD numbers (high or low) are generally not a representation of sound quality.

See my last comment (above)

 

[tw99]

@signalpath do you have a recommendation for alternative measurements ? Are you advocating subjective listening as the way to establish the best sound quality ?

 

[daftcombo]

No.

Ultra-low THD numbers do not necessarily correlate with worse sound.

THD numbers (high or low) are generally not a representation of sound quality.

See my last comment (above)

Ok, I agree that under 0.1% or so distortion nobody will hear the difference with 0.01% or less.

But I don't agree that some DAC would have magic properties which couldn't be measured.

 

[March Audio]

And using THD as an indication of "engineering quality" would be misleading. As if a 0.0001% THD product is necessarily better-engineered than a 0.01% product. No. That's not right.

As I just wrote to anmpr1, substituting a 0.0001% IC opamp ($3) for a 0.01% well-engineered class-A discrete FET amplifier ($50) is not necessarily an indication of "good engineering." Good engineering uses heroic measures (endless blind AB circuit listening tests) to determine "better" -- not THD numbers. I remain that ASR's "qualitative ranking" presentation of distortion numbers should be modified.

And, yes, I do think a number of test metrics SHOULD be presented as scientific qualitative statements: noise, linearity, jitter, maybe the 32-tone test, filter responses, etc.. By all means, rank these by color and magnitude, and represent the green-blue candidates as "better."

Thanks.

I disagree. A product that distorts the input signal less IS better engineered.

We have seen plenty of times in tests here that technical performance does not relate to the cost of the product. In any case you should not conflate value for money, which is a subjective opinion, with technical engineering quality.

 

[daftcombo]

I disagree. A product that distorts the input signal less is better engineered.

Otherwise, what can we call "well engineered"? Pretty? Sturdy? Uses gold?

 

[signalpath]

@signalpath do you have a recommendation for alternative measurements ? Are you advocating subjective listening as the way to establish the best sound quality ?

I'm not advocating for alternative measurements. I'm advocating that ASR not ascribe qualitative inference to distortion measurements.

Lower THD does not (necessarily) infer (1) better engineering, (2) better sound, (3) better __________.

 

[signalpath]

Otherwise, what can we call "well engineered"? Pretty? Sturdy? Uses gold?

Correct.

I'm speaking of distortion metrics below a scientifically-known threshold whereby perceptual performance is not effected. As another poster noted, these thresholds have been well-established over the decades, and our own AB lab tests have confirmed those findings.

 

[March Audio]

Correct.

I'm speaking of distortion metrics below a scientifically-known threshold whereby perceptual performance is not effected. As another poster noted, these thresholds have been well-established over the decades, and our own AB lab tests have confirmed those findings.

You are conflating technical performance with audibility. The test results presented here do not ascribe audibility to the numbers or rankings. As mentioned Amir is usually at pains to point this out.

 

[signalpath]

I disagree. A product that distorts the input signal less IS better engineered.

Sometimes it helps to strip an argument to its most basic, or most extreme, to see the reality.

Let's take a black box, has a power supply, and a place for an amplifier -- we'll call it a buffer. A simple input and output, say with A=2 (gain of +6dB). That's it. Nothing else. Actually, we have two of these boxes tied to an "AB" switch.

One black box (B) has a socket that allows us to swap amplifiers. Which we do! In fact, we swap a whole bunch of amplifiers and then compare the sonic performance of each amplifier with a known reference (box A).

In this test, we find that a $1.00 opamp with 0.001% THD sounds more pure and accurate and spatial, more true to the reference, than a $5.00 opamp with 0.0001% THD. We decide to ship the product with $1.00 opamp with poorer quantitative THD.

Tell me, which is "better engineered?"

 

[scott wurcer]

1.) Good engineering uses heroic measures (endless blind AB circuit listening tests) to determine "better" -- not THD numbers.

Simple low order non-linearity is one thing but we have had two cases here of transformers or RLC filters with level dependent frequency response at the .03-.1% level. They both have a strong following of people that love them while (IIRC) on one of them someone here found the effect audible and flawed.

Proper DBT's are very difficult to conduct properly, I have seen no evidence in most cases that they were. I don't know if you ever saw the "blind' taste test on Bobby Flay's show. The judges start criticizing while watching the faces of the two chefs, this is about the level of some of the "blind" listening tests I have seen.

 

[March Audio]

Sometimes it helps to strip an argument to its most basic, or most extreme, to see the reality.

Let's take a black box, has a power supply, and a place for an amplifier -- we'll call it a buffer. A simple input and output, say with A=2 (gain of +6dB). That's it. Nothing else. Actually, we have two of these boxes tied to an "AB" switch.

One black box (B) has a socket that allows us to swap amplifiers. Which we do! In fact, we swap a whole bunch of amplifiers and then compare the sonic performance of each amplifier with a known reference (box A).

In this test, we find that a $1.00 opamp with 0.001% THD sounds more pure and accurate and spatial, more true to the reference, than a $5.00 opamp with 0.0001% THD. We decide to ship the product with $1.00 opamp with poorer quantitative THD.

Tell me, which is "better engineered?"

The lower distortion amp which modifies the input signal the least IS the better engineered one in that particular measured aspect.

The problem here is that you are over simpifying the argument. No one here thinks this should be distilled down to one metric. That's a total straw man on your behalf.

Can we study the efficacy of your AB tests before we judge if your choices equal better engineered ?

Your implication is that your subjective conclusions drawn through your AB tests are not relate able to measured parameters. This would be odd considering that you have previously stated that below certain (easily measurable) limits we can't hear the difference.

 

[signalpath]

The test results presented here do not ascribe audibility to the numbers or rankings. As mentioned Amir is usually at pains to point this out.

I wish that were the case.

On the ASR page "Audibility Thresholds" it's noted that -120dB is a "strict" THD threshold (check me on that -- it's been a long time).

In a conversation with Amir (long ago), he shared that "yes, there are multiple ways based on psychoacoustics that we can calculate the distortion-free channel we need for full transparency" and restated that -120dB THD is required for "full transparency."

No.

If ASR wants to be a "scientific" gathering site, then this kind of non-scientific "belief" needs to be corrected. We do not need -120dB THD for "full transparency." This is not scientific. This isn't a metric arrived at by any known ABX perceptual testing. It's simply a "belief" that has no basis in scientific reality.

And this "belief" has clearly influenced ASR's qualitative description of SINAD and THD testing claiming that "higher is better" and using color codes to infer better performance and better engineering.

 

[pkane]

Sometimes it helps to strip an argument to its most basic, or most extreme, to see the reality.

Let's take a black box, has a power supply, and a place for an amplifier -- we'll call it a buffer. A simple input and output, say with A=2 (gain of +6dB). That's it. Nothing else. Actually, we have two of these boxes tied to an "AB" switch.

One black box (B) has a socket that allows us to swap amplifiers. Which we do! In fact, we swap a whole bunch of amplifiers and then compare the sonic performance of each amplifier with a known reference (box A).

In this test, we find that a $1.00 opamp with 0.001% THD sounds more pure and accurate and spatial, more true to the reference, than a $5.00 opamp with 0.0001% THD. We decide to ship the product with $1.00 opamp with poorer quantitative THD.

Tell me, which is "better engineered?"

As already been said, you are equating better sounding with better engineered. The two are not the same. It’s possible to have a better engineered, more transparent and true-to-the source design that sounds the same or worse to some people.

I agree with you, though, that SINAD ranking charts can make it look like higher SINAD is always necessary and desirable to those who are just taking a quick look. It would be much better to have some sort of audibility curve overlaid on top to indicate at what level SINAD becomes an audible concern. Ranking components only by SINAD is also incorrect, since there are other metrics that can be just as important for both, audibility and for transparency.

 

[March Audio]

I wish that were the case.

On the ASR page "Audibility Thresholds" it's noted that -120dB is a "strict" THD threshold (check me on that -- it's been a long time).

In a conversation with Amir (long ago), he shared that "yes, there are multiple ways based on psychoacoustics that we can calculate the distortion-free channel we need for full transparency" and restated that -120dB THD is required for "full transparency."

No.

If ASR wants to be a "scientific" gathering site, then this kind of non-scientific "belief" needs to be corrected. We do not need -120dB THD for "full transparency." This is not scientific. This isn't a metric arrived at by any known ABX perceptual testing. It's simply a "belief" that has no basis in scientific reality.

And this "belief" has clearly influenced ASR's qualitative description of SINAD and THD testing claiming that "higher is better" and using color codes to infer better performance and better engineering.

Considering that @amirm background was heavily involved with abx perceptual testing I don't think he would say that without basis, but I will let him answer that one. I think it will be an entertaining response. Amir simply doesnt make unsupportable statements.

 

[signalpath]

Simple low order non-linearity is one thing but we have had two cases here of transformers or RLC filters with level dependent frequency response at the .03-.1% level. They both have a strong following of people that love them while (IIRC) on one of them someone here found the effect audible and flawed.

Proper DBT's are very difficult to conduct properly, I have seen no evidence in most cases that they were. I don't know if you ever saw the "blind' taste test on Bobby Flay's show. The judges start criticizing while watching the faces of the two chefs, this is about the level of some of the "blind" listening tests I have seen.

Yup. Agree entirely.

Somewhere beyond the 0.5% range, my argument does not apply. But how much audio kit today exhibits 0.5% or worse THD? In those cases, I'm fine with a qualitative statement, color codes, etc.. at least as a caveat that says "this product has entered into the zone of THD audibility."

 

[March Audio]

In this test, we find that a $1.00 opamp with 0.001% THD sounds more pure and accurate and spatial, more true to the reference, than a $5.00 opamp with 0.0001% THD. We decide to ship the product with $1.00 opamp with poorer quantitative THD.

Tell me, which is "better engineered?"

Just curious, can I ask a, question? This is the second reference in the last few posts implying you design and manufacture products. Can ask which company?

 

[DonH56]

Like many design issues in audio, "it depends."

We've tested .0001% IC opamps that sound wrong and colored, and .0001% IC opamps that sound really good (we use them). We've done this kind of blinded AB testing for 30 years. I'm not saying we're the end-all, be-all in this conversation, but I will stand by our research and say confidently that distortion numbers very often have no correlation to the AB perceptual audio performance of an amplifier.

Is massive internal IC amplifier feedback the culprit? Maybe. In some cases. I frankly can't make the unequivocal statement. I remember a conversation with an audio IC designer from Burr Brown who shared his experience with massive internal feedback vis a vis THD and he said pretty much the same thing, and that's always stuck with me.

Yes, rather than using SINAD as a qualitative indication of audio engineering, I suggest presenting SINAD/THD/IM as quantitative measurements ONLY. No editorial as to "better" or "worse" or color ranking, or "better engineered," etc.. Split it out. Would rather see a pure signal-to-noise number without pilot tone, represented broadband and unweighted, naked and exposed without THD baggage along for the ride. Such is an example of a scientifically-accurate, qualitative representation, as are jitter, freq resp, linearity, filter responses, and many others.

I had some friends of BB (now TI) some years back FWIW. Who is "we" and what products? You are speaking from a position of authority, just curious the basis (NOT disputing it -- I do not know).

Many others have noted the low correlation between THD and audible performance, at least when THD drops below a fairly high (for these days) level. Given folk desire a simple, probably single, plot of "what's best" do you propose SNR then? Whilst many on ASR do understand all the technical details, at least to some extent, many others do not, and many folk visit to get some sort of ranking based on some sort of technical metric.

Aside: I had to set up a 3D plot of ENOB and SNR vs. frequency for a report ages ago. It made a pretty picture but wasn't that easy to follow if you wanted to pick numbers off the plot.

 

[signalpath]

BIO: Millennia Media, FPC. A 30-year-old pro audio house. Inducted into the Technical Hall of Fame, 2019. More classical music productions and film scores over the last 20 years have been recorded with our analog front-end. The U.S. Library of Congress (and scores of other labs) use our phono preamp for archiving priceless historical recordings. etc.

Read more: Forbes
Brief client list: Website

But "who" we are isn't the issue here.

Audio science is a passion, and I happen to really like this community and think Amir is doing a wonderful service to audio. But I want to see ASR rooted in unassailable good science. ASR's current qualitative approach to distortion, in my opinion, needs refinement.

 

[signalpath]

Given folk desire a simple, probably single, plot of "what's best" do you propose SNR then?

Personally, I wouldn't change much of anything. Specs are specs, if tested properly (and I have no reason to think Amir is doing anything to invalidate his test numbers).

All I'm saying is that ASR's existing method of discussing THD ("THD must be -120dB to be fully transparent) and displaying THD numbers ("higher is better") doesn't fit well with settled science. We don't need -120dB THD to deliver "fully transparent" audio signals. Implying that higher SINAD is "better" will be taken by non-technical readers as a qualitative statement.

I suggest that ASR's editorial on THD be revised. Remove the belief that "-120dB THD" is required for "full transparency." Remove the "better/worse" rankings of SINAD, and probably the "green/red" SINAD / THD graphical rankings.

Add a simple noise reading to the measurement mix: no signal, broadband (10Hz-20kHz), unweighted. In dBu or uVrms.

That's my suggestion.