Why SINAD is not important (article)

[mhardy6647]

Single score to rank devices is meaningless and brings useless results. Deep analysis is always needed.

cf. e.g., IQ.

(edit: or standing start quarter-mile time for automobiles)

The problem with surrogate markers is it's often easy to game the system.
For some reason, I was thinkin' about VW Diesels when I typed that last sentence.

 

[amirm]

SINAD is not important but look this amp we just reviewed has liquid mids!

That's really the motivation behind online complaints. Let's get rid of objective measurements so that we can invent what we want about sound. Where is Andrew's video with science references in how he judges audio products? Missing in action....

 

[Mad_Economist]

That's really the motivation behind online complaints. Let's get rid of objective measurements so that we can invent what we want about sound. Where is Andrew's video with science references in how he judges audio products? Missing in action....

We're working on that, actually - while I hesitate to commit to something on Andrew's behalf, I know he's specifically solicited my input on forthcoming work on an explanation of the Harman target, headphone measurement methodologies and metrics, and an ABX comparator.

I think you read Andrew's motivations very wrong - he's profoundly interested the science of audio, and wants to bring value to viewers and readers with measurements. He's still in the process of learning, but that can be said of all of us.

 

[Newman]

There's a bunch of metrics that have significantly better correlation with audio quality, for example the GedLee metric i wrote a small bit about here in a similar post https://www.reddit.com/r/headphones/comments/ltwyz9
This is not the only such metric in existence either.


Edit: i would i agree with the sentiment others have mentioned here that maybe we shouldn't want a single metric, but if we wanted to, there's very much better ones.

Bear in mind though, that the Gedlee metric has no relevance to real life gear. You are much over-promoting it.

This thread looks like a storm in a teacup.

 

[amirm]

https://www.headphones.com/communit...SwYDvUwaDFbEiIW3f0BwyHPCd462GvSonth2QibjyBvrY

Didn't see this posted, thoughts?

There are some issues in there.

1. No one here had said SINAD is about audibility. SINAD is always, always shown in the dashboard in my reviews where you see the full spectrum of noise and harmonics in the FFT. I routinely analyze that and comment on whether there are audible concerns. I have also talked about limitations of THD+N in my article I wrote for Widescreen Review: https://www.audiosciencereview.com/...derstanding-digital-audio-measurements.10523/

"Below THD+N is the SINAD figure I referenced earlier. SINAD is actually the same value as THD+N but expressed in dB relative to the signal amplitude. This is good because we can, at a glance, compare the SINAD to our hearing dynamic range of 116 dB and know if we are better or worse than it. This is a back of an envelope computation for audibility so don’t completely run with it. Use it as a first order approximation. If a DAC has better SINAD than 116 dB, then we have very high confidence of transparency. If the SINAD is below 116 dB, then the nature of the distortion spectrum will tell us if the distortion is audible or not (i.e. we have to look at the FFT). "

As I have always said, when you achieve high enough SINAD, you get a free pass in having to learn psychoacoustics. If you do not, now you are in a hell of a situation having to analyze all aspects of distortion and prove such is inaudible. And for what? There is a ton of gear today that achieves my transparency metric. Why on earth do you want to dabble in more non-linear gear as to then defend it this way?

2. The masking thresholds are for general population. While with linear distortions (e.g. frequency errors) we all have similar acuity, when it comes to non-linear distortions, this is not the case. Importantly, training helps a ton. I am able to pass transparency ABX tests that 99% of the people cannot. Training lowers masking thresholds substantially.

Beyond trained listeners, I have also run into individuals that not only matched my listening abilities, but exceeded it! Someone younger for example should be able to do much better than me.

So be careful in running with graphs in textbooks. Read books like Zwicker and Fastl and see how their listening tests are done (usually a handful of people -- likely university students).

To be sure, most audiophiles are bad at hearing non-linear distortions so if you want to go by that, you can. But please don't generalize if you are not trained yourself or don't know about training.

The above also applies heavily to some distortion tests of speakers and headphones. One was mentioned in Andrew's video which was a collaboration between Sean Olive and Listen Inc. That result simply is not informative. You have to use a) two headphones with identical frequency response and b) including trained listeners who are able to hear non-linear distortions. Otherwise you have ad-hoc results.

Tests of audio compression in research always include this type of trained listeners for the above reasons. We don't want to put our head in the sand and think everyone is as bad as the general public and leave audible distortions in the codec.

3. Minor thing but the historical context is quite wrong. SINAD is an old term but was never used in the manner we are. Here is the Wiki definition of it:

As you see, it is a power metric and is strictly used in the context of communication/RF. We don't use power in SINAD or THD+N.

SINAD's use as equivalent to THD+N (inverted and expressed as dB) is an invention or extension made by Audio Precision in its measurement software. And this use has only been the case in the newer AP software so it has only been around 10 or so years, not a century as implied in the article and video.

It has become famous and gotten the meaning due to my use of it, happy or sorry to say. When I first started to measure DACs, I only looked at THD+N. Quickly I lost track of what DAC produced what. I turned on SINAD in the dashboard and immediately liked that it was in dB and bigger numbers meant better unlike THD+N which is backward and fully of decimal places. In order to keep track of things, I started to make a bar graph out of them which has not grown to incredible number of products.

Conclusions
The premise of the article/video is wrong. It promotes SINAD to audibility metric and then complains with limited view that it is not. Well shoot, ask us and we would have told you the same thing. To declare it useless and something to be avoided would take a hell of a lot more than what is in the article and video.

 

[amirm]

I think you read Andrew's motivations very wrong - he's profoundly interested the science of audio, and wants to bring value to viewers and readers with measurements. He's still in the process of learning, but that can be said of all of us.

I see him parroting what you have told him/written in that video. He needs to be more clear about that and not try to act as an authority on this topic which he can't possibly own. He should have had you do the video, not read and repeat as if he understands it all. He doesn't.

I have watched a ton of his videos as I research headphones that I am reviewing. While a lot of his content is good, his commitment to science is lukewarm. He either doesn't care more about it or worries about alienating his audience. So no, I am not where you are with him unless he is in the process of changing beyond what he has published so far.

 

[Newman]

I see him parroting what you (Mad_Economist) have told him/written in that video.

…and I see he has included the red herring that is the Gedlee metric.

 

[restorer-john]

@amirm, The entire unnecessary debate can be settled by posting a residual noise figure (with a specified bandwidth) in addition to the SINAD and FFT you already post in each review.

 

[Doodski]

It has become famous and gotten the meaning due to my use of it, happy or sorry to say. When I first started to measure DACs, I only looked at THD+N. Quickly I lost track of what DAC produced what. I turned on SINAD in the dashboard and immediately liked that it was in dB and bigger numbers meant better unlike THD+N which is backward and fully of decimal places. In order to keep track of things, I started to make a bar graph out of them which has not grown to incredible number of products.

Conclusions

Just a slight spelling error there? @amirm

 

[antdroid]

I see him parroting what you have told him/written in that video. He needs to be more clear about that and not try to act as an authority on this topic which he can't possibly own. He should have had you do the video, not read and repeat as if he understands it all. He doesn't.

He credits him in the very first minute of the video and also posts links to all his references in the video description.... (at about 35 seconds in...)

 

[Mad_Economist]

"Below THD+N is the SINAD figure I referenced earlier. SINAD is actually the same value as THD+N but expressed in dB relative to the signal amplitude. This is good because we can, at a glance, compare the SINAD to our hearing dynamic range of 116 dB and know if we are better or worse than it. This is a back of an envelope computation for audibility so don’t completely run with it. Use it as a first order approximation. If a DAC has better SINAD than 116 dB, then we have very high confidence of transparency. If the SINAD is below 116 dB, then the nature of the distortion spectrum will tell us if the distortion is audible or not (i.e. we have to look at the FFT). "

Bear in mind the the qualification holds if and only if the SINAD is measured at a level which would produce an acoustic output approximately at the threshold of pain with the playback device. A 2Vrms SINAD does not approximate this unless the headphone in question has a sensitivity of approximately 114dB/V, and variations bother above and below this exist - you're also using SINAD as an SNR proxy here, so, again, why not just use SNR?

2. The masking thresholds are for general population. While with linear distortions (e.g. frequency errors) we all have similar acuity, when it comes to non-linear distortions, this is not the case. Importantly, training helps a ton. I am able to pass transparency ABX tests that 99% of the people cannot. Training lowers masking thresholds substantially.

This is an interesting contention - do you have citations on the impact of training on consistent differentiation thresholds for harmonic distortion? I looked into that briefly in the background for this work, but I'm always happy for new papers to chew on.

SINAD's use as equivalent to THD+N (inverted and expressed as dB) is an invention or extension made by Audio Precision in its measurement software. And this use has only been the case in the newer AP software so it has only been around 10 or so years, not a century as implied in the article and video.

While I'm unsure of whether Audio Precision is the first to invert THD+N in audio testing, lumped distortion+noise measurements that are mathematically equivalent should surely be considered equivalently. I actually wanted the video title to be "THD+N (a.k.a. SINAD)", FWIW.

Re: "nobody using it as equivalent to THD+N", however, I know for a fact that Analog Devices used that usage prior to 10 years ago.

Edit:

Conclusions
The premise of the article/video is wrong. It promotes SINAD to audibility metric and then complains with limited view that it is not. Well shoot, ask us and we would have told you the same thing. To declare it useless and something to be avoided would take a hell of a lot more than what is in the article and video.

Apologies to edit this in, I somehow missed this line. If SINAD is not pertinent to audibility, what is the use of it?

I see him parroting what you have told him/written in that video. He needs to be more clear about and not try to act as an authority on this topic which he can't possibly own. He should have had you do the video, not read and repeat as if he understands it all. He doesn't.

If you'd like, I can be the voiceover on the next one I'm a lot less easy on the camera, though.

I have watched a ton of his videos as I research headphones that I am reviewing. While a lot of his content is good, his commitment is science is lukewarm. He either doesn't care more about it or worries about alienating his audience. So no, I am not where you are with him unless he is in the process of changing beyond what he has published so far.

I think you'll be intrigued with the direction that Headphones.com seems to be intending to move with regards to measurements and audio science in the near term. However, it is going to be dominated by questions of audibility, so if that's a contention point, perhaps not.

 

[Mad_Economist]

…and I see he has included the red herring that is the Gedlee metric.

FWIW, I'm not Chocomel, and the Gedlee papers were only one of a few with alternatively structured nonlinearity weighting schemes I provided for this piece. A future piece may look at the work of Brunet, Geddes, Pahomov, and others in producing different refinements on nonlinear distortion weighting, the Geddes reference here was primarily to target THD, which shouldn't be a contentious position.

 

[JSmith]

There is also a car called Sinad;

https://en.wikipedia.org/wiki/Kish_Khodro

The purpose for creation of Kish Khodro was to produce a vehicle named Sinad. The air conditioned vehicle has a 1600 cc Renault K7M engine and mechanical components. The Sinad body was manufactured using the Vacuum Injection Shell Process (VISP) The Sinad was successfully crash tested and obtained the MIRA/VCA certification allowing Sinad circulation in the UK, Europe and in other parts of the world. The front traction Jeep look- alike vehicle was named Sinad in Iran where some 1800 Sinads were manufactured.

… just sayin'.

If SINAD is not pertinent to audibility, what is the use of it?

Did you miss this part, or?

If a DAC has better SINAD than 116 dB, then we have very high confidence of transparency. If the SINAD is below 116 dB, then the nature of the distortion spectrum will tell us if the distortion is audible or not (i.e. we have to look at the FFT).

JSmith

 

[restorer-john]

While I'm unsure of whether Audio Precision is the first to invert THD+N in audio testing, lumped distortion+noise measurements that are mathematically equivalent should surely be considered equivalently.

THD(+N) has been expressed either as a percentage or x dB down forever. My vintage THD meters express S/N in dB and the distortion ranges/meter scales are marked in both dB and percentage. Once we started getting several zeroes below 1%, dB is far easier to read.

SINAD is a funny metric as it flips the number on its head to be a positive. When I see major manufacturers getting confused with their own literature and specifications saying things like "impressively low S/N ratio" it becomes obvious there is a lot of confusion. SINAD as a single lumped metric certainly doesn't help with that IMO.

 

[Mad_Economist]

If SINAD is not pertinent to audibility, what is the use of it?

Did you miss this part, or?

If a DAC has better SINAD than 116 dB, then we have very high confidence of transparency. If the SINAD is below 116 dB, then the nature of the distortion spectrum will tell us if the distortion is audible or not (i.e. we have to look at the FFT)

No, I was being somewhat rhetorical - SINAD is clearly being used as a proxy for a threshold of audibility here, it's just a very poorly suited measurement for it.

 

[MC_RME]

I always measure THD+N in dB. Even my oldest audio analyzer (1992) can do this. Using the inverse SINAD just gets rid of the minus in front. Claiming one is in percent and the other in dB is not correct.

Oh, @restorer-john beat me to it.

 

[Mad_Economist]

When I see major manufacturers getting confused with their own literature and specifications saying things like "impressively low S/N ratio" it becomes obvious there is a lot of confusion.

I still find it kind of weird that the world settled on THD expressed as a ratio of nonlinearity to total, and SNR expressed as a ratio of total to noise, yeah. To be honest, the "signal over" format makes perfect sense, but it's not really meaningful whether you choose to put the signal or the not-signal of the left side of the /, so long as you're clear about which is which.

 

[Mad_Economist]

I always measure THD+N in dB. Even my oldest audio analyzer (1992) can do this. Using the inverse SINAD just gets rid of the minus in front. Claiming one is in percent and the other in dB is not correct.

Oh, restore-john beat me to it.

Er, wait, did somebody say that THD/+N is always expressed in percent? If that's in the video, I'll let Resolve know, we should edit that.

 

[restorer-john]

Er, wait, did somebody say that THD/+N is always expressed in percent?

The issue was/is that old analogue THD meters had no way of separating the THD and the noise from the residual after the notched out fundamental. So much of the single 'THD' figures in the past, were really THD+N figures. Now of course, it's trivial with computer based FFT to measure the harmonics alone.

I'm not sure when HiFi first started specifying THD numbers that weren't THD+N. Probably around the time when spectrum analyzers got good enough to have noise floors low enough. The thing is, specified numbers all of a sudden got better in the late 1970s and I often attribute that to the removal of the residual noise in the THD+N, and becoming THD only.

It meant comparisons were not really fair and there was plenty of great specified gear with considerably high noise floors that were not specified separately. Hence my call for a residual noise number.

 

[JSmith]

... it's just a very poorly suited measurement for it.

One can only assume that is why many other measurements are conducted and posted in ASR measurement reviews. I think it suits the purpose of a metric for ranking products tested in a graph format.



JSmith