Why SINAD is not important (article)

[solderdude]

Is there any way to compare this SINAD to what Stereophile reports, i.e. SNR for 2.83 V, 8 ohm, audio band bandwidth and A-weighted?

Not really.
1: SINAD at max power may have a lot more harmonics than at 2.83V
2: The SN ratio is hard to deduct from the SINAD. The 'seen' noise floor isn't the actual noise as that noise floor is averaged so one can see mains hum and distortion products more clearly.
One can make deductions based of the measured S/N ratio and the output voltage of the 1kHz peak voltage though.
A weighting also makes it difficult as hum and highest frequencies are lower and thus different.

 

[RHO]

I think you read Andrew's motivations very wrong -

I'm sorry to say that I seriously starting to doubt it.
Judging from his remarks during his live-streams I'm getting the impression that he's more interested in discrediting ASR with misrepresenting the views of @amirm on measurements and subjectieve experience.
Never straight forward. Always insinuations.
I liked the guy, but lately he's disappointing me with this stuff time and time again.

 

[Dealux]

I'm sorry to say that I seriously starting to doubt it.
Judging from his remarks during his live-streams I'm getting the impression that he's more interested in discrediting ASR with misrepresenting the views of @amirm on measurements and subjectieve experience.
Never straight forward. Always insinuations.
I liked the guy, but lately he's disappointing me with this stuff time and time again.

Even though I like his channel I get that impression as well. Yeah, people here will sometimes trash expensive gear but meanwhile him and his friends are on a weird hate tirade against TOPPING and they all seem to collectively think that their amps sound bad somehow. It almost seems like they're angry that people are starting to make purchase decisions based on measurements instead of (questionable) subjective impressions.

The measurements just show that you can buy something more affordable and get the same sound quality. I'm aware that TOPPING has had QC issues with some products but that shouldn't completely destroy them as a company. I mean plenty of audiophile brands like Focal and Audeze have had horrendous failure rates with specific products (though they've probably gotten better) and people still buy their stuff.

 

[TheTalbotHound]

One can only assume that is why many other measurements are conducted and posted in ASR measurement reviews. I think it suits the purpose of a metric for ranking products tested in a graph format.



JSmith

If only those other measurements actually affected the recomendation. Sadly it seems that they don't. Aune S8, The original version of the Soncoz SGD1, and the SMSL SU8v2 all still got recommended despite their IMD humps, which, for ESS DACs at least, are known to coincide with audible issues. The v1 firmware version of the Tone2 pro, and Soncoz LA-QXD1 were still recommended despite their poor filter attenuation.

 

[Deleted member 27694]

Before you quit this topic, I legitimately am very interested in any citations - or even researcher or institution names - to look for in this regard. I've tried to make a fairly thorough study of the AES's work on distortion, as well as a body of research from some European universities, but I don't claim to know every paper ever published, and I'm always happy to read a new one.

Exactly. The topic should be solved if someone could provide research papers. Or does only anecdotal evidence exist?

Me and my untrained ears believe that most of what the audio community is obsessing about is of utterly unimportance for 99% of consumers. That doesn’t mean its wrong. But it is like someone has a perfectly clean apartment done by the best cleaning service imaginable and someone comes and says: „No, your apartment is not really clean and I can prove it“… and pulls out a microscope and a blacklight. Yeah right but…

Also, it's a bit vexing to be accused of bad faith both while I'm actively defending this community (and yourself) in the (predictably vile) Youtube comments on the video, and while you appear to be trying to "gotcha" me about proving a negative.

After reading the exchange in this thread without any knowledge of any wider context. I am totally with you.

 

[RHO]

Even though I like his channel I get that impression as well. Yeah, people here will sometimes trash expensive gear but meanwhile him and his friends are on a weird hate tirade against TOPPING and they all seem to collectively think that their amps sound bad somehow. It almost seems like they're angry that people are starting to make purchase decisions based on measurements instead of (questionable) subjective impressions.

The measurements just show that you can buy something more affordable and get the same sound quality. I'm aware that TOPPING has had QC issues with some products but that shouldn't completely destroy them as a company. I mean plenty of audiophile brands like Focal and Audeze have had horrendous failure rates with specific products (though they've probably gotten better) and people still buy their stuff.

Yes, the constant insinuations that the A90 sounds bad because it measures good... cringy ...
He never says it like that but I feel like that's how he wants it to come across for the subjectivist part of his audience. Even if he doesn't believe it himself.

 

[Dealux]

Yes, the constant insinuations that the A90 sounds bad because it measures good... cringy ...
He never says it like that but I feel like that's how he wants it to come across for the subjectivist part of his audience. Even if he doesn't believe it himself.

I think he believes it but it's based on flawed testing methodology and ultimately subjective impressions are unreliable where differences would be either very small or nonexistent.

He's said multiple times that he thinks it sounds compressed or lacks "dynamics". It's anyone's guess what that could mean really. We don't even know if he properly volume matched amps in his testing if such claims could have any validity at all.

 

[RHO]

I think he believes it but it's based on flawed testing methodology and ultimately subjective impressions are unreliable where differences would be either very small or nonexistent.

He's said multiple times that he thinks it sounds compressed or lacks "dynamics". It's anyone's guess what that could mean really. We don't even know if he properly volume matched amps in his testing if such claims could have any validity at all.

I do believe he actually thinks it sounds bad. I don't think he believes it sounds bad because it measures so good/clean.
I think most of his subjective claims about (non tube-based) amp/DAC soundsingnatures are commercially inspired.

 

[MikeJ]

I must totally disagree with that vague and poorly written article. But who really cares? From a mastering engineer's point of view (I am a long time studio guy), I can assure you that low THD+N/SINAD in your master AD/DA signal chain will introduce unwanted, audible non-linearity elements, which are difficult to deal with later in the process.

Simply concluding that "SINAD is not important" is either a pretty big misunderstanding of this whole topic (importance of THD+N has been discussed for years) or it's just a poorboy's attack to fight ASR. I see the latter, sorry. If you don't have serious, valid arguments, you then fight.

I think we all cannot be grateful enough for @amirm 's outstanding efforts, which are continuously helping to raise the bar for the whole audio industry, to never before seen levels. Think of it - why are some audio manufacturers afraid of posting important technical specs, just to later find out when someone measures, that their devices have some serious design flaws? And no, pretty marketing will not help there.

My advice to moderators - can you just lock / edit this thread so we can continue to discuss again more what we are used to discuss here on ASR?

No need to confuse existing / new members with this misleading stuff on topic of SINAD importance.

 

[Sugarbubble]

My take is that the article has academic merit but the title is pure journalism. Academic papers often have titles that run on for fifteen or more words. Changing the title to something like… an evaluation of the merits of using SINAD as a metric for audibility in typical listening environments…. might have been a better approach.

 

[dshreter]

This brings us to the conclusion, and a suggestion to readers who are looking to purchase a new piece of source equipment: Don’t bother with SINAD …. So long as the distortion is less than -80dB, and the noise is below 10dBSPL with headphones that have a similar sensitivity to the ones you're trying to run, you’re not going to hear a problem.

So couldn’t I just glance at SINAD to verify well designed gear has both inaudible distortion and inaudible noise levels? I don’t understand how the authors explain the importance of all the contributors to THD + N then arrive at a conclusion that it’s unimportant.

It’s like saying lead is the worst contaminant in your water but total dissolved solids is a stupid metric for evaluating a water filter.

 

[JJB70]

I think the mad_economist has elucidated his argument very eloquently. A key part of the scientific method is to test ideas and for exchange of ideas and for peer review/criticism.

 

[JJB70]

One issue I have with use of metrics is the assumption that they tell you something has been well engineered. No, they tell you how something performs relative to a specified criteria. Whether or not something has been well engineered is a much wider question and includes all sorts of things a performance metric doesn't tell you ( including, inter alia, component quality, assembly quality, thermal management, compliance with any applicable regulations, ease of servicing etc).

 

[MikeJ]

Academic merit / scientific paper? Posted on that site / YT in a form of short, misleading article about SINAD non-importance? C'mon guys. You seem to have absolutely zero idea how a proper scientific review / publishing of academic paper looks like. My good friend who successfully defended his PhD thesis in Computer Science last year can surely talk a bit about this process - X revisions / corrections, Y opponents from several different institutions / universities, W argument wars with them, and Z sleepless nights to finally get his paper widely accepted and most importantly, published in academic community. If @Mad_Economist ´s scientific efforts regarding this very topic ever get published in AES papers / journals, I will humbly stand corrected to accept new findings, but now I highly doubt that is going happen. Unless he's willing to argue with folks like Bruno Putzeys, John Siau or Audio Precision guys, just to name a few, to re-invent the wheel. Good luck!

 

[Jimbob54]

Academic merit / scientific paper? Posted on that site / YT in a form of short, misleading article about SINAD non-importance? C'mon guys. You seem to have absolutely zero idea how a proper scientific review / publishing of academic paper looks like. My good friend who successfully defended his PhD thesis in Computer Science last year can surely talk a bit about this process - X revisions / corrections, Y opponents from several different institutions / universities, W argument wars with them, and Z sleepless nights to finally get his paper widely accepted and most importantly, published in academic community. If @Mad_Economist ´s scientific efforts regarding this very topic ever get published in AES papers / journals, I will humbly stand corrected to accept new findings, but now I highly doubt that is going happen. Unless he's willing to argue with folks like Bruno Putzeys, John Siau or Audio Precision guys, just to name a few, to re-invent the wheel. Good luck!

I'm not sure the point of the article was to further the science in the field, more summarise it and suggest the way it is sometimes applied /presented is less than useful.

Let's not go overboard and critique it as something it doesn't pretend to be.

 

[PierreV]

It is a bit sad that valid discussions about technical issues almost always end up buried in some kind of low-grade mud fight between Internet influencers. We actually get more info about the main protagonists personal antipathies, possible covert agendas etc... than we get about the core question.

In the online audio-community the mere mention of "SINAD" has now become so strongly associated with this site that its mere mention immediately puts you in one group ( "with a SINAD of 115dB device x trashes the 112dB device y" -> "pro ASR") or the other( "SINAD doesn't tell you how a device sounds" -> "anti-ASR"). That, in itself, wouldn't be a problem, if it did not always devolve fairly quickly.

Yes, the title of the article/video is definitely superficial, catchy and, pure clickbait.

But on the other hand, the various SINAD ranking/tables on this site also are. How many times have we seen people say "wow, we have a new champion, I want it now!"?

FWIW, I tend to put a lot of faith in SINAD when it comes to DACs, signal reconstruction is, after all, a purely mathematical process and some single metric seems sufficient to assess its performance. But I trust it much less when it comes to amplifiers, especially at max power as John nicely explained. Yet, we see people enthusiastically endorse amplifiers over others for a 5dB difference at max power...

That's certainly not Amir's fault, as he provides a lot of additional measurements and commentary but this is, unfortunately, the catchy click-baity metric that has emerged in most audiophiles' mind. And it is always the higher number, in most cases the one obtained at full rated power, that is used while I feel the lower power one is the most interesting in practice.

OTOH, dismissing SINAD entirely (which is not what this article does), is of course, falling into magical thinking.

 

[Phorize]

My take is that the article has academic merit but the title is pure journalism. Academic papers often have titles that run on for fifteen or more words. Changing the title to something like… an evaluation of the merits of using SINAD as a metric for audibility in typical listening environments…. might have been a better approach.

@Mad_Economist perhaps consider tweaking the framing of the article. The article is well researched and interesting, but the framing is somewhat sensationalist and will predictably cause some number of people less well qualified than you to confuse the nuanced proposition that ‘SINAD alone isn’t a sufficient measure’ for ‘SINAD tells us nothing at all’. The former is obviously defensible, the latter an illogical audiophile trope.

 

[Jim Matthews]

I'm sorry to say that I seriously starting to doubt it.
Judging from his remarks during his live-streams I'm getting the impression that he's more interested in discrediting ASR...

I get the sense that there is no replacement to the ASR regimen at the ready. If there is a "better mousetrap" available, package it for sale in the marketplace of ideas.

To appeal to rubes like me, it must be explainable without requiring letters from another alphabet.

It should be readily reproducible and it should correlate to practical aspects I (as a consumer) will buy.

The fact that AmirM willingly entertains this challenge - on his own website - baffles me. If I have learned anything in the past 20 years of forum activity, it's that constructive criticism takes place in PM.

This has that 1980's cologne "Bad Faith debate" all over it.

 

[Scgorg]

I'm not sure the point of the article was to further the science in the field, more summarise it and suggest the way it is sometimes applied /presented is less than useful.

Let's not go overboard and critique it as something it doesn't pretend to be.

Absolutely. This article does not present any new research at all, nor does it pretend to. It's simply a summary of findings across a large body of work. In that sense it should be treated more like a secondary or tertiary source, not some new grand and illuminating information on the usefulness (or lack thereof) of SINAD. The papers this article was based on are all referenced at the end, so any interested individual is free to read and interpret them.

Personally I think @Mad_Economist has defended his position well, and if anyone wishes to harp on the contents of the article at this point they should at the very least do a similar level of diligence. That is, refer to studies that support their case. I agree that the title comes across as sensationalistic, but it's intended for a layman's audience after all. A forgivable sin, if you will.

 

[MikeJ]

I'm not sure the point of the article was to further the science in the field, more summarise it and suggest the way it is sometimes applied /presented is less than useful.

Let's not go overboard and critique it as something it doesn't pretend to be.

Why not give critique to something which aspires to be the new-truth and tries to somehow dismiss everything that has been said on this topic?

How many "ordinary" people can discern types of harmonic distortion products (odd / even) just by looking at the FFT and make a meaningful conclusion of it?

Do you think a DAC's output stage with massive odd harmonics (3rd, 5th, ...) and 80dB SINAD would sound good? I highly doubt so. Not to say that you need at least 96dB SINAD to faithfully reproduce CD audio (44kHz/16bit), unless you'd be using some unobtainium form of dithering process to mask truncation of ENOB. It's not that simple as it seems. I think only having a FFT profile to let the viewer decide the outcome would make things much more confusing.

Personally, SINAD is just as good as it gets to tell me the big picture about the device. It can tell you a lot of things.