Is SINAD important? - "Myths" about measurements! [Video YT]

[Sokel]

SINAD can be harmless when choosing amps and things are ok overall but can be very misleading in some cases.
And that cases have to be highlighted right beside it as their result can be VERY audible and can often cause debates if we don't know the specific set-ups.

Think of an amp that is load dependent for example.The best 1kHz SINAD in the world cannot predict if it will be true to source.

 

[Tell]

GoldenSound really is hit or miss, sometimes he brings good stuff and other times not so good, this time though I agree with him almost fully.

His demonstration with three different SINADs clearly shows that the same figure can sound quite differently and why people shouldn't use that figure alone to rank things or base their purchase on. Because that's really what's happening here on ASR, DACs and amplifiers are clearly ranked according to their SINAD and it's a bit misleading. Sure it might be a purely technical ranking based on how good the engineering is, but that's not what so many less technical people see when they read these measurements, they see a clear ranked list of stuff that's "Excellent", "Very good", "Fair" and "Poor" and might not buy that thing when they see that it's only on the yellow part of the scale, while in reality they probably won't hear a difference between even something on top of the list and something on the red. And yeah sure they should read on with the rest of the measurements, but even some long time readers here might not even understand it all so why should a newbie? They most probably just go for that ranking instead.
So even though I really appreciate what Amir is doing with all these measurements I do think this ranking is misleading, at least if it's not made very clear that it's about how good the engineering might be and not the actual audible sound quality. And tbh I think the only interesting thing to actually know is how good or bad it actually sounds imo.

Though I also think that GoldenSound should have been a bit more clear what SINAD those three examples where at, since apparently it was at 57dB which is really low. Sure it was a good very demonstration to compared how different the same SINAD can sound, but it should also be clear that (almost) no devices today even comes close to being that bad.

 

[K3N3]

Question about one part of the video, Cameron mentions something about this DAC which has a spatial effect enhancement feature. He says that usual metrics don't showcase the difference in sound (SINAD); but couldn't this just be explained by the FR output of the DAC? He showcases a spectrogram here to explain the difference, but I'm not sure if I might be misinterpreting it.

I think what he's trying to say is that SINAD doesn't showcase FR changes, but because he doesn't explicitly say that, I want to double check my own understanding.

Edit: https://forum.headphones.com/t/myths-about-measurements-discussion-thread/24879/29 gives a better look at what he means by this

 

[Chagall]

Are you sure? I saw at least 2 direct ASR references in his video and the single Harman target curve critique comes later and seemed a pretty obvious (if indirect) ASR reference.

Haven't check the whole video again but:

1st direct reference
..measurements are more widely available with resources like ASR, headphones.com, goldensound.com and L7audiolabs...more information being available is very good thing

2nd direct reference
He talks about how Amir and his (Golden) score differ based on the setup of AP, saying that it's important to be aware of test setups when comparing different sources of measurements...nothing serious.

Single Harman line can be an indirect reference to anyone, because only now headphones.com have started measuring and showing 2 lines, but filled in.

I don't know, but this to me is nothing.

As with most members am open to subjectivity and some latitude along those lines as long as it is not pushed as though it were equivalent to objective evidence. It is the subjectivist camp that seems to want to paint ASR members as not accepting of subjective opinion.

IMO we can do better. Some members can be very dismissive and can't be bothered. I mean we are asking new members did they ABX...they have no clue what that even is. If a new member has a problem with a specific amp, we just say sell that and buy Purifi....what use is that to the member? And get a UMIK, where are your in-room measurements? It's overwhelming...People don't want to get involved and leave nasty comments against ASR.

 

[Chagall]

What I would say to the membership at large is, if you don't have the patience to explain why someone is wrong *kindly* then just leave it alone. Catch more flies with honey, etc.

+1

 

[antcollinet]

Sure it was a good very demonstration to compared how different the same SINAD can sound

Except it isn't. If he'd used exactly the same distortion and noise profiles, but scaled to result in a SINAD equivalent to (say) the bottom of the green part of the chart (around 80dB) they wouldn't have sounded different at all. Why after all, do you think, he had to go down below 60 to make his point.

 

[krabapple]

Though I also think that GoldenSound should have been a bit more clear what SINAD those three examples where at, since apparently it was at 57dB which is really low. Sure it was a good very demonstration to compared how different the same SINAD can sound, but it should also be clear that (almost) no devices today even comes close to being that bad.

And isn't omitting that, rather more potentially misleading than Amir's ranking and bundling of SINADs? Your concern is that people won't properly appreciate the 'message' of SINAD. But no one will go wrong, sonically, by dumbly adhering to Amir's bins, while dumbly concluding that SINAD doesn't matter, from the kid's demo, could lead to buying an audibly poor device.

 

[Chagall]

Question about one part of the video, Cameron mentions something about this DAC which has a spatial effect enhancement feature. He says that usual metrics don't showcase the difference in sound (SINAD); but couldn't this just be explained by the FR output of the DAC? He showcases a spectrogram here to explain the difference, but I'm not sure if I might be misinterpreting it.
View attachment 419198

I think what he's trying to say is that SINAD doesn't showcase FR changes, but because he doesn't explicitly say that, I want to double check my own understanding.

Of course, it's changing FR. That's like saying RME DAC with EQ on is more preferable to listeneres but has the same SINAD as with EQ off... not a strong point.

 

[ta240]

I don't have time to waste watching videos. Please can you summarise?

This should be standard for any video that is posted.

 

[Giulio]

Of course, it's changing FR. That's like saying RME DAC with EQ on is more preferable to listeneres but has the same SINAD as with EQ off... not a strong point.

I don't think so. He shows also the FR.

I think it has something to do with crosstalk.

But that's actually a good point he is rising about not everything being detectable by standard ASR set of measurements.

 

[kyuu]

But that's actually a good point he is rising about not everything being detectable by standard ASR set of measurements.

I don't think detecting a spacial enhancement feature is something that's normally worried about? If there's an actual audible problem with a piece of gear that's not being captured in the current measurement suite, that would be interesting. I don't think that example demonstrates anything to that effect however.

 

[beagleman]

TLDR

Lots of talk about SINAD and how little it says and makes an example of 3 files with the same SINAD sounding different.
That SINAD is 57 ... yes folks 57 where the whole article is about certain people (ASR) are moaning about differences between 120 and 119 and stating 120 is better.
This is typical for ASR haters to say this.
What they gladly forget is that Amir has 4 tiers called Excellent (130-110), very good (110-100), fair (100-90) and poor (below 90) with the worst ones between 60 and 43.
The nagging about 1 SINAD difference and how that does not relate to sound quality goes on and on.
Yet the audible example is 57 SINAD... I wonder how many viewers notice this.
Cameron would have been more convincing if he showed samples at 90 SINAD but sadly that won't do well as an example. Why ... ? because SINAD between fair and excellent is not really audible (assuming FR and other aspects are also O.K.).
If he would repeat the same 3 'same SINAD' files but with say 100 SINAD the 3 files would be indistinguishable... very good thus is very good regardless of what type of 'unwanted signal' determines the SINAD number.
That does not invalidate the conclusion that fussing over this number (> 90 SINAD) from a 'sound quality' p.o.v. is pointless which basically was the 'beef' he has with the SINAD number and its (alleged) importance.

Yes there is ranking but NOT ... again .. NOT because it has better sound quality but just that it has better engineering (signal fidelity) at a certain level and 1kHz.
Amir nowhere states that this ranking has direct relevance to (perceived) sound quality. It only shows signal fidelity at 1kHz in a comparable way.

Otherwise I have to agree with most that was said in the first 30 mins (yes I used 2x speed to make it 15 mins).
The SINAD part could have been said in 2 minutes and without discrediting Amir and ASR as often as he did (without mentioning ASR and Amir but we all know it... don't we ?).

Funnily enough in the last 10 minutes about cables he does give Amir kudos about his cable video...

Agreed on all the headphone stuff... Cameron is right until he touches on the new way of displaying the tolerance band they use.
That totally does not jive with all that he said about HRTF and fixtures nor targets.
They measure a headphone on 2 fixtures they have and the the difference between these 2 would be the 'tolerance band'.
This is ridiculous... it is as if those 2 fixtures they have represent the outer bands of HRTF. It isn't.

Funny how he uses the 'Harman tile' I made but does not give credits... sure it was something Dr. Olive stated but still...

The last 10 mins is all bla bla and opinions on how things should be handled between 'camps' and acc. to Cameron how it 'should' be handled. Funnily this is almost exactly how this (often) is done at ASR. Granted.... not everyone does and indeed some folks can react 'harsh' if the same claim is done day after day, year after year.

I think SINAD is more audible when you know its not a great number.

If that makes no sense, meaning people will "hear" based on measurements to some degree.
I agree higher is better, but audibility, should be based on it being an audible problem, before a measurement is known.

 

[krabapple]

What I would say to the membership at large is, if you don't have the patience to explain why someone is wrong *kindly* then just leave it alone. Catch more flies with honey, etc.

Who wants to catch flies?

 

[beagleman]

The video is disappointing in reality,

but I thought it could open up debate In fact, the conclusion is "don't rely only on measurements" and "don't rely only on the ears"


if this OP is a problem, indeed it would be better to delete

That has been one of my big things in audio.

Measurements are great, love em.......

Can I hear it? audible at all, yes or no?

Can I hear it in real world usage? Will I really hear it and it will call attention to itself, or is it more an exercise in Sheer audibility?

 

[sonci99]

I think one has to chose a device parameter to chart those dac, it can be Sinad or Jitter or Fr, in the end none will make 2 dacs in the chart to be clearly superior to one another.

 

[RickS]

Haven't check the whole video again but:

1st direct reference
..measurements are more widely available with resources like ASR, headphones.com, goldensound.com and L7audiolabs...more information being available is very good thing

2nd direct reference
He talks about how Amir and his (Golden) score differ based on the setup of AP, saying that it's important to be aware of test setups when comparing different sources of measurements...nothing serious.

Single Harman line can be an indirect reference to anyone, because only now headphones.com have started measuring and showing 2 lines, but filled in.

I don't know, but this to me is nothing.

Will readily admit bias in favor of ASR, but regardless of his specific statements, my point was Cameron claims to want to bridge the objective/subjective divide and falls flat in my opinion. Here is why…

1. His video starts with promoting goldensound.com and headphones.com. Clearly he has very commercial interests.
2. No attempt is made to show why he is more credible than any other internet reviewer/influencer/hobbyist. Why should he get special consideration when he does not demonstrate any authoritative industry credentials.
3. If he had industry creds, my earlier post critique still applies. His overall arguments are a compromise that (while I agree with some if his stances) that are ultimately unconvincing.
4. He could make his arguments without any reference at all to ASR. So what is his motivation to do so? Clearly the overall tone is not to bolster measurements but to tear them down when they may come up shy. If a bridge was to be built, a collaboration with Amir (or other measurement advocate) would have been much more convincing.

IMO we can do better. Some members can be very dismissive and can't be bothered. I mean we are asking new members did they ABX...they have no clue what that even is. If a new member has a problem with a specific amp, we just say sell that and buy Purifi....what use is that to the member? And get a UMIK, where are your in-room measurements? It's overwhelming...People don't want to get involved and leave nasty comments against ASR.

Agree that some members are dismissive but that does not represent Amir or the ASR membership as an entity. We are also dealing with subjectivist trolling and some new members that are anxious to post opinions without investing any effort to read previous content beforehand.

It is easy to find fault, but is much harder to offer meaningful solutions. Maybe some sort of new member orientation or some topic areas that are required reading before posting? Maybe staffing up moderators for specific topic areas? Thought would try a little quick brainstorming as a more positive approach.

 

[Chagall]

I don't think so. He shows also the FR.

I think it has something to do with crosstalk.

But that's actually a good point he is rising about not everything being detectable by standard ASR set of measurements.

Specifically, what they do to achieve the effect I don't know, but the test is a delta spectrogram that shows changes in frequency over time between the same file with the feature on and off. The feature deasn't matter but his point is that wouldn't be captured by a standard test and that is fine as it's not a standard feature.

 

[Tell]

Except it isn't. If he'd used exactly the same distortion and noise profiles, but scaled to result in a SINAD equivalent to (say) the bottom of the green part of the chart (around 80dB) they wouldn't have sounded different at all. Why after all, do you think, he had to go down below 60 to make his point.

He was demonstrating that the same SINAD can sound different and that's most easily done with a exaggerated example like this, but you are of course correct as always. But then the question is why SINAD is used at all for ranking? If it's for engineering masturbation of sort then that should be very clear to not confuse people, because LOTS of them clearly are since this is even a topic in so many places.

 

[Giulio]

I don't think detecting a spacial enhancement feature is something that's normally worried about? If there's an actual audible problem with a piece of gear that's not being captured in the current measurement suite, that would be interesting. I don't think that example demonstrates anything to that effect however.

Not being able to detect spacial enhancement is worrisome to me: that also means that if a device has a flaw which results in a spacial "diminution", we wouldn't detect that

 

[Chagall]

Will readily admit bias in favor of ASR, but regardless of his specific statements, my point was he claims to want to bridge the objective/subjective divide and falls flat in my opinion. Here is why…

1. His video starts with promoting goldensound.com and headphones.com. Clearly he has very commercial interests.
2. No attempt is made to show why he is more credible than any other internet reviewer/influencer/hobbyist. Why should give him any consideration when he does not demonstrate any authoritative industry credentials.
3. If he had industry creds, my earlier post critique still applies. His overall arguments are a compromise that (while I agree with some if his stances) that are ultimately unconvincing.
4. He could make his arguments without any reference at all to ASR. So what is his motivation to do so? Clearly the overall tone is not to bolster measurements but to pick apart when they may come up shy. if a bridge was to be built, a collaboration with Amir (or other measurement advocate) would be much more convincing.

1. yes, especially when he talks about WANDLA, a DAC Ferrum made in collaboration with him - he should have used a different product to make his point
2. ok, sure
3. some are unconvincing but some are on point - our bias plays a part here for sure and there is nothing wrong with that, we just need to be aware
4. I don't know...if he didn't mention ASR at all, we would still find something to complain about

Agree that some members are dismissive but that does not represent Amir or the membership as an entity.

Yes agreed.

We are also dealing with subjectivist trolling and some new members that are anxious to post opinions without investing any effort to read previous content beforehand.

I won't act like it's no big deal, but trolls are usually spotted and, thanks to moderators, dealt with fast. But your average Joe Shmo, maybe would investigate and make an effort if we (not talking about Amir) don't make him feel like a total noob. And speaking of no patience: 6 people asked for the video summary. Short fuses all around

It is easy to find fault, but is much harder to offer meaningful solutions. Maybe some sort of new member orientation or some topic areas that are required reading before posting? Maybe staffing up moderators for specific topic areas? Thought would try a little quick brainstorming as a more positive approach.

Agreed. Something like those Kurzgesagt – In a Nutshell short videos would do wonders to orient new members and explain basic terms and methodology...30 sec video on why we say: Did you do a blind test?..for example. Maybe don't let them post if they didn't watch 2 or 3 short ASR introductory videos.