Erin's Critique of the Sinad-Based Evaluation System

[Talisman]

This interesting comment is a reply to one of his latest videos about an abx test between a McIntosh and a Fosi audio.
Erin takes a veiled jab at the sinad-based electronics rating system when it is clearly above audibility (and I agree with him on this).

 

[KenMasters]

I don't see it as a criticism of SINAD as a metric, but rather of the practice of comparing metrics in general by individuals for who's use case they have little to no impact.

For me that's no fault of the person or persons conducting the testing and posting their findings, it's the fault of the individual who hasn't taken the time to understand their implications. It's no different from someone who compares the stickers on the box to decide which TV to buy.

At the end of the day the consumer needs to bear some responsibility for their choices.

 

[Luvchampagne]

My take on it is very similar to yours. Bad measuring devices will usually be poor performers. Good measuring devices is a start. Non audible differences in measurements don't make one device necessarily better than another.

 

[Chester]

His statement is a little bit of a contradiction, to me.

If he’s talking about devices that are well beyond the threshold of hearing as he appears to be, ABX testing is also pointless, surely.

 

[Thomas Savage]

Folks playing SINAD top trumps should be distinguished from folks looking for actual audio performance relative to the use of the equipment as a play back system .

Should there be more of a focus on ABX rather than SINAD , possibly. Certainly it would be more helpful imo if there were , but as has been mentioned the responsibility lay at the feet of the consumer. As mentioned by Erin , if ones going to start talking about how things sound then ABX is a must . But then thats a contradiction in terms as once past audio thresholds things all sound the same ... .. .

To a degree, if we were to just accept audible thresholds then really most of us should then all go home as audio is a solved problem outside of making the speakers work in your listening space.

All said i see no criticism in what Erin is saying regarding all things SINAD. It has become something it isn't, to some audiophiles imo .

 

[dweeeeb2]

His statement is a little bit of a contradiction, to me.

If he’s talking about devices that are well beyond the threshold of hearing as he appears to be, ABX testing is also pointless, surely.

Well ABX is used to confirm that a component is good enough. SINAD is just one metric which can discriminate between items which actually perform subjectively the same

 

[BeeKay]

The catch is SINAD is easily repeated and proven. ABX results on the other hand, well …

 

[solderdude]

Erin takes a veiled jab at the sinad-based electronics rating system when it is clearly above audibility (and I agree with him on this).

I don't think so... it is rather an explanation of the usefulness of SINAD number.
Erin also mentions he does not know of review sites doing (well executed and documented) blind tests of gear.
That is logical as the 'alleged' differences would not be provable... alas it is difficult to do in most cases, extremely tiring and time consuming, requires expert knowledge and suitable test equipment.

SINAD is just a metric showing a single number.
SINAD is NOT a measure of sound quality.
Above a certain number the metric is not telling much (other than that gear has low noise and low distortion at 1kHz at a certain level.
Even low SINAD gear can still sound great.

It is a metric indicating signal fidelity of a 1kHz sine-wave at a specified level combined with noise. Nothing more, nothing less.

It is the anti-ASR folks that use the SINAD ranking chart as a reason to whack the dog.

 

[Multicore]

I really have no idea what Erin is trying to say here.

1. He says he doesn't know any data-oriented reviewers who ABX test electronics, i.e. amps. Is this supposed to mean that data-oriented reviewers should know that amps do in fact sound different when operating inside their performance envelope?

2. He says objective reviewer's rankings of amps criticize amps just because they don't have enough SINAD even when it's beyond hearing. I don't know who does that. Amir doesn't do that. He measures amps' performance envelopes so that we may better understand their applicability and value proposition.

3. He posits the dichotomy: it's either a) subjective "trust me bro" or b) objective reviewers who criticize amps just because they don't have enough SINAD even when it's beyond hearing. That's a false a false dichotomy. If he's trying to imply Amir and others who post measurements on ASR then they may reasonably feel insulted.

4. He said that ABX tests (presumably listening tests) are a fundamental aspect of amp reviews. But why are they? It suggests he believes that amps sound different inside their performance envelope. But just a moment ago he acknowledged that they don't with the beyond audibility remark.

Let's hope Erin can elaborate a bit and clarify.

 

[Palladium]

The catch is SINAD is easily repeated and proven. ABX results on the other hand, well …

Also who is even claiming electronics SINAD after an easily attainable threshold that is way beyond transducers is audible?

Talk about a strawman.

 

[Audionaut]

It is interesting what Amir himself says about the relationship between measurements and listening:

I have a scale for how much measurements matter for each category of products:

DACs: 100%
Amplifiers (headphone and speaker): 80 to 90% due to variability of available power. Hard to internalize how much power is available/enough without listening tests.
Speakers: 70 to 80%
Headphones: 50 to 80% (measurements too variable)

This is why you see me do listening tests for the last two categories and half of second (headphone amps).

 

[Multicore]

It is interesting what Amir himself says about the relationship between measurements and listening:

Yes, thanks for finding that.

Amplifiers (headphone and speaker): 80 to 90% due to variability of available power. Hard to internalize how much power is available/enough without listening tests.

And we can close that gap by over-spec'ing amp, as Amir recommends. I don't have a quote for that but several times I read something to the tune that clean amp power is cheap these days so get more than you'll need and then you can trust that the amp is transparent.

 

[RickS]

Have plenty of discussion of the value of SINAD here already. Not sure how quoting Erin without more context helps and, since he does not participate here any longer, seems very speculative. Locking while I consider how to handle.