Proposal: New SINAD Ranking Design (Histogram)

[Chromatischism]

Zooming & Panning in CanvasJS HTML5 & JavaScript Charts example.

How To - Image Magnifier Glass example.

No Photoshop required. Just good coding skills.

The first one yes, the second one wouldn't have a use here.

 

[amirm]

Zooming & Panning in CanvasJS HTML5 & JavaScript Charts example.

How To - Image Magnifier Glass example.

No Photoshop required. Just good coding skills.

Can't allow scripting/HTML in posts due to security reasons. So none of that works in the middle of a review page.

 

[sam_adams]

Can't allow scripting/HTML in posts due to security reasons. So none of that works in the middle of a review page.

Is there no way to allow for one user to do it? Maybe a way to embed a separate viewport into the page? Possibly consult with the Xenforo code-bros to see if it can be done?

 

[amirm]

I don't know what you mean by separate viewport. In the Review Index we can do this since the full page is under our control. To allow the same in the middle of any post means allowing scripting/html insertion by any user. Xenforo lets you enable this but with the biggest warning possible that it is a major security flaw. So no, it is not going to happen this way.

 

[JustJones]

Slice it

View attachment 222281

If it begins to get to long couldn't the relevant slice be used in the review instead of the whole list?

 

[Ra1zel]

I am on the camp of discarding the ranking all together to try to tame the blind sinad following. My 2c.

Voice of reason

On the second page we have the total opposite of the thread's intent where people are veering off into philosophical discussions about changing the entire grading system of SINAD itself...

No no, don't change it, just throw it out the window

 

[JSmith]

I suppose it doesn't matter too much, but this is more what I was referring to when suggesting "vertical". The review index is really the key, the point of this in general I suppose is just to show where the product ranks compared to others rather than something to actually choose a product from, without looking into it further and reading the whole review.


JSmith

 

[JeremyFife]

Are we mixing up the uses of these metrics here?

#1 The Review section (Review Index?) could usefully have a summary of all products reviewed - that full list; horizontal or vertical, interactive or not. That's interesting to me and useful for research

#2 Individual Product review could change. That increasingly large horizontal ranking is increasingly irrelevant (to me anyway).
It's very helpful to have a clear graphical representation of where the reviewed product places - but I only need to see if it measures "Very Good" or better. A useful view would be the four colour bands with a big arrow pointing to where this product lands. If you wanted to change the bands to give more resolution then that would be cool too.
Having sight of the products with similar SINAD doesn't help much in the context of the review - because if I'm considering that product I have probably already narrowed down my budget and the features that I need. If all the comparable (SINAD) products are 3x the price or e.g. don't offer USB then I'm not interested and I cannot tell this from the chart.

So, if we're making changes;
#1 Add a nice comprehensive graphic to the Review Index
#2 Simplify the graphic on the product review to just show where this product lands on the Excellent to Poor scale (no other products)

For what it's worth, I don't mind the current view - I just use it as #2 above to see where on the scale the product lands. I ignore the other detail

 

[Lambda]

If i would be shopping around for a new DAC (and i'm new to ASR) i would looks at this and i would be overwhelmed.
What I would care about is:
Waht’s better. In ruffly the same Price range or cheaper.
Or what is "The best" for my budget.

It would be nice to link to an interactive version of this overview in every review but i don't think including this picture in every review is helpfull

 

[JiiPee]

To keep it simple, forget the picture and just mention three things:
1. Measured SINAD in dB.
2. Is the measured SINAD clearly above, roughly on par, or clearly below the SINAD measured from other DACs.
3. Is the measured SINAD good enough in the sense that improving it would not produce improved sound quality that could be observed in normal music listening situation.

 

[rdenney]

It's not clear to me that Amir is ready to consider any change at all. But if he were, it would be the ideal time to replace the entirely arbitrary (and therefore unscientific) quality grades with those that have physical meaning. Let me try this again: How about these categories?

Absolutely Transparent: Distortion and noise artifacts imperceptible by specially trained listeners using special listening techniques such as maxing the volume during the reverberation tails and silent parts. (>115 dB?)

Functionally Transparent: Distortion and noise artifacts imperceptible by experienced listeners attempting to discern distortion and noise by increasing gain and listening in headphones. Assurance of sufficient distortion and noise headroom to play 16-bit encoded CD's or source files with no measurable degradation. (>90-100 dB?)

Practically Transparent: Distortion and noise artifacts imperceptible by experienced listeners in home listening environments, but with the possibility of audible artifacts if source equipment at this level is subject to significant downstream amplification. (>70-80 dB?)

Audible Distortion and Noise: Possibly audible by experienced listeners in home listening environments without using special listening techniques (<70 dB)

I put question marks for the exact boundaries because the research taken as a whole doesn't seem any more precise than the next lower decade on the logarithmic scale, and the exact numbers used can be discussed. These numbers are, as I recall from reading it a while back, consistent with the thread on audibility limits, which summarizes psychoacoustic research and respected expert judgments (including Amir's).

There are several categories below that, of course, and noise at -80 dB is more detectable in the silent bits than is harmonic distortion at -40 dB. And source devices subject to downstream amplification need different thresholds than amps and speakers, though that fact is certainly acknowledged in the reviews. But it seems to me that this would fulfill ASR's mission of informing the buying public using data and measurements rather than subjective opinions.

It is often argued (and I have argued it myself) that a high SINAD is evidence of good engineering and therefore has value independent of its audibility. But engineering admiration is rather unscientific as the basis for grading for those non-engineers trying to choose products intelligently, though entirely appropriate for the comments included in the review. Amir's review of, for example, the ifi Zen Blu reports digital Bluetooth processing of 124 dB SINAD, but the DAC (analog output) is "poor" at 80 dB in the worst channel. (The Big SINAD Graphic was not shown in that review.) I'm plugging that poor analog output into a 45-year-old budget-model Kenwood integrated amp that might have a S/N of 70 dB dripping wet, and that's feeding 30-year-old Canton bookshelf speakers, and it sounds excellent up to the limits of a 40-watt amp, playing music from my (limited in the Bluetooth domain) iPhone. I'm thinking that even with 40ish dB of input gain into that integrated amp, the amp runs out of room long before any artifacts from the Zen Blu will be audible, at least in the bedroom where it is set up. Had it been graded as "Absolutely Transparent" in the digital path and "Practically Transparent" in its analog output, it would provide more effective advice for those unable to read between the lines as I am able to do. (I did buy this device before Amir reviewed it.) Amir's recommendation was to use a better-measuring external DAC and use the Zen Blu just for its Bluetooth reception and processing, but that recommendation does not seem to me based on audibility limits in practical applications, but rather his admiration for other DACs with much higher SINAD measurement.

Anyway, enough of that. Summary: on the assumption that Amir would be willing to change anything at all (not in evidence), then maybe it's time to consider replacing the arbitrary and grading system with one that is linked to psychoacoustic audibility thresholds.

Rick "respectfully submitted" Denney

 

[Lambda]

is often argued (and I have argued it myself) that a high SINAD is evidence of good engineering and therefore has value independent of its audibility.

But SINAD is only THD+N at 1Khz at 2/4Vrms with a 44.1khz signal.
You can’t tell it something is transparent from it alone. you can only tell if it is not.

I would only make 2 main categories:
< 100/96dB "Below "CD quality""
"Not below "CD quality""

, and the exact numbers used can be discussed.

It all depends on listening level.

 

[JanesJr1]

Since its conception, the SINAD Ranking has grown from this:
View attachment 208069
to this:
View attachment 208066

Instead of just making the graph wider and wider with each addition, why not switch to a histogram:
View attachment 208067

Others probably will disagree, but I think that either a histogram or ranking has one flaw of too much info, and another flaw of the wrong info.

I agree with the idea of condensation, and as far as that goes, having a histogram without individual equipment labels may be helpful.

But my larger point is: what info about SINAD is actually useful? Does the graphic answer for any piece of equipment the key question of audibility? The actual information required for interpretation is something like (1) SINAD inaudible; (2) possibly audible under careful test conditions; (3) audible with critical listening; (4) routinely audible. Or something like that.

I know that a secondary purpose of the SINAD measurement is to detect the presence of a systematic, measurement-feedback engineering process. Wouldn't the previous four categories cover that, too? If not, what info would do that best?

The horse-race aspect of the current graphic may have by itself spurred better engineering. But it may have gone beyond that into a meaningless numbers game. I've been watching spates of competing-audio-spec fads since the 1970's, and we have to ask if we are feeding into another one.

EDIT: I just read RDenny's post a couple of lines up the list. I like that thinking, too.

 

[Keened]

I would only make 2 main categories:
< 100/96dB "Below "CD quality""
"Not below "CD quality""

Absolute transparent (120db)
Somatic transparent (110db)
Commercial transparent (100db)
Analog transparent (90db)
Cassette transparent (80db)
Vinyl transparent (70db)
Activity transparent (60db)
Phone-speaker-on-public-transportation transparent (50db)

 

[Lambda]

Absolute transparent

But you can’t read Absolute transparent from SINAD. IMD or frequency response might be terribly.
Waht’s the point of having low distortion 1khz if the high frequency is role off?

 

[Keened]

I know that a secondary purpose of the SINAD measurement is to detect the presence of a systematic, measurement-feedback engineering process. Wouldn't the previous four categories cover that, too? If not, what info would do that best?

The issue is multiple overlapping audibility curves for different problems as Rick pointed out. But I think now is a good time to see if we can't get all the different types charted together and see where they overlap. It won't tell us what is but it might give us a better idea of what we should be looking for. There might be pockets of tolerability that are open further down than one might think, but close as you go further alone another variable's curve. So it might not be in the best interest of people to shoot for better absolute products by metrics if it makes the holistic result worse (in the context of what they already own).

EDIT:

But you can’t read Absolute transparent from SINAD. IMD or frequency response might be terribly.
Waht’s the point of having low distortion 1khz if the high frequency is role off?

True, which is why we probably need to push for a new CEA standard where SINAD is exclusively measured using multi-tones. That should capture some more of the common distortion problems into a single number since any IMD will show up (along with failure to accurately reproduce different portions of the spectrum

 

[amirm]

The main purpose of bar graph is to give context for SINAD value. For any single value measurement it is super helpful to know the range of values. US EPA does this for appliance ratings or car gas mileage. The graph is a factual presentation which cannot be argued. Anything else will get arguments of its own.

 

[Chromatischism]

It all depends on listening level

And as Amir has said, the spectrum. You're more biologically tuned to detect distortion at 2 kHz than at 200 Hz.

 

[Omar Cumming]

The histogram is misleading, at a quick glance it implies that the Chord Mojo 2 is the best when it is not. Thus it does not serve the intended purpose.

Cheers

 

[Lambda]

And as Amir has said, the spectrum. You're more biologically tuned to detect distortion at 2 kHz than at 200 Hz.

Sure this as well.
That's why many rate there products in A weighted thd+n
Sadly ASR has no weighted Sinad or noise measurement