• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

SINAD vs. $

OP
pozz

pozz

Слава Україні
Forum Donor
Editor
Joined
May 21, 2019
Messages
4,036
Likes
6,827
@GrimSurfer That would mean just having desktop headphone amps and DACs in one sheet. I'd be removing bareboard devices, AVRs, amps, and other miscellaneous gear.

Top 5 by $:
  1. Wyred4Sound DAC-2v2 SE: $3800, 90dB SINAD
  2. Exasound E32: $3500, 114dB
  3. Matrix Audio Element X: $3000, 120dB
  4. Lynx Hilo: $2300, 110dB
  5. RME ADI-2 Pro: $2000, 117dB
Bottom 5 by $:
  1. SMSL M100: $80, 102dB
  2. Behringer UMC-204HD: $80, 80dB
  3. Massdrop Grace Design SDAC: $75, 97dB
  4. SMAKN X5: $25, 74dB
  5. ELE EL-D02: $16, 78dB
1570646628112.png

1570646637762.png


$ log scaled
1570646826246.png
 
Last edited:

GrimSurfer

Major Contributor
Joined
May 25, 2019
Messages
1,238
Likes
1,484
Interesting, @pozz. The mean and StdDev don't seem to highlight the main feature of the graph, where the mode really stands out @ approximately 105 dB. Perhaps that (mode) is the reportable feature when dealing with dispersed, small sample data?

(I shouldn't have sleep walked through my 3rd year stats course.)

Excluding the possibility that the small sample size is preventing the clear emergence of a trend or tendency, the graphs seem to drive home the point that the industry is completely screwed up. How else would one logically explain the differences in performance for low, medium, and high priced gear? (Rhetorical question, which Don answered in his previous post.)
 
Last edited:

Juhazi

Major Contributor
Joined
Sep 15, 2018
Messages
1,725
Likes
2,910
Location
Finland
Spreadsheet excersices are fun! It is quite easy to make the report (graph) to look like you want, I learned to do that in my previous life some 20 years ago... Using log scale for price separates hits better, but Joe A. Verage perhaps doesn't too easily notice the foolery... Using dB(log) too would be even "better looking"!

But seriously, mixing dacs and amps is not good in my point of view. Did you pozz write them randomly or by name or by groups? I would like to see dacs, AVRs, hpamps and poweramps separately. This is easy task if you edit the table so that all dacs are listed first, then other etc. Then just paint the amp or dac group to make a table (calc). When new measurements appear, just add rows to the main table.
 
OP
pozz

pozz

Слава Україні
Forum Donor
Editor
Joined
May 21, 2019
Messages
4,036
Likes
6,827
OP
pozz

pozz

Слава Україні
Forum Donor
Editor
Joined
May 21, 2019
Messages
4,036
Likes
6,827
Spreadsheet excersices are fun! It is quite easy to make the report (graph) to look like you want, I learned to do that in my previous life some 20 years ago... Using log scale for price separates hits better, but Joe A. Verage perhaps doesn't too easily notice the foolery... Using dB(log) too would be even "better looking"!

But seriously, mixing dacs and amps is not good in my point of view. Did you pozz write them randomly or by name or by groups? I would like to see dacs, AVRs, hpamps and poweramps separately. This is easy task if you edit the table so that all dacs are listed first, then other etc. Then just paint the amp or dac group to make a table (calc). When new measurements appear, just add rows to the main table.
Tracking these values is more work than you may think. Just consider that "DAC" refers to anything with a D/A stage and "headphone amp" to anything with a headphone output. Assigning specific groups has ballooned the sheet to 8k cells at this point.

I don't see an issue with grouping desktop devices together, regardless of function, given that the prices and SINAD values are similar.

As for the plots you wanted:
1570655811177.png

1570655764758.png

1570655750954.png

1570655735553.png
 

GrimSurfer

Major Contributor
Joined
May 25, 2019
Messages
1,238
Likes
1,484
It's interesting that the price-independent mode of AVR/Ps is ~85 dB SNR, ~105 dB for headphone amps, and ~89 dB for power amps.

Now @pozz is working with really small sample sizes here... but there might be a logical explanation for this. Are some manufacturers trying to engineer to a minimum acceptable performance point based exclusively on subjective listening tests? One in which is unlikely to offend an average movie viewer, critical music listener, and casual music listener?

I understand these are very broad categories and very small samples but the clustering deviates from what would otherwise be quite coherent or utterly random. (Though power amp results appear by far the most random of all.)

Engineering to a minimum performance point is, btw, a reasonable strategy to reduce production costs.
 
Last edited:
OP
pozz

pozz

Слава Україні
Forum Donor
Editor
Joined
May 21, 2019
Messages
4,036
Likes
6,827
Well the amps Amir has measured have tended to be budget, Class D and integrated designs. Crown and Benchmark are pro companies which maximize certain performance aspects. The PS Audio S300 gives decent value for money strictly due to the ICEpower module.

So in a sense the entire sample is made of outliers, and the dedicated power amp mainstream is unrepresented.
 

DonH56

Master Contributor
Technical Expert
Forum Donor
Joined
Mar 15, 2016
Messages
7,894
Likes
16,708
Location
Monument, CO
Amir tests what we choose to send him.
 

GrimSurfer

Major Contributor
Joined
May 25, 2019
Messages
1,238
Likes
1,484
Well the amps Amir has measured have tended to be budget, Class D and integrated designs. Crown and Benchmark are pro companies which maximize certain performance aspects. The PS Audio S300 gives decent value for money strictly due to the ICEpower module.

So in a sense the entire sample is made of outliers, and the dedicated power amp mainstream is unrepresented.

I think one firm conclusion that we can make based on the test results is that, despite the bluster about hifi, hires, sota, etc., a great many amp, AVR and DAC manufacturers produce components with noise and distortion that, at least in a theoretical sense and quite possibly in a practical sense, may be audible at certain levels of playback.
 

bogart

Active Member
Forum Donor
Joined
Sep 10, 2019
Messages
251
Likes
312
Some good points being made here. I wonder what some of the other data professionals would say...

For my part, first, total applause for assembling the data to support this! That’s a chore and a thankless one too. The comments are spot on if we are thinking that $ is a possible explanatory variable for SINAD. More likely it’s potentially influential when we control for features and form factor. Probably more work than we want to do here. I suppose if someone wanted to make this a hosted analysis on Tableau we could start playing with more in depth stuff collectively.

...or we can proceed to the conclusion we already arrived at that price is a poor predictor of performance, all else being equal. :)
 

restorer-john

Grand Contributor
Joined
Mar 1, 2018
Messages
12,706
Likes
38,861
Location
Gold Coast, Queensland, Australia
Amir tests what we choose to send him.

True, but how many of us are going to ship 30kg power amplifiers across the Pacific or risk multi thousand dollar esoteric pieces to freight companies?

I'll bet there are plenty of very high performing amplifiers sitting in the lounge rooms of many of the members here. Amplifiers bought in a different time at significant cost and designed by incredibly skilled engineers and manufacturers. Amplifiers that were already very competently reviewed by magazines of the past where measurements were everything.

So ultimately, Amir gets mostly plenty of lightweight little Chinese toys to test because they are, let's face it, disposable and no-one is going to get too upset if they get lost or wrecked in transit.
 

bogart

Active Member
Forum Donor
Joined
Sep 10, 2019
Messages
251
Likes
312
@restorer-john is totally right that there's legacy gear that's going to perform splendidly not tested here. I'd expect that's primarily analog-domain stuff that's especially under-represented both for reasons of weight and gear preciousness.

Stratifying the analysis into digital-only pieces, or those most reliant on integrated circuits is probably a reasonable path. In addition, I'd expect the shifting profile of headphone design and usage has led to more demanding engineering work on HP amps vs. speaker amps in the class A and A/B mold; class D is clearly having its own moment for reasons related to integrated circuits. So, clearly not every piece is on equal footing in our scatter plots.

Thinking as an analyst trying to build a model, including a weight covariate in the analysis is an interesting control for the selection bias of what gets sent. It's unlikely to be a causal variable (except in the case of power supply - plastic cases aren't *necessarily* indicative of worse performing gear, right JDS Labs?).

If we had time and wanted to waste it, I'd love to plot SINAD and other metrics against reviews to prove that the correlation approaches 0 for most publications :)
 

Tks

Major Contributor
Joined
Apr 1, 2019
Messages
3,221
Likes
5,497
IME, SINAD is not a valid/sufficent measure of audio quality. It's a measured quantity, and a very crude and condensed one at that. And retail prize is not mainly influenced by design effort, technical competence and flawless execution. So I would think those scatter plots have no deeper value to them than, well, we can see that SINAD does not correlate much with cost.

Nonetheless, thanks @pozz for compiling.
To improve the plots I suggest to use log scale on the money axis as well.
EDIT: Ah, @Juhazi just had the same idea ;-)

Why would IME, SINAD not be a valid/sufficient measure of audio quality? I mean sure, you have FR to taken into account for, or jitter/IMD and such as well. But I can't think of a time here where I've seen blue-tier SINAD measurements followed by bad metrics otherwise.

Unless I am missing the point of what you're trying to say; being that - "the music itself is a measure of audio quality" or something off field I am completely missing.

Also retail prize, what does that mean? Success in the market?
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,657
Likes
240,879
Location
Seattle Area
So ultimately, Amir gets mostly plenty of lightweight little Chinese toys to test because they are, let's face it, disposable and no-one is going to get too upset if they get lost or wrecked in transit.
I have a number of heavy amplifiers here to test. Including one vintage one. Indeed our entire living room entrance is taken over by massive boxes.

Members are kind enough to pay for shipping to me. And I now have access to a discounted plan to ship them back.

The expenses are much higher but I don't think we have a choice but to go after testing these larger amplifiers and audio products.
 

restorer-john

Grand Contributor
Joined
Mar 1, 2018
Messages
12,706
Likes
38,861
Location
Gold Coast, Queensland, Australia
The expenses are much higher but I don't think we have a choice but to go after testing these larger amplifiers and audio products.

The other issue to consider is ensuring the older/vintage amplifiers are performing to original spec before they get shipped to you. That's a tough one to solve as most people aren't technicians. A channel low in gain or with a noisy component/switch/pot or power supply rail will compromise the test results in an unfair way to the product.

I would encourage anyone sending you an older/vintage piece of gear to at least confirm all functions and features work perfectly and check for any noise/s before packing and shipping a big piece of equipment to you- particularly something they haven't used for a while. Otherwise Amir, you'll be warming up your soldering iron and chasing down minor issues before you can get to the review part.
 

amirm

Founder/Admin
Staff Member
CFO (Chief Fun Officer)
Joined
Feb 13, 2016
Messages
44,657
Likes
240,879
Location
Seattle Area
The other issue to consider is ensuring the older/vintage amplifiers are performing to original spec before they get shipped to you.
I am told the one that was sent is completely refurbished. We will see how it does.
 

Tks

Major Contributor
Joined
Apr 1, 2019
Messages
3,221
Likes
5,497
True, but how many of us are going to ship 30kg power amplifiers across the Pacific or risk multi thousand dollar esoteric pieces to freight companies?

I'll bet there are plenty of very high performing amplifiers sitting in the lounge rooms of many of the members here. Amplifiers bought in a different time at significant cost and designed by incredibly skilled engineers and manufacturers. Amplifiers that were already very competently reviewed by magazines of the past where measurements were everything.

So ultimately, Amir gets mostly plenty of lightweight little Chinese toys to test because they are, let's face it, disposable and no-one is going to get too upset if they get lost or wrecked in transit.

What a preposterous comment. First off, lets' grant your premise about "plenty of light weight little Chinese toys".

1) Weight has almost no correlation to performance with respect to the topic of contention on this thread. The only thing size seems to have some relevance toward, is power. And all the tertiary reasons something that consumers lots of power, requires larger housing (cooling, etc..)

2) If there was any correlation, it would seem power amps, which are bigger in size than desktop amps, do far more poorly, so in actually, your premise - if true, would actually be unfavorable to you two-fold. On the one hand you're getting lighter products that perform better at the same time. But then again I think this is the only point you could reasonably defend, seeing as how desktop amps consume less power, and keeping distortion and other undesirable aspects down in such packages is easier than something capable of 100's of W's. But if we made the comparison 1:1, this point still flies in your face when you bring up the "Chinese toys" part. DX3 Pro still a king in it's own right from last year, and the Matrix DAC the king period. Don't see how those are "toys no one would get too upset if they got lost or wrecked".

3) Why would something that performs, very good, or as good as any one of those behemoths(you allude to as the antithesis to the toys of today) be "disposable"? This makes zero sense, unless you're speaking with varied intent as if to say "yeah these are so light, acquiring another wouldn't be a problem to have shipped to you". But even there it doesn't make sense because anyone that is worried about shipping costs, isn't going to be buying boutique hunks of equipment they need aid lifting.

4) To be frank on a similar level as you... Anytime I see older equipment, based on experience and measurements here, I instantly prepare myself with "how bad's this gonna be?" So this whole "Amplifiers bought in a different time at significant cost and designed by incredibly skilled engineers and manufacturers." is nonsense in the same sense your insinuation of "little Chinese toys" being "disposable" is..

5) Lastly, I don't even know what point you're making addressing DonH56's post. The tonality seemed to be a jab against current-day Chinese products. As if to suggest any sizable group of people that care about fidelity would go out of their way to procure some of these undiscovered older gems only waiting in unknown living rooms to potentially shatter the SINAD metrics gathered here. Made worse by the fact that EVEN IF these products perform as good as their Chinese counter parts, you're still paying an arm and leg. Now I have an RME DAC, and you might ask why I would get that since it's basically a big-boy DX3 Pro. Well it's not because of it's "Made in Germany" designation, but everything that follows that with respect to RME themselves, and the fact it offers more creature comforts than any DAC on the planet to this day.

Outside of simply having a curiousity-driven desire to archive where we were in the past - I couldn't care less about owning some of that gear over any of the devices you consider lightweight toys. Even more importantly, calling such devices disposable? I don't get it.. Why even say that?
 
OP
pozz

pozz

Слава Україні
Forum Donor
Editor
Joined
May 21, 2019
Messages
4,036
Likes
6,827
Again, thanks very much, pozz, for those graphs. I have a question about its data. The bar chart here shows only 4 DACs with a SINAD < 60 dB, with the lowest SINAD being 51 dB. However, in your chart, there are 7 DACs < 60 dB, two of them even under 50 dB.

How come?
I included both desktop DACs and desktop headphone amps in that chart. I excluded all other gear, including portable DACs/amps.

See: https://www.audiosciencereview.com/...easurement-comparison-charts.8246/post-207210
 
Top Bottom