Master Thread: Are Measurements Everything or Nothing?

[escksu]

Like what many has pointed out, what measures extremely good and what sounds good to ears need not always correlate. Although the measurements for the C658 may not be that great, its by no means crap. Its still pretty decent. I have also been saying that up to a certain point, you get dimishing returns. Again, using THD example. 0.1% vs 0.001%, i can assure you that you dont notice that 100x difference. In fact you are unlikely to even notice any difference.

So, it depends alot on what you want. Do you want your sound to be as similar to the source as possible or do you prefer something that sounds pleasant to your ears?

Folks here prefer something that is as close to the recording as possible, i once pursue that but realise that i just dont like it. Its just personal preference.

 

[solderdude]

To me, it's the main gut check. Reading the rest of what you wrote and what Amir just wrote, nothing changes my opinion on that. It's the canary in the coal mine of it being an excellent measuring product - if there's a short fall elsewhere, it's usually relatively minor.

It shouldn't be the main gut check because it isn't.
A device could have great SINAD but poor FR under specific loads (class D power amps for instance).
A DAC could have good SINAD when only measured on a 'proper' filter yet perform poorly when it could also be set to NOS mode for instance.

Good thing to remember: SINAD only says something about the Signal to unwanted signal ratio (in dB) for a 1kHz signal at a specific output level into a specific load.
It isn't and never will be the most important aspect that tells you all you need to know. It does say something but is NOT the aspect to look at.

 

[brandall10]

It shouldn't be the main gut check because it isn't.
A device could have great SINAD but poor FR under specific loads (class D power amps for instance).
A DAC could have good SINAD when only measured on a 'proper' filter yet perform poorly when it could also be set to NOS mode for instance.

Good thing to remember: SINAD only says something about the Signal to crap distance for a 1kHz signal at a specific output level into a specific load.
It isn't and never will be the most important aspect that tells you all you need to know.

But those are easily knowable conditions based on usage scenarios. For the DAC example, it just has different SINADs conditionally - we're still talking about SINAD in the end.

I mean, to say it has poor SINAD because it's in NOS mode means it performs poorly because it has poor SINAD... sorta validating what I'm saying here.

 

[antcollinet]

Like what many has pointed out, what measures extremely good and what sounds good to ears need not always correlate. Although the measurements for the C658 may not be that great, its by no means crap. Its still pretty decent. I have also been saying that up to a certain point, you get dimishing returns. Again, using THD example. 0.1% vs 0.001%, i can assure you that you dont notice that 100x difference. In fact you are unlikely to even notice any difference.

So, it depends alot on what you want. Do you want your sound to be as similar to the source as possible or do you prefer something that sounds pleasant to your ears?

Folks here prefer something that is as close to the recording as possible, i once pursue that but realise that i just dont like it. Its just personal preference.

Even if you prefer a sound with added distortion - you need to START from a clean sound. If you have a device that creates distortion that is not to your liking, you are screwed. You can never get rid of it.

If on the other hand you have a device that is capable of providing true to source sound, you can add whatever distortion you like. Add any effect you wish in your PC sound output - or an effects processor just before your amp. : EQ it to hell, add reverb, do whatever you like.

Otherwise you are just playing trial and error with expensive kit until you find the sound you like.

 

[solderdude]

we're still talking about SINAD in the end

That's why it isn't the main gut check. A SINAD of 80 may be just as good as 120 in normal usage scenarios (listening to music).
A substantial deviation in FR is much more audible than 80 SINAD opposite 120 SINAD.

All measurements including harmonics spread, load differences, 0dBFS vs -3dBFS, overload conditions, frequency response etc. are important.
SINAD says nothing about those aspects. Don't think high SINAD is the best.
High SINAD could be an indicator of good technical performance but isn't a guarantee.

 

[escksu]

Talking about cars. Here is an analogy.

Some sports cars are designed to give the driver accurate and realistic condition of the road. Every bump can be felt, even down to steering. Thats why hydraulic steering is still used for these cars. Even sound proofing is kept to a minimum to reduce weight. They also tend to use harder materials like polyurethane for their bushings instead of rubber (can be upgraded to alu ones). However, they are also extremely uncomfortable. NVH is extremely poor. Even sports tyres are noisy on the road.

Those saloons designed for comfort are the exact opposite. Comfort is the key. You dont feel like you are on the road at all (hardly any feedback). Steering is also effortless but again lack of feedback about road condition.

So, accurate vs comfort.

 

[amirm]

So, accurate vs comfort.

For the most part though, we have not had to make that compromise here. We have DACs at $100 with incredible fidelity and just as usable as a $5,000 DAC. It is a bit harder to reach the top with amplifiers but we are getting better at that as well.

The problem with audio is that the company takes out all the sound proofing and still gives you crappy handling while charging more!

 

[brandall10]

That's why it isn't the main gut check. A SINAD of 80 may be just as good as 120 in normal usage scenarios. A substantial deviation in FR is much more audible than 80 SINAD opposite 120 SINAD.

How would 80 be just as good as 120? Are you saying it can be inaudible in a blind test? I'm just trying to understand what argument you're making.

Again, an amp is going to have a usage profile based on load, I get that, but I see that as totally separate. It's a tool that is designed to perform at a operation profile, because well, it amplifies stuff.

 

[escksu]

Even if you prefer a sound with added distortion - you need to START from a clean sound. If you have a device that creates distortion that is not to your liking, you are screwed. You can never get rid of it.

If on the other hand you have a device that is capable of providing true to source sound, you can add whatever distortion you like. Add any effect you wish in your PC sound output - or an effects processor just before your amp. : EQ it to hell, add reverb, do whatever you like.

Otherwise you are just playing trial and error with expensive kit until you find the sound you like.

Yes, i fully agree on this.

 

[escksu]

For the most part though, we have not had to make that compromise here. We have DACs at $100 with incredible fidelity and just as usable as a $5,000 DAC. It is a bit harder to reach the top with amplifiers but we are getting better at that as well.

The problem with audio is that the company takes out all the sound proofing and still gives you crappy handling while charging more!

Haha, yes. I fully agree on this!! Thats why i can now understand why this forum is so good and so important (sorry , it took me quite a while). Thanks for the hard work!! Cheers!!

 

[solderdude]

A SINAD of 80 that is distortion dominated but only 2nd harmonics can not be distinguished from a SINAD 120 DAC when listening to music.
A SINAD 80 DAC that is noise or hum dominated and used with digital volume control can well be distinguished from an 80 or 120 SINAD DAC that is distortion dominated.

So 80 SINAD could be enough and could not be. 120 SINAD will always be enough but 80 could also be.
That is if FR and other aspects are otherwise the same.

Don't stare blind on SINAD it only says something and is no insurance for best sound but is assurance for technical performance at 1kHz into a specific load at a specific level... nothing more.

 

[brandall10]

So 80 SINAD could be enough and could not be.

Fair enough - so the big question is - do we have actual examples? Not just in theory, are there devices out there with a SINAD of 80 that perform well in a battery of tests?

Again, going back on the canary in the coal mine hypothesis - if something is measuring in the top of the range, the manufacturer has done their homework. They know this is how ranking is done and will actually be instrumenting and tuning the perfomance.

 

[solderdude]

A DAC of 80 is almost universally going to be a hot mess.

Nope.

https://www.audiosciencereview.com/...ments-of-breeze-audio-es9018-dac-hp-amp.8818/ for instance. It has a SINAD below 80 but improves to when the volume is turned down a bit and otherwise performs well.
Multitone test wasn't done though as well as some other tests but you may get the idea. Only THD dominated at a specific level at a specific load but otherwise isn't a hot mess.

Still I won't buy one because you can get better performance from cheaper gear but based on the measurements I am quite certain you can't tell it apart with music when used as a DAC (the headphone section is under powered but not a mess.

 

[brandall10]

^ but it's a terrible product on the whole. The conclusion could hardly be worse.

Again, going back on the canary in the coal mine hypothesis - if something is measuring in the top of the range, the manufacturer has done their homework. They know this is how ranking is done and will actually be instrumenting and tuning the performance.

 

[escksu]

Fair enough - so the big question is - do we have actual examples? Not just in theory, are there devices out there with a SINAD of 80 that perform well in a battery of tests?

Again, going back on the canary in the coal mine hypothesis - if something is measuring in the top of the range, the manufacturer has done their homework. They know this is how ranking is done and will actually be instrumenting and tuning the perfomance.

I would say plenty. Look at those subjective listening tests reviews and compare them with measurement results. I can assure you that you won't find a good sounding gear that measures terribly and vice versa. A good sounding gear will have generally decent measurements. Doesn't have to be excellent but still good. Thats why I said NAD DAC SINAD is considered pretty ok.

We also need to take note about individual preferences.

 

[solderdude]

They know this is how ranking is done and will actually be instrumenting and tuning the performance.

There is no 'tuning' performance. There is correct engineering in the light of signal fidelity. For this reason Amir shows the SINAD as it points towards high signal fidelity.
However, above certain limits (lets call this the left 2 tiers 'excellent' and 'good') it is all ONLY a number game.
A DAC with SINAD 121 is technically superior to one with SINAD 100. This is very measurable.
It could be better in S/N ratio, it could be better in distortion for 1kHz at a specific level, it could be better in both.
In practice it is moot.

It is pointless to buy a DAC that is 120 SINAD over one that is 115 SINAD when the 115 SINAD DAC has better features, better filtering etc.
So in that sense SINAD is NOT an indication of best sound or best engineering when we are talking about usage in audio reproduction.
It is only interesting to see how technically correct waveform reproduction is at 1kHz on a test bench.
Not a 'buying guide' or insurance of high quality products.

We have seen DAC's and amps that measure well on standard tests but still have issues that fall through the cracks on other tests despite good SINAD.

 

[Lupin]

It shouldn't be the main gut check because it isn't.
A device could have great SINAD but poor FR under specific loads (class D power amps for instance).
A DAC could have good SINAD when only measured on a 'proper' filter yet perform poorly when it could also be set to NOS mode for instance.

Good thing to remember: SINAD only says something about the Signal to unwanted signal ratio (in dB) for a 1kHz signal at a specific output level into a specific load.
It isn't and never will be the most important aspect that tells you all you need to know. It does say something but is NOT the aspect to look at.

Technically very true but..
People will put the most value in SINAD number as long as Amir will use the SINAD number to rank different DACS/AMPs.

 

[JSmith]

People will put the most value in SINAD number as long as Amir will use the SINAD number to rank different DACS/AMPs.

Not if they read and understand the reviews properly.

SINAD is a fair metric to use for ranking in general, then one looks further at the results and makes an informed decision from there.

If people don't want to educate themselves and rely on the SINAD ranking only, well that's their business. That said, a quick pick from the top of a SINAD product ranking will likely ensure they buy a decent product anyway, measurements wise.

I think this is the 3rd thread in a few weeks about SINAD... would be great if people search first before starting a new thread too.



JSmith

 

[torgeirs]

SINAD could also be used to decide how much you have to spend on a DAC or an amp.
Use REW white noise generator.
Adjust volume on the high side of normal listening volume. (Nobody cares about noise during a party;-))
Set white noise at -60 dB FS
Adjust until you can just hear it. Thats your needed SINAD. Maybe 6 dB lower?
Use the SINAD stats here to see what gear is not usable in your system. The rest should be okay.

That is way SINAD was defined. Engineers needed å number that defined how good a design needed to be regarding noise. (Telecom, space, measurement ++)
So first find out what the SINAD need to be, then design/buy the equipment

Personal experience is that gear driving horns or 15"+ pro drivers directly needs better than 80 dB SINAD.
Worked at our national broadcaster 30 years ago. The norm there was -80 dB THD+N for each component. Because with the chain of equipment in the studio they wantet to get under 60 dB for the whole chain. (At that time they really spent a lot on money on equipment)

 

[Lupin]

Not if they read and understand the reviews properly.

That is a very big if.
Just look around the forums and you'll quickly notice by the messages from people that very often it doesn't work that way.
For a whole lot of people it is as black and white as high number equals good and low(er) number equals trash.

This is the "danger" of ASR reviews.
People don't understand or misinterpret the numbers but are still quick to draw (wrong/false) conclusions and are more then willing spread those online.
You know how it goes online, everything becomes automatically true if one shouts it loud enough and long enough. This is how many misconceptions are started and brought into the world.