Master Thread: Are measurements Everything or Nothing?

[voodooless]

I contacted Archimago and he added some D90SE tests.

REVIEW & MEASUREMENTS: Cayin RU6 R-2R USB Dongle DAC/Headphone Amp. Considering "Intentional Imperfections". (Topping D90SE -60dBFS input graphs.)

A blog for audiophiles about more objective topics. Measurements of audio gear. Reasonable, realistic, no snakeoil assessment of sound, and equipment.

archimago.blogspot.com

So nothing was found, why does that not surprise me?

 

[AW61]

I realise this may be controversial but here goes..

I’ve been a HiFi man all my teenage and now adult life – I’m now 61.

Like the rest of the world, where I never thought to buy a standalone DAC, in this streaming age, I now need to.
And here’s where I have encountered the whole “measurements vs sound” debate, with fiercely entrenched views

So, because it interests me, here is a perspective.

There is a lot of guff about the difference in sound of one cable vs the other.
Even specialist power supplies – I remember doing AB comparisons with a Cyrus Amp with/without the smoothed power supply – there was no audible difference (that I could detect anyway)

However, I cannot agree that if two devices measure the same, then they will necessarily sound the same (in this case DACs)

Why?

I come from an engineering background in silicon chip manufacturing. Here you measure everything to death, and then some more. All kinds of real and derived parameters, correlations between one and another etc. Hundreds of measurements on stock components to authenticate each wafer is what is wanted.

But – Your vision is only as good as the parameters you measure. You absolutely cannot fully describe the behaviour of an electronic device by measurements alone.

Sometimes, despite the standard components on a wafer measuring exactly as required, the product chip fails to work as expected – because measurements only give you a veiled picture of things.

The point is, for all the measurements you can think of doing, (and those done on this site look pretty comprehensive) there are always some characteristics that you have never thought of measuring, or that you simply cannot get at to measure.
Or it can be a eg a correlation between parameters at particular voltages that causes two seemingly identically measuring products to end up functioning slightly differently.

So a set of measurements (on a particular DAC) that looks the same as those of a different DAC absolutely cannot be a guarantee that the sound will be the same.

So I love the measurements done here, but they are not the full story - would be great if they were supplemented by blind testing AB sound checks.

 

[storing]

So a set of measurements (on a particular DAC) that looks the same as those of a different DAC absolutely cannot be a guarantee that the sound will be the same.

And likewise, measurements which do show the DACs are in fact different - which happens all the time - are no guarantee the sound as perceived by the human brain will be different. So theoretically yes one cannot measure everything. But whether that is needed in practice is something very different. Blind tests are indeed the perfect tool to prove that the measurements are not sufficient, but it's not like there's much evidence pointing in that direction, on the contrary.

 

[Frgirard]

I realise this may be controversial but here goes..

I’ve been a HiFi man all my teenage and now adult life – I’m now 61.

Like the rest of the world, where I never thought to buy a standalone DAC, in this streaming age, I now need to.
And here’s where I have encountered the whole “measurements vs sound” debate, with fiercely entrenched views

So, because it interests me, here is a perspective.

There is a lot of guff about the difference in sound of one cable vs the other.
Even specialist power supplies – I remember doing AB comparisons with a Cyrus Amp with/without the smoothed power supply – there was no audible difference (that I could detect anyway)

However, I cannot agree that if two devices measure the same, then they will necessarily sound the same (in this case DACs)

Why?

I come from an engineering background in silicon chip manufacturing. Here you measure everything to death, and then some more. All kinds of real and derived parameters, correlations between one and another etc. Hundreds of measurements on stock components to authenticate each wafer is what is wanted.

But – Your vision is only as good as the parameters you measure. You absolutely cannot fully describe the behaviour of an electronic device by measurements alone.

Sometimes, despite the standard components on a wafer measuring exactly as required, the product chip fails to work as expected – because measurements only give you a veiled picture of things.

The point is, for all the measurements you can think of doing, (and those done on this site look pretty comprehensive) there are always some characteristics that you have never thought of measuring, or that you simply cannot get at to measure.
Or it can be a eg a correlation between parameters at particular voltages that causes two seemingly identically measuring products to end up functioning slightly differently.

So a set of measurements (on a particular DAC) that looks the same as those of a different DAC absolutely cannot be a guarantee that the sound will be the same.

So I love the measurements done here, but they are not the full story - would be great if they were supplemented by blind testing AB sound checks.

No one has succeeded an abx with dacs.
All dacs sound the same even they doesn't have the same measurements.
Nb DAC without a big tube in output or an eq.

 

[Pdxwayne]

No one has succeeded an abx with dacs.
All dacs sound the same even they doesn't have the same measurements.
Nb DAC without a big tube in output or an eq.

Really????

Just one example:

DAC ABX Test Phase 1: Does a SOTA DAC sound the same as a budget DAC if proper controls are put in place? Spoiler: Probably yes. :)

I was finally able to do the test, I scored a 10 correct out of 16. About as good as I would have expected: some dice could have done better as @solderdude showed :mad: That was really hard! I thought one was a bit brighter and there was more "black between the notes" (pardon my audiophile...

www.audiosciencereview.com

 

[Steve57]

Blind tests are indeed the perfect tool to prove that the measurements are not sufficient, but it's not like there's much evidence pointing in that direction, on the contrary.

I'm sure blind tests could prove alot but n my experence there's been people who I would not trust to give an accurate assessment of any piece of kit..

 

[rdenney]

I have several DACs:

- Musical Fidelity V.90. Recommended by Sam Tellig with all the associated adjectives. Measures “okay”.

- Topping E30. Measures superbly. Recommended by Amir.

- ifi Zen DAC. I bought it for its Bluetooth capability, its weirdness, its looks, and its price. It was well reviewed subjectively and it measures adequately.

- various DACs in CD players—at least half a dozen that also have coaxial digital outputs so that a comparison can be made.

I can’t tell the difference in any of them.

So, whatever differences there are exceed audibility thresholds for me. Are my thresholds low? Possibly.

I’ve concluded that I’m free to buy whatever DAC has the necessary features and appearance.

Rick “hearing all this through speakers that add 1000 times more distortion” Denney

 

[Purité Audio]

Almost every reviewer and every dealer?
Keith

 

[Pdxwayne]

I have several DACs:

- Musical Fidelity V.90. Recommended by Sam Tellig with all the associated adjectives. Measures “okay”.

- Topping E30. Measures superbly. Recommended by Amir.

- ifi Zen DAC. I bought it for its Bluetooth capability, its weirdness, its looks, and its price. It was well reviewed subjectively and it measures adequately.

- various DACs in CD players—at least half a dozen that also have coaxial digital outputs so that a comparison can be made.

I can’t tell the difference in any of them.

So, whatever differences there are exceed audibility thresholds for me. Are my thresholds low? Possibly.

I’ve concluded that I’m free to buy whatever DAC has the necessary features and appearance.

Rick “hearing all this through speakers that add 1000 times more distortion” Denney

Yet, amazingly, we still have Rick "who still can perform distortion tests, got -33db, and beat out almost half of the test participants" Denney.

 

[AW61]

All dacs sound the same even they doesn't have the same measurements.

Here's where I absolutely disagree - from firsthand experience.

The DAC I'm using atm is that on my Musical Fidelity KW250s - taking the optical output from Chromecast Audio.
Then out from that to my active speakers, Beolab 17, via RCA.
That unit has to go hence my need for a new DAC.

The Beolab 17 themselves have optical inputs ie they have on board DACs so I thought " no need for a new DAC, just use their DACs"
I was not prepared for the degradation in sound vs the KW250S.
Where with the KW250S there was such separation, I felt I could see into all the instruments, now it was more like a wall of sound. I was so surprised.
I actually didn't believe my ears so I swapped back and that transparency returned.

So no, not all DACs sound the same - and here I wasn't looking for a difference (so found one) - I wasn't expecting a difference and it hit me in the face.

 

[Purité Audio]

Yes… you really have to ask someone else to make that swop for you, unless the MF is so very poor it is adding audible distortion.
Keith

 

[rdenney]

Yet, amazingly, we still have Rick "who still can perform distortion tests, got -33db, and beat out almost half of the test participants" Denney.

Like I said, my threshold might not be yours. But it also means yours is not mine.

Rick “you do you” Denney

 

[Pdxwayne]

Like I said, my threshold might not be yours. But it also means yours is not mine.

Rick “you do you” Denney

Sure, Rick "didn't know he got praised" Denney.

: P

 

[Sgt. Ear Ache]

Really????

Just one example:

DAC ABX Test Phase 1: Does a SOTA DAC sound the same as a budget DAC if proper controls are put in place? Spoiler: Probably yes. :)

I was finally able to do the test, I scored a 10 correct out of 16. About as good as I would have expected: some dice could have done better as @solderdude showed :mad: That was really hard! I thought one was a bit brighter and there was more "black between the notes" (pardon my audiophile...

www.audiosciencereview.com

Lol...10 of 16. 2 more than 50%. Do a hundred tests...you'll be even closer to 50%. And of course that ignores the fact that your initial claims were that you were hearing more "details" on the khadas that were missing on the E30...which upon doing a few tests you soon realised was an illusion and then you started a long and bizarre quest to prove you could hear a difference of some kind.

 

[Pdxwayne]

Let's see:

Member A: nobody can hear difference between dac, even if the DAC measured differently.

Member B: Memeber A, here is a link to a thread where such tests were performed and multiple members said they can hear a difference and have abx results to support their claims.

Member C: I really doubt member B's ability to hear a difference.

So, what kind of behavior member C exhibited? Ignore the facts and go on personal attacks instead?

Hmmm, let's see, member C loves to use this word: strawman. Is strawman the correct word to describe member C's behavior?

Sad indeed.

 

[Sgt. Ear Ache]

Let's see:

Member A: nobody can hear difference between dac, even if the DAC measured differently.

Member B: Memeber A, here is a link to a thread where such tests were performed and multiple members said they can hear a difference and have abx results to support their claims.

Member C: I really doubt member B's ability to hear a difference.

So, what kind of behavior member C exhibited? Ignore the facts and go on personal attacks instead?

Hmmm, let's see, member C loves to use this word: strawman. Is strawman the correct word to describe member C's behavior?

Sad indeed.

A quick perusal of that thread seems to suggest that what was determined from the testing done there was that when 2 dacs that actually do have some measurable differences are compared head to head, almost nobody can really tell a difference. And even the few results that might look like they indicate a difference was heard, the number of tests is small enough that they don't really prove anything. I'd also wonder if there's any possible way the 100% correct results could have been gamed in some way. In any event, the results certainly seem to suggest that anyone claiming to hear differences in "imaging and soundstage and clarity and musicality" in any sort of classic audiophile listening situation (you know, in their listening space with their new toy connected and the old toy in the closet collecting dust and their comparison based on what they remember of it's qualities from a week or 2 ago) is probably totally imagining it...

Member B likes to pretend there's proof where there really isn't any.

 

[Sgt. Ear Ache]

Here's where I absolutely disagree - from firsthand experience.

The DAC I'm using atm is that on my Musical Fidelity KW250s - taking the optical output from Chromecast Audio.
Then out from that to my active speakers, Beolab 17, via RCA.
That unit has to go hence my need for a new DAC.

The Beolab 17 themselves have optical inputs ie they have on board DACs so I thought " no need for a new DAC, just use their DACs"
I was not prepared for the degradation in sound vs the KW250S.
Where with the KW250S there was such separation, I felt I could see into all the instruments, now it was more like a wall of sound. I was so surprised.
I actually didn't believe my ears so I swapped back and that transparency returned.

So no, not all DACs sound the same - and here I wasn't looking for a difference (so found one) - I wasn't expecting a difference and it hit me in the face.

The nature of cognitive bias is such that it simply doesn't care what you think you are expecting to hear. "I wasn't expecting a difference so I was shocked to hear such an obvious one" is right up there with "my wife heard the difference from the kitchen" in the pantheon of classic audiophile exclamations...

That's not to say all dacs sound the same though. It's certainly not impossible that dacs might sound different...but if they do it's likely one or both are pretty crummy dacs and easily measurably so. So yes I agree, the statement that "all dacs sound the same regardless of measurements" isn't necessarily true.

 

[AdamG]

A quick perusal of that thread seems to suggest that what was determined from the testing done there was that when 2 dacs that actually do have some measurable differences are compared head to head, almost nobody can really tell a difference. And even the few results that might look like they indicate a difference was heard, the number of tests is small enough that they don't really prove anything. I'd also wonder if there's any possible way the 100% correct results could have been gamed in some way. In any event, the results certainly seem to suggest that anyone claiming to hear differences in "imaging and soundstage and clarity and musicality" in any sort of classic audiophile listening situation (you know, in their listening space with their new toy connected and the old toy in the closet collecting dust and their comparison based on what they remember of it's qualities from a week or 2 ago) is probably totally imagining it...

I imagine you are correct Sir!

 

[Pdxwayne]

I imagine you are correct Sir!

Member B was responding to this statement from member A:
....All dacs sound the same even they doesn't have the same measurements....

So, it would seems you agree with member B that if DAC measured different, there is possibility of certain members that can sense a difference.

 

[Sgt. Ear Ache]

Member B was responding to this statement from member A:
....All dacs sound the same even they doesn't have the same measurements....

So, it would seems you agree with member B that if DAC measured different, there is possibility of certain members that can sense a difference.

Of course if 2 things measure differently they could sound audibly different. lol. If one thing is sitting at 90db spl and the other is 40db spl, they will sound easily distinct from one another. If one thing has a 5db bulge between 4 and 12khz and the other doesn't, they will sound different. Most dacs don't have very substantial differences at all, and only a very bad one would be audibly different from a good one. But whatever, you win! lol...yes, 2 things that measure differently from one another might sound audibly different.