Master Thread: Are measurements Everything or Nothing?

[dped90]

So you are saying that you are too scared to do this because it might prove you wrong.
Here is another challenge, prove that everything that can be heard is measured. do not just say it can.

Prove that there is something that can be heard that cannot be measured.

Jim

Exactly why both numerical and blind listening tests need to be conducted to properly evaluate the audio device.

 

[antcollinet]

Exactly why both numerical and blind listening tests need to be conducted to properly evaluate the audio device.

Blind testing is either impractical or useless for that purpose.

If only conducted by one (or a small quantity of listeners) the results are only valid for them. If conducted with a statistically useful number of participants, it is clearly impractical as a way to evaluate an individual device.

Further it can only be used to compare two or more devices. It is not useful as a way of evaluating a single device.


Finally (and again) before we ask people to invest the time in controlled listening tests someone needs to demonstrate that the measurements are not (against all the evidence) telling us what we need to know.

 

[dped90]

It has been done often enough. So far no one has been able to demonstrate that they can hear a difference between two well measuring DACS.

Really? Visit any audio hardware forum and tell me then that you haven't found numerous DAC users who vigorously claim they can hear substantial differences between DACs. So, as audible differences among DACs are claimed to exist very often among dozens if not hundreds of users, why is it not justified to expect blind listening tests to be a fully legitimate-indeed an essential-part of DAC product evaluation?

 

[Nuni]

Blind testing is either impractical or useless for that purpose.

If only conducted by one (or a small quantity of listeners) the results are only valid for them. If conducted with a statistically useful number of participants, it is clearly impractical as a way to evaluate an individual device.

Further it can only be used to compare two or more devices. It is not useful as a way of evaluating a single device.


Finally (and again) before we ask people to invest the time in controlled listening tests someone needs to demonstrate that the measurements are not (against all the evidence) telling us what we need to know.

So you cannot rise to the challenge. That is alright, just admit it and say so.

 

[antcollinet]

Really? Visit any audio hardware forum and tell me then that you haven't found numerous DAC users who vigorously claim they can hear substantial differences between DACs. So, as audible differences among DACs are claimed to exist very often among dozens if not hundreds of users, why is it not justified to expect blind listening tests to be a fully legitimate-indeed an essential-part of DAC product evaluation?

Show me one who has done it under valid controlled conditions - level matched, blind and statistically relevant.

That is the only reliable form of evidence. Sighted listening claims are not.

 

[Beave]

 

[dped90]

Further it can only be used to compare two or more devices. It is not useful as a way of evaluating a single device.

As the "primary premise" here seems to be that all DACs which measure the same will sound the same, then could not virtually any DAC be chosen by Amir as a "referance" DAC for the DAC under test to be audibly compared against for the blind test?

OTOH, would not the very act of ever launching such comparisons invalidate the "primary premise"?
Therefore, all manner of presumptions and excuses to preclude blind listening tests as part of ASR hardware reviews?

 

[Blumlein 88]

As the "primary premise" here seems to be that all DACs measure the same will sound the same, then would not virtually any DAC could then be chosen by Amir as a "referance" DAC for the DAC under test to be audibly compared against?

OTOH, it would not the very act of ever launching such comparisons invalidate the "primary premise"?
Therefore, all manner of presumptions and excuses to preclude blind listening tests as part of ASR hardware reviews?

In answer to the question in the latter part of your post, no that would not invalidate the premise.

 

[antcollinet]

So you cannot rise to the challenge. That is alright, just admit it and say so.

I feel no need to rise to the challenge as I know what the result would be - further if I did the test and stated "no difference" you could accuse me of pretending not to hear it - or of having tin ears, or my system not being "resolving enough". So it wouldn't convince you in any case.

You on the other hand have the opportunity to be the first to reliably prove that the sound can be heard. And you can't cheat, because only by actually hearing a difference could you pass a properly controlled test. If you can detect a difference under controlled conditions that cannot be measured, you'll be a hero amongst audiophiles. Imagine the kudos you'd have in all the other forums for proving ASR wrong.

You should definitely do it.

But - expect to be challenged on the veracity of your method if you do. Like all innovative and controversial science you'll have to defend it rigorously.

 

[oleg87]

As the "primary premise" here seems to be that all DACs which measure the same will sound the same, then could not virtually any DAC be chosen by Amir as a "referance" DAC for the DAC under test to be audibly compared against for the blind test?

OTOH, would not the very act of ever launching such comparisons invalidate the "primary premise"?
Therefore, all manner of presumptions and excuses to preclude blind listening tests as part of ASR hardware reviews?

Why? What would time-consuming blind listening tests for measurably transparent DACs - having already been "heard" by instrumentation far more sensitive than the human ear - add to Amir's reviews? Do we need to verify the laws of physics every time we weigh something before trusting the number that shows up on the screen?

 

[BDWoody]

Visit any audio hardware forum and tell me then that you haven't found numerous DAC users who vigorously claim...

Too bad there is no follow up with actual evidence. Claims are easy, and pretty much required to be part of that club.

 

[BDWoody]

So you cannot rise to the challenge. That is alright, just admit it and say so.

I sell forks that I claim will make food taste better.

Do you need a taste test to be able to dismiss my claim?

 

[Killingbeans]

Really? Visit any audio hardware forum and tell me then that you haven't found numerous DAC users who vigorously claim they can hear substantial differences between DACs. So, as audible differences among DACs are claimed to exist very often among dozens if not hundreds of users, why is it not justified to expect blind listening tests to be a fully legitimate-indeed an essential-part of DAC product evaluation?

Because people either don't do those tests or don't care about them... or both.

On more than one occasion we've seen respected hi-fi gurus accept the challenge of doing a proper blind test. They are always super confident and expect it to be a piece of cake, and they all fail miserably in the end. No surprise there.

Some of them go straight to denial and start desperately looking for any excuse to make the test invalid. Others have a "I'll be damned...." -moment, but then we fast forward a couple of weeks, and they happily pretend none of it ever happened. They are back to their old ways of sighted listening with mandatory lyrical waxing.

People just don't want to know what's actually going on. Tons of things that make this hobby fun for people are deeply dependent on blissful ignorance.

 

[Palladium]

Because people either don't do those tests or don't care about them... or both.

On more than one occasion we've seen respected hi-fi gurus accept the challenge of doing a proper blind test. They are always super confident and expect it to be a piece of cake, and they all fail miserably in the end. No surprise there.

Some of them go straight to denial and start desperately looking for any excuse to make the test invalid. Others have a "I'll be damned...." -moment, but then we fast forward a couple of weeks, and they happily pretend none of it ever happened. They are back to their old ways of sighted listening with mandatory lyrical waxing.

People just don't want to know what's actually going on. Tons of things that make this hobby fun for people are deeply dependent on blissful ignorance.

Golden Ears squaring the "I can hear every difference no matter how tiny" with the "but you fooled me by saying the LP I supposedly listened was actually a CD" circle.

 

[Killingbeans]

Exactly.

If I were to do one of those tests on a guru/golden ear, I'd allow them to do their normal sighted comparison with the switching gear in place first, and make careful notes of how they still point out monumental differences.

That way they can't weasel their way out of the results of the actual blind test, by claiming that the switching gear masked the difference.

Although, they'd probably just do the old "Blind tests are stressful, and I can't hear anything when I'm stressed!"

 

[krabapple]

So you are saying that you are too scared to do this because it might prove you wrong.

It's laughable that you would think that's what I'm saying.

Here is another challenge, prove that everything that can be heard is measured. do not just say it can.

So you are saying you're clueless about science?

Anyway, dped90, whom I replied to, proposed a flawed but at least on the right track blind test of DACs. However , such blind tests as he proposed requires getting the multiple participants together, obtaining the venue, the DACs, the switching setup, the playback setup, and an investment of time...that all costs money. But he wants Amir to do it gratis, apparently.

 

[krabapple]

Really? Visit any audio hardware forum and tell me then that you haven't found numerous DAC users who vigorously claim they can hear substantial differences between DACs. So, as audible differences among DACs are claimed to exist very often among dozens if not hundreds of users, why is it not justified to expect blind listening tests to be a fully legitimate-indeed an essential-part of DAC product evaluation?

That's not a demonstration 'that they can hear a difference between two well measuring DACS' .
That is a *claim* that they can hear such difference.
No one ever said THOSE don't exist.

How can you not understand the difference at this point?

 

[krabapple]

So you cannot rise to the challenge. That is alright, just admit it and say so.

As the science is firmly on the side that says you can't hear a difference between DACs that measure identically (enough) in the audible range, the onus is on *you* to prove you can.

I suggest you contact Amir and offer to to pay him to proctor a proper blind test of yourself on this point.

C'mon, step up. If you succeed you'll be internet-famous.

 

[Lux_ury]

I sell forks that I claim will make food taste better.

Do you need a taste test to be able to dismiss my claim?

I use sporks just in case best of both worlds. Reality some folks buy big so they don't want to know more. I think anyone who has more than two DaCs has a problem.
The only choice is AK or ES why buy the same DAC with the Same chip manufacturer (rubs chin emoji)

 

[Killingbeans]

The only choice is AK or ES

And TI, and Analog Devices and Cirrus Logic... just to name a few of the big ones.

I don't really see a problem in owning multiple DACs. Sometimes you need more than one to have different setups covered. Although, in many cases the "hidden" built in DACs in various source devices would be perfectly adequate.

It's the practice of having multiple DACs at hand, just to switch between them and have a "taste", that I find really bizarre. But to each their own.