Serious Question: How can DAC's have a SOUND SIGNATURE if they measure as transparent? Are that many confused?

[MaxBuck]

Lots of recent verbiage here that can be summarized: if you want to believe, you'll believe.

 

[ahofer]

I agree with the idea that if you want to test people's beliefs, make a bet. But as you can see, a properly controlled test is non-trivial and easily subject to gaming. I think it is worth establishing a good protocol for each type of equipment. I wish I had more equipment lying around and experience with the pitfalls. Our beliefs around here are generally "all else equal", yet it seldom is.

 

[SIY]

And what is that?

$200/hr plus travel expenses. I'd also budget for a magician, but they come cheap.

 

[krabapple]

I have some long unexplained memories of certain cables sounding, well, wrong and more recent experiences of a certain make of amps which simply don't present a 'hear through' presentation (I got my a*se kicked politely when I mentioned it before), but time moves on so I just have to 'shut my noise' on this. I've fooled myself on other things a good few times over the years but yeah, I like to discuss it sometimes, much to the despair of my subjectivist friends and a few fellow objectivist people here

Are they really 'long unexplained' or were they simply sighted, nonlevelmatched comparisons?

If the latter, they aren't 'unexplained'.

 

[krabapple]

Maybe a recap is in order. What exactly is the claimant claiming here? That he can routinely tell two DACs apart 'just by listening'? Or is it generally a hard/sporadic thing for him to achieve? Has he done it with specific DACs he can name? Details please.

If there are two specific DACs in mind, measure them before bothering with a DBT. Send them to Amir, for instance. Is there a difference that can be eliminated by output level matching?

If yes. then it moves on to a DBT stage, and if $50K is on the line, you for sure want a competent proctor there.

 

[ahofer]

Maybe a recap is in order. What exactly is the claimant claiming here? That he can routinely tell two DACs apart 'just by listening'? Or is it generally a hard/sporadic thing for him to achieve? Has he done it with specific DACs he can name? Details please.

If there are two specific DACs in mind, measure them before bothering with a DBT. Send them to Amir, for instance. Is there a difference that can be eliminated by output level matching?

If yes. then it moves on to a DBT stage, and if $50K is on the line, you for sure want a competent proctor there.

More specific - CHORD Dave with the upsampler/'scalar' thingy versus a cheap but transparent DAC to be determined (I suggest a $79 SMSL above). Claims to have done it successfully at home with some dongle or other and IEMs. Knowing the CHORD specs from Amir's test I suspect level-matching as a potential confounder, but I'm trying to get to a broadly acceptable trial format (as suggested above) and I'm willing to make the investment, to a point.

 

[DSJR]

Are they really 'long unexplained' or were they simply sighted, nonlevelmatched comparisons?

If the latter, they aren't 'unexplained'.

They're unexplained because nobody took the products and analysed them properly to find out why they did what they did, because almost nobody did that in the UK subjectivist 80's and 90's.

 

[krabapple]

You don't even know if they really did what they did. As such your experience can be readily explained as the outcome of sighted comparison.

 

[ahofer]

And what is that? You may recall my original proposed terms included reimbursement for a technical expert.

I've been thinking this through and looking at cheap equipment to do it properly. I'm not interested in interpersonal beefs, but I am interested in a test that would be somewhat resilient to the usual objections (switching time and control, level matching, simultaneous program material, etc.) Right now I'm thinking as follows:

Visible side of barrier:
-Computer source (audirvana, Roon, whatever)
-USB switch
-Headphone preamp (Topping L50 is what I have as a standalone)


Other side of barrier
-two USB connectors go to two separate DAC (chains, with Chord Upsampler inserting jitter as shown, a variable I'm a little concerned about). I'm thinking the alternate DAC might be the SMSL SU-1. $80, well above threshold AFAICT.
-DACs feed a passive mixer, where levels are matched
-single output back under the barrier to the headphone amp.

Listener can switch back and forth with the USB switch (I have used one of these in one of my desktop setups. It takes a second, but pretty fast, and I believe the unused channel is pretty silent). On the other side of the curtain someone is switching the setup periodically. Theoretically we could hide the dacs from them to make it double blind.

Still no "X" preference alternative, but it seems like a decent start for AB. I haven't purchased the extra stuff yet, but interested in your thoughts.

Most good DACs should be shielded from the M-Scalar's insertions, but it does make one wonder.

In the alternative, you could do a simultaneous optical feed to the DACs connected to an (attenuator and a) RCA switch on the visible side.

Realizing this doesn't quite work. If we switch the cables to the mixer (as we should), we'd have to adjust the mixer to swap the volume adjustment. Takes too much time.

Probably better to go with simultaneous optical streams (via SPDIF conversion and splitter) into the two DACs and add an RCA switch before the headphone amp (my headphone amp has only one RCA input).

 

[krabapple]

Things like instant switching , short test samples, listener training, etc are really there to maximize the sensitivity of the test. That's entirely appropriate when using naive subjects in a pure research setting testing a general hypothesis (e.g. is there a population that can hear a difference between A and B)

But this is not that. This is testing one person's confident claim that they can tell A and B apart under conditions C. The conditions they use are already 'sensitive enough' for them.

When a subject already insists that they can tell A and B apart under conditions C (typically: sighted, not level matched, 'slow' switching, listening for various amounts of time to various lengths of music), the first hurdle would be simply for them to demonstrate this performance under those same conditions except one: double blinded instead of sighted.

I'd expect this to weed out the vast majority of claims right away.

If the claimant fails* but then says 'oh but the test wasn't sensitive enough', they are arguing in bad faith.

If they succeed, move on to output measurement, and repeat the DBT with levels matched.



*and again, given the confidence of the claimant in their unlikely claim, a success should be high-confidence, p significantly less than 0.05, not marginal

 

[ahofer]

Things like instant switching , short test samples, listener training, etc are really there to maximize the sensitivity of the test. That's entirely appropriate when using naive subjects in a pure research setting testing a general hypothesis (e.g. is there a population that can hear a difference between A and B)

But this is not that. This is testing one person's confident claim that they can tell A and B apart under conditions C. The conditions they use are already 'sensitive enough' for them.

When a subject already insists that they can tell A and B apart under conditions C (typically: sighted, not level matched, 'slow' switching, listening for various amounts of time to various lengths of music), the first hurdle would be simply for them to demonstrate this performance under those same conditions except one: double blinded instead of sighted.

I'd expect this to weed out the vast majority of claims right away.

If the claimant fails* but then says 'oh but the test wasn't sensitive enough', they are arguing in bad faith.

If they succeed, move on to output measurement, and repeat the DBT with levels matched.



*and again, given the confidence of the claimant in their unlikely claim, a success should be high-confidence, p significantly less than 0.05, not marginal

I agree that if my purpose were to win, I would design a test to exploit those disadvantages that he left. But, as I said, I'm not in this for a personal beef, I'm in it for more generalizable results. If we did allow slow switching, that would become the excuse for any test failure, as you suggest.

I'm not sure, though, that allowing a level mismatch to persist is a good idea. Most anyone would pass that test.

 

[krabapple]

I agree that if my purpose were to win, I would design a test to exploit those disadvantages that he left.

You needn't insult me by 'agreeing' to a 'purpose' I don't claim. I don't propose this more vernacular test to 'win', I propose it because it is well-targeted to test whether a specific claim made under specific conditions is the result of common sighted bias.

But, as I said, I'm not in this for a personal beef, I'm in it for more generalizable results.

In that case you'll need far more subjects and scientific methods than anything this could provide.

Really, better to just test a specific claim.

If we did allow slow switching, that would become the excuse for any test failure, as you suggest.

Not if it's how the subject came to their claim in the first place.

I'm not sure, though, that allowing a level mismatch to persist is a good idea. Most anyone would pass that test.

You just don't seem to be getting my point.

 

[Mnyb]

…. And the test should be repeatable by others independently for verification otherwise its just an unicorn

 

[Platypus20]

$200/hr plus travel expenses. I'd also budget for a magician, but they come cheap.

Magicians are okay, but please no clowns <shudder>


Will the test be open for paying spectators ??

 

[ahofer]

You needn't insult me by 'agreeing' to a 'purpose' I don't claim. I don't propose this more vernacular test to 'win', I propose it because it is well-targeted to test whether a specific claim made under specific conditions is the result of common sighted bias.



In that case you'll need far more subjects and scientific methods than anything this could provide.

Really, better to just test a specific claim.





Not if it's how the subject came to their claim in the first place.




You just don't seem to be getting my point.

I'm sorry I insulted you, that was not my intention. And yes, I am clearly still missing some part of your point. At any rate, I'm less interested in testing the specific claim than doing something more broadly applicable. Quixotic, perhaps.

 

[olieb]

I agree that if my purpose were to win, I would design a test to exploit those disadvantages that he left. But, as I said, I'm not in this for a personal beef, I'm in it for more generalizable results. If we did allow slow switching, that would become the excuse for any test failure, as you suggest.

I'm not sure, though, that allowing a level mismatch to persist is a good idea. Most anyone would pass that test.

Absolutely, this whole betting and gambling invokes the mind set of the homo economicus instead of a scientific approach.

To make the test sensitive but reliable I would try to take care of some things
• (double) blind
• match levels
• use quick but not instant switching [to avoid switching artifacts]
• avoid audible noise [from DAC]
• I would NOT do 100 trials. This will make the experiment VERY annoying and exhausting. And if that many is necessary for p≤0.05 then the difference is negligible in the first place. Possibly it would only be an exploration of artifacts. [p<0.01 is better anyway]

open questions:
ABX? It is more sensitive than AB but more cumbersome too.
Do the selected DACs have sufficiently matching frequency response curves?

This whole endeavor will not deliver a general result, as this is only one run with one test person. But that is more than nothing of course.

 

[ahofer]

 

[ahofer]

this whole betting and gambling invokes the mind set of the homo economicus instead of a scientific approach.

I see your perspective, but the whole economic point of a betting market is to reveal information and make better predictions.

Betting vs. the Nuclear Option - Econlib

I have a long-standing dispute with Tyler Cowen about the epistemic value of betting. To my mind, my position is modest: Bets advance our knowledge by clarifying contentions and raising the price of error. While isolated bets don’t “prove” anything, they tip the argumentative scales in favor...

www.econlib.org

 

[DLS79]

I see your perspective, but the whole economic point of a betting market is to reveal information and make better predictions.

Betting vs. the Nuclear Option - Econlib

I have a long-standing dispute with Tyler Cowen about the epistemic value of betting. To my mind, my position is modest: Bets advance our knowledge by clarifying contentions and raising the price of error. While isolated bets don’t “prove” anything, they tip the argumentative scales in favor...

www.econlib.org

Imo, betting is bs, it's just a way of artificially limiting who can participate.

 

[ahofer]

Imo, betting is bs, it's just a way of artificially limiting who can participate.

You mean when people throw around large numbers? Or just betting in general.