Could someone help me to think through my ABX result using Bayesian reasoning?

[TunaBug]

I'm curious what you mean by "bit identical". Do you mean that the connection from the PC to DAC is digital and lossless? I'm willing to take that on faith. Our do you mean that you have captured the bitstream output using the different players, compared them, and found them identical? If there is an audible difference between the two players then I would assume they are not producing the same bits. I have no experience with XXhighend except for a bit of reading this thread has prompted. At https://audiophilestyle.com/forums/topic/62886-tutorial-for-setting-up-xxhighend/ I see a lot of talk about its built-in filters, so I'm assuming it is deliberately not a faithful decoding of the audio files?

 

[Killingbeans]

A sharp increase in amplitude.

Alright, doesn't really make sense to me. I think I'm just gonna wait for the measurements.

 

[manisandher]

I'm curious what you mean by "bit identical". Do you mean that the connection from the PC to DAC is digital and lossless?

That's certainly necessary, but not sufficient.

Our do you mean that you have captured the bitstream output using the different players, compared them, and found them identical?

For the ABX test, we captured the bitstream using the same player throughout. The bitstream remained the same, i.e., bit-identical, for two different settings on the player. So, you might conclude that they couldn't be doing anything. But I scored 9/10. And as I mentioned in the Priors, this was actually worse than I was expecting.

If there is an audible difference between the two players then I would assume they are not producing the same bits.

The bits definitely did not change in the ABX. Nor do they change when I change USB cables, and yet I hear differences (not tested in this particular ABX).

I see a lot of talk about its built-in filters, so I'm assuming it is deliberately not a faithful decoding of the audio files?

The filter settings in XXHighEnd remained the same for both A and B, so can be ruled out.

 

[Killingbeans]

But I scored 9/10. And as I mentioned in the Priors, this was actually worse than I was expecting.

But why did you only do one set of 10(?) and then stop?

 

[manisandher]

Alright, doesn't really make sense to me. I think I'm just gonna wait for the measurements.

Cool.

I intend to measure the effects of:
- different USB cables
- different 'transports' (SOtM streamer vs. PC as Roon endpoints)
- different software players

Chain will probably be:

digital 'transport' -> USB -> Okto dac8 PRO (analogue out) ->
i. RME ADI-2 Pro -> USB -> laptop (ADC capture)
ii. RME ADI-2 Pro -> (digital out) -> digital recorder [to ensure everything bit-identical]

 

[manisandher]

But why did you only do one set of 10(?) and then stop?

It took 18 minutes... of hard concentration. Not at all pleasant. And nothing like the normal experience of listening to music. Hence why I won't repeat it.

 

[MRC01]

... Any idea how a software setting might be able to cause an increase in jitter (keeping everything bit-identical)?

If the PC is sending the audio bits to an external DAC over SPDIF, then it is possible that the PC operating system temporarily pauses the process delivering the bits, in order to allow other processes to run. This could cause varying time delays in the SPDIF data stream -- jitter. The downstream DAC should buffer and reclock the data to eliminate this. But maybe it isn't.

We've seen in Amir's testing that some DAC have more jitter over SPDIF than over USB. This might be exacerbated if the source device sends the SPDIF data unevenly.

BTW, differences like this would still be "bit-identical" as the samples are exactly the same data, in the same order. It's just the timing between them that varies slightly.

 

[Killingbeans]

It took 18 minutes... of hard concentration. Not at all pleasant. And nothing like the normal experience of listening to music. Hence why I won't repeat it.

I don't get it. You said you were genuinely curious. You got hold of an expert... and even paid for his train ticket.

And then 18 minutes of hard work was enough to kill your curiosity? You could have made a weekend out of it. Take some nice long breaks, and do a set of 10 when you had the energy. 10 sets of 10 would have been way more usefull than a single set.

This might be a rude assumption, but it sounds like you got the result you wanted on the first try, and then used that as an excuse to call off the subsequent tests that could point to it as a fluke.

 

[mansr]

This might be a rude assumption, but it sounds like you got the result you wanted on the first try, and then used that as an excuse to call off the subsequent tests that could point to it as a fluke.

It was the third try, with some variations. Other than that, you're spot on.

 

[manisandher]

You said you were genuinely curious.

Of course.

... and even paid for his train ticket.

Yep.

And then 18 minutes of hard work was enough to kill your curiosity?

I got 9/10 in an ABX. That's a 99% probability that I was actually hearing something.

This might be a rude assumption, but it sounds like you got the result you wanted on the first try, and then used that as an excuse to call off the subsequent tests that could point to it as a fluke.

You're totally dismissing the Priors.

 

[manisandher]

It was the third try, with some variations.

Wow!

Test 1: A, B, X, X, X, X, X, X, X, X, X, X (~guessing)
Test 2: A, B, X, X, X, X, X, X, X, X, X, X (~guessing)
Test 3: A, B, X, A, B, X, A, B, X, A, B, X, A, B, X, A, B, X, A, B, X, A, B, X, A, B, X, A, B, X ( 9/10)

In what way is test 3 in any sense equivalent to tests 1 and 2, "with some variations"?

Totally disingenuous on your part.

 

[Pdxwayne]

Of course.



Yep.



I got 9/10 in an ABX. That's a 99% probability that I was actually hearing something.





You're totally dismissing the Priors.

I thought you said priors were not real abx?

If you think prior counts, then maybe you should add total numbers of tests together to get to a probability number? Something like 19 success out of 30, which is p value of 0.1? Typically 0.05 is considered significant.

 

[manisandher]

We've seen in Amir's testing that some DAC have more jitter over SPDIF than over USB. This might be exacerbated if the source device sends the SPDIF data unevenly.

The measurements I intend to take will be using USB, with a DAC and ADC that have both been tested by Amir. Their respective jitter measurements are very good.

 

[manisandher]

I thought you said priors were not real abx?

Sorry, I don't understand. The Priors are before any testing.

If you think prior counts, then maybe you should add total numbers of tests together to get to a probability number? Something like 19 success out of 30, which is p value of 0.1? Typically 0.05 is considered significant.

No, the first two tests were NOT Priors.

Should they be included? I don't think so. But they acted as a good 'warm-up' for the ABX itself... and were useful in proving there were no 'tells' during the ABX.

 

[Pdxwayne]

Sorry, I don't understand. The Priors are before any testing.




No, the first two tests were NOT Priors.

Should they be included? I don't think so. But they acted as a good 'warm-up' for the ABX itself... and were useful in proving there were no 'tells' during the ABX.

I see. Then priors are just your own observations, likely all sighted, correct?

The first two tests set were just warm up, but unsighted.

The last one is real abx.

I do understand that abx is exhausting. I have done lots of them. You can see my thread here that track the blind tests I have done:

Tracking my listening tests

I am using this to track my listening test results as posted in other threads. Online blind tests using my Samsung Note 9 with headphone. 0.5db - pass 0.2db - can't do up 5c pitch - pass 2 ways, 10ms - pass Screenshots in...

www.audiosciencereview.com

It would have been great, if you could take advantage of the present of third party, to run another 10 tests, after several hours of rest. That would have been even more convincing.

BTW, for me, warm up tests are for finding the tells.
; )

 

[manisandher]

Yes, this is what I mean by 'Priors'. It comes from Bayes's Rule:

Posterior probability = Prior probability x (likelihood of data / commoness of data)

"Your credence in a hypothesis after looking at the evidence should be your credence in the hypothesis before looking at the evidence, multiplied by how likely the evidence would be if the hypothesis is true, scaled by how common that evidence is across the board."

 

[manisandher]

It would have been great, if you could take advantage of the present of third party, to run another 10 tests, after several hours of rest. That would have been even more convincing.

I'm going to go on to replicable measurements. Much easier .

You can see my thread here that track the blind tests I have done:

Tracking my listening tests

I am using this to track my listening test results as posted in other threads. Online blind tests using my Samsung Note 9 with headphone. 0.5db - pass 0.2db - can't do up 5c pitch - pass 2 ways, 10ms - pass Screenshots in...

www.audiosciencereview.com

Will take a look.

 

[mansr]

The first two tests set were just warm up, but unsighted.

They were not "just to warm up." To prevent any accusations of imposing "unfair" test conditions, I let Mani choose the protocol, and he opted for a run of 10 unknowns after listening to both alternatives for reference. When he failed, he requested a change of test track. When he still failed, he switched to the repeated ABX sequence. I am certain that had either of those first two runs been "successful," the entire test would have been declared finished on the spot. There was no prior agreement regarding the number of runs, a failure on my part.

 

[Pdxwayne]

I'm going to go on to replicable measurements. Much easier .

Hopefully measurements showing something significant. Else people will say not audible even if you found differences. Then you will ended up having to run blind listening tests again just to prove the measurements differences are audible....
; )

 

[manisandher]

Hopefully measurements showing something significant. Else people will say not audible even if you found differences. Then you will ended up having to run blind listening tests again just to prove the measurements differences are audible....
; )

I intend to use a form of Signal Detection Theory. But it all depends on getting the signal to noise ratio as high as possible to see any effect.