# Could someone help me to think through my ABX result using Bayesian reasoning?

#### manisandher

##### Senior Member
Priors:
1. For many years, I'd been certain that I could hear clear differences between bit-identical playback means (e.g., different USB cables, streaming vs. local playback, different software players and their bit-identical settings, etc.) - something that has bugged me no end over the years
2. I wanted to understand what could be causing what I was hearing, with the hope that an interesting insight (a way of further improving the SQ of digital audio perhaps?) might be gained
3. I was confident that I could demonstrate what I was hearing in a controlled listening test
4. I invited @mansr to my place (paid for his train ticket, no less) to help conduct a listening test (I chose 2 different but bit-identical setttings in a software player as the single variable)
5. the test was almost double-blind
- Mans controlled the test from my office, two doors and a corridor away from me in my listening room (I could not see or hear anything he was doing)
- Mans used a random generator on his phone to determine whether X=A or X=B
6. I was expecting to get close to 100% correct, so clear were the differences I'd been hearing for years

Likelihood:
- I scored 9/10 in the only ABX we conducted
- I got #9 incorrect - we were 15 minutes into ABX at that point, and I was feeling fatigued (we'd conducted 2 non-ABX tests beforehand - see below)
- input into DAC during ABX was shown to remain bit-identical*

Commoness:
- p = 0.01

Disclosure:
- we'd conducted 2 non-ABX tests before the ABX, where I scored no better than guessing
- even here, I was certain I could hear differences, but had no reference against which to assess, so simply guessed A or B in my responses
- I don't think it's fair to include the results of these non-ABX tests in the ABX results (also, I'd never been involved in any tests beforehand, and was adjusting to the strange/unnatural setup)
- the results of these two non-ABX tests suggest there were no 'tells' during any of the tests
- * there was a glitch in the auto-start function of the digital recorder being used, so the first few ms (out of ~15,000 ms) of each digital capture had to be ignored
- there were no conflicts of interest on my part - I was neither selling nor promoting any product/service (I'm not a manufacturer or fan of any particular product)
- this test was conducted out of pure curiosity on my part
- I HATE that these differences seem to exist (to me, at least) - I do NOT want USB cables or different software players to sound different... and yet they seem to (to me)

It's not inconcievable that someone selected randomly from the street would score 9/10 in the first ABX they sat - there's a 1 in 100 chance that they would.

However... given the Priors above, surely the 9/10 result shouldn't be easily dismissed as a simple fluke?

Could someone help me think this through using Bayesian reasoning please?

FWIW, I'm still certain I can hear differences in bit-identical replay, to my genuine chagrin. No more ABX tests for me though - I hated the whole experience. However, using a low-noise DAC-ADC chain and @pkane 's DeltaWave, I'm pretty confident I can measure these differences now. I intend to do just that over the Christmas break, and will come back with what I find.

Mani.

#### mansr

##### Major Contributor
I invited @mansr to my place (paid for his train ticket, no less)
Since you keep making a big deal out of that, why don't you also mention that you picked me up at the railway station in your limited edition Porsche. You know, just so we all understand how wealthy you are.

OP

#### manisandher

##### Senior Member
Wow!

The fact that I was willing to pay for your train ticket is an important part of the Priors. Sorry if that's lost on you.

And in any event, I don't measure my wealth in terms of what I own, but in terms of the people in my life (hence the tattoo on my arm).

(I still have the 911_50 though )

Mani.

#### mansr

##### Major Contributor
Is that a Rolls Royce logo?

#### MRC01

##### Major Contributor
Bayesian inference is a double-edged sword. The benefits are well known. Yet if the prior information is wrong, mis-interpreted, or mis-applied, it works against you. Choose your priors carefully. Most of the priors you mention can't be quantified anyway.

Why not take it at face value? 9 of 10 correct is 98.93% confidence. Even if the sources were bit identical, there must have been something different. Whatever it was, odds are, you heard it.

BTW, what exactly was different?

#### Pdxwayne

##### Major Contributor
I think you should not consider practice runs.

What exactly are the chains? How do you switch between the chain? What kind of difference did you listen for?

#### Chester

##### Active Member
From reading the first post I had assumed the two of you were friends, but you don’t sound like friends

#### Pdxwayne

##### Major Contributor
From reading the first post I had assumed the two of you were friends, but you don’t sound like friends
Maybe we don't get the British humor.....

OP

#### manisandher

##### Senior Member
Most of the priors you mention can't be quantified anyway.

How about my expectation of getting 100% correct (based on my experience over the years)? (Hence why I set up the test... hence why I paid for train ticket.)

BTW, what exactly was different?

The difference was a particular setting in the software player (XXHighEnd), known as 'SFS'. As far as I can tell (the developer is coy), different SFS settings 'load' the playback PC in different ways... but maintain the digital signal entering the DAC (as proven by the bit-identical captures from the digital recorder during the ABX).

The hypothesis is then that different levels of noise manage to make their way into the DAC due to the different PC loads (mechanism unknown), which have different effects on the digital-to-analogue conversion (mechanism unknown).

These effects don't seem to appear on any current measurements at the output of the DAC though, including jitter. My hypothesis here would be that the effects are amplified in highly varying signals with transients, like music, as opposed to steady-state signals like single- or multi-tone sine waves.

Mani.

#### MRC01

##### Major Contributor
The difference was a particular setting in the software player (XXHighEnd), known as 'SFS'. As far as I can tell (the developer is coy), different SFS settings 'load' the playback PC in different ways... but maintain the digital signal entering the DAC (as proven by the bit-identical captures from the digital recorder during the ABX). ...
You said "the difference" which implies no other differences - everything else identical?
The DAC was running on the PC? If so, the idea is that CPU/RAM loading of the PC changes its power consumption, which can cause slight variations in DC voltages which can affect a DAC's output?

OP

#### manisandher

##### Senior Member
From reading the first post I had assumed the two of you were friends, but you don’t sound like friends

We could have been, perhaps. But after the test, some animosity seemed to have crept in. After positing a number of other explanations for the 9/10 result, all of which were refuted by data, Mans simply claimed that it must have therefore been a fluke.

After all the trouble I'd been through getting the test set up and then actually done, I suppose I took some umbrage with this. And also, cheap 'n' cheerful as it was (conducted in my home - a domestic setting after all), I felt that he was dismissing something that was possibly valuable and insightful.

Mani.

OP

#### manisandher

##### Senior Member
You said "the difference" which implies no other differences - everything else identical?

Yes, everything else identical.

The DAC was running on the PC?

No. DAC had own power supply. In the case of the test, the DAC was connected to PC via BNC spdif.

#### MRC01

##### Major Contributor
Perhaps CPU/RAM loading of the PC caused timing issues creating jitter in the SPDIF data the PC sent to the DAC, and the DAC did not do a sufficient job re-clocking the data?

Without knowing details of the setup, this is mere speculation. If you're looking for opinions on where an audible difference could have emerged, it would be helpful to have a diagram of the setup.

#### Killingbeans

##### Major Contributor
My hypothesis here would be that the effects are amplified in highly varying signals with transients, like music, as opposed to steady-state signals like single- or multi-tone sine waves.

In other words: A non-flat frequency response? That should be pretty easy to measure.

- I scored 9/10 in the only ABX we conducted

What was the reason you only did one?

OP

#### manisandher

##### Senior Member
If you're looking for opinions on where an audible difference could have emerged, it would be helpful to have a diagram of the setup.

No, actually I'm not. This test was conducted a few years ago now, and is pretty much done and dusted I think.

But I'd like to learn more about Bayesian reasoning, because I suspect it will apply when I do some measurements over Christmas. I just thought this test would be a useful example from me to learn from.

#### pkane

##### Major Contributor
Forum Donor
No. DAC had own power supply. In the case of the test, the DAC was connected to PC via BNC spdif.
The hypothesis is then that different levels of noise manage to make their way into the DAC due to the different PC loads (mechanism unknown), which have different effects on the digital-to-analogue conversion (mechanism unknown).

This didn't test for this specific hypothesis. Other differences, caused by some "unknown" software setting, could've resulted in increased jitter, for example. There were no tests done to demonstrate that PC noise was the cause.

OP

#### manisandher

##### Senior Member
What kind of difference did you listen for?
In other words: A non-flat frequency response? That should be pretty easy to measure.

The differences I (still) hear between various bit-identical replay schemes have nothing to do with frequency response. It's ususally the leading edges of instruments and voices that are the give away for me. In the case of the ABX, it was the leading edges of piano notes.

OP

#### manisandher

##### Senior Member
This didn't test for this specific hypothesis... There were no tests done to demonstrate that PC noise was the cause.

Agreed.

Other differences, caused by some "unknown" software setting, could've resulted in increased jitter, for example.

Any idea how a software setting might be able to cause an increase in jitter (keeping everything bit-identical)?

#### Killingbeans

##### Major Contributor
The differences I (still) hear between various bit-identical replay schemes have nothing to do with frequency response.

You were talking about transients in music. They are a sum of single frequencies. A D-A conversion doesn't care about the "bustle" of music. If all of the frequency content is treated equally, then you have nothing to worry about.

It's ususally the leading edges of instruments and voices that are the give away for me. In the case of the ABX, it was the leading edges of piano notes.

Feel free to call me dumb, but what on earth is a "leading edge" of an instrument?

OP

#### manisandher

##### Senior Member
Feel free to call me dumb, but what on earth is a "leading edge" of an instrument?

A sharp increase in amplitude.

Replies
21
Views
1K
Replies
71
Views
3K
Replies
4
Views
515
Replies
33
Views
3K
Replies
10
Views
759