What to do about the ABX test?

[Killingbeans]

If we just stop pushing people so hard, they'll stay around and keep reading. And the chances are good that they will have a 'revelation' at some point. Sounds like a win-win to me.

Definitely. But the trick is to identify whether they are genuinely curious, or just here to wreak havoc.

A lot of new users appears to see themselves as missionaries of the shining truth of sighted listening. They have zero interest in reading anything ASR has to offer.

 

[Ken1951]

Definitely. But the trick is to identify whether they are genuinely curious, or just here to wreak havoc.

A lot of new users appears to see themselves as missionaries of the shining truth of sighted listening. They have zero interest in reading anything ASR has to offer.

Perfect example is the recent postings on the Pass Labs headphone amp. All the $100 devices you test with great measurements sound like crap. I love this amp. Why bother to test. The tested item is broken. Etc, Etc, Etc.

That's not going to change no matter what evidence is presented, what FAQs are provided. Maybe some new visitor who's lurking might be guided, but so many drive-bys convinced of their and their equipments superiority are not.

 

[Thomas_A]

ABX is fun! At least when there is a chance to score positive outcomes.

That should be the message.

 

[RichB]

Thanks @Blumlein 88 for this topic.

I did my first listening test to decide if I should bi-amp my Salon2s with two extra channels.
Years before, I used the AVR to do the channels but decided against it because it introduces another variable.

I settled on this testing methodology:

- Y Split a single channel
- Use two amplifier channels and a single speaker (inspired by Harmon speaker testing)
- Remove the straps and create a 12" path cable and stacking banana plugs to allow quick swapping between 1 amp and 2
- Use an iPad to allow the user to select tracks and ask for channels to be switched
- Listen at listening levels in the low to mid 80 dB at a few feet from the speaker to limit room interaction
- Level matching is not required
- Switching is done out of sight by an person behind the speakers, takes a few seconds.

This was a single blind test with the 3 listeners.
I did not record the results or tries but it was about 3 sessions for each person, and all were reliably able to pick identify the differences.
All felt there was better clarity with the bi-amp option.

As a result, the Salon2s owner and I have decided to bi-amp the mains.
Is there a technical explanation, sure the load is changed on the amplifiers and the crossovers, that are physically isolated (to reduce interference) are now electrically isolated.

The power used was low given the listening levels.
Another controversial topic, where online arm-chair posters will flay you.
This is a simple inexpensive test at home. Costs you maybe some cable, connectors and time.
Then, decide for yourself.

This proves nothing from a scientific perspective, but that was not the purpose.
It comes back to the question, what experiments can be done at home to aid in buying decisions?
This was one of those.

- Rich

 

[RichB]

ABX is fun! At least when there is a chance to score positive outcomes.

That should be the message.

Listening tests are tough to get right, adding an X to the selections make it worse.
Harmon does A/B listening tests for their speaker tests for good reason.
Add a third and people minds start to melt

- Rich

 

[SIY]

Two common things keep popping up here:

1. Poor (fluid) specification of specifically what is being tested.

2. Conflation of the use of basic controls with ABX.

 

[lashto]

Definitely. But the trick is to identify whether they are genuinely curious, or just here to wreak havoc.

A lot of new users appears to see themselves as missionaries of the shining truth of sighted listening. They have zero interest in reading anything ASR has to offer.

true. But one can just use the "ignore" button.
The true havoc-masters can and should be managed by admins.

Much easier said then done, I know.
My ignore list is actually empty, I find that function kind of rude. And I might just be too curious for it..

 

[lashto]

ABX is fun! At least when there is a chance to score positive outcomes.

That should be the message.

not sure how are you going to make that gruesome stuff "fun", but +100 for the message/spirit

 

[levimax]

After reading about ABX testing here I finally took the time to compare 2 amps which were quite different (tube 60 SINAD, SS 105 SINAD). This did take quite a bit of DIY setting up especially because I didn't want to switch the tube amp into "open" so instead when the tube amp was switched out it went to a dummy load. Eventually I got to where I had instant level matched switching. You can read about it all you want but actually experiencing the hopelessness (at least for me) of hearing any difference between these 2 radically different amps really gave me complete confidence about where to focus attention (speakers and room and recordings) and how much BS and bias there is regarding the electronic components. It would be nice if everyone who comes here could easily be given a chance to experience this for themselves but unfortunately that is not going to happen as the set up is quite involved. I wonder if someone with good speakers and a good microphone made level matched recordings of different amps, DAC's, cables, etc. and let people try to ABX them if that would be a practical way to get people to open their minds.

 

[krabapple]

Listening tests are tough to get right, adding an X to the selections make it worse.
Harmon does A/B listening tests for their speaker tests for good reason.
Add a third and people minds start to melt

- Rich

it doesn't really add a third. The X is either A or B. Harman doesn't use them for speakers because ABX is a test for difference (or identity, to put it another way), not preference among things that do differ.

 

[MattHooper]

After reading about ABX testing here I finally took the time to compare 2 amps which were quite different (tube 60 SINAD, SS 105 SINAD). This did take quite a bit of DIY setting up especially because I didn't want to switch the tube amp into "open" so instead when the tube amp was switched out it went to a dummy load. Eventually I got to where I had instant level matched switching. You can read about it all you want but actually experiencing the hopelessness (at least for me) of hearing any difference between these 2 radically different amps really gave me complete confidence about where to focus attention (speakers and room and recordings) and how much BS and bias there is regarding the electronic components. It would be nice if everyone who comes here could easily be given a chance to experience this for themselves but unfortunately that is not going to happen as the set up is quite involved. I wonder if someone with good speakers and a good microphone made level matched recordings of different amps, DAC's, cables, etc. and let people try to ABX them if that would be a practical way to get people to open their minds.

I mentioned your test earlier in the thread. I really appreciated your contributing that blind test report!

(Funny thing is when I blind tested my tube pre-amp I came to a slightly different conclusion than you did. Life's A Box Of Chocolates....)

 

[levimax]

While it would probably not move the needle much for the skeptics I am curious about how one could "capture" the signal at the speaker terminals with an ADC so you could for instance capture the amp output at the speaker terminals for 2 different amps and see if you could hear or measure any difference between these "captures". Not sure the best way to do this without blowing things up.

 

[RichB]

it doesn't really add a third. The X is either A or B. Harman doesn't use them for speakers because ABX is a test for difference (or identity, to put it another way), not preference among things that do differ.

Not the Wikipedia is a trusted source but:

ABX test - Wikipedia

en.wikipedia.org

An ABX test is a method of comparing two choices of sensory stimuli to identify detectable differences between them. A subject is presented with two known samples (sample A, the first reference, and sample B, the second reference) followed by one unknown sample X that is randomly selected from either A or B. The subject is then required to identify X as either A or B. If X cannot be identified reliably with a low p-value in a predetermined number of trials, then the null hypothesis cannot be rejected and it cannot be proven that there is a perceptible difference between A and B.

- Rich

 

[Cbdb2]

No-one has claimed otherwise. I am simply interested in what level of digital resolution is needed for absolute transparency in perfect listening conditions with trained listeners.

If you listen to the end of the fade out where its down 70db with 70db of gain above normal listening level you will probably be able to tell the difference between Cd and Hi Res. This is what Amir did. So does absolute transparency exist if you keep pushing this? IMHO this is a cheat, it does show the DUT limits, but has nothing to do with normal listening. Like taking 2 100 watt amps and turning them up to full volume with no source and comparing the hiss that is inaudable at normal levels. Does it matter?

 

[Blumlein 88]

While it would probably not move the needle much for the skeptics I am curious about how one could "capture" the signal at the speaker terminals with an ADC so you could for instance capture the amp output at the speaker terminals for 2 different amps and see if you could hear or measure any difference between these "captures". Not sure the best way to do this without blowing things up.

A resistive attenuation. There are some devices made for this purpose also.

 

[Blumlein 88]

If you listen to the end of the fade out where its down 70db with 70db of gain above normal listening level you will probably be able to tell the difference between Cd and Hi Res. This is what Amir did. So does absolute transparency exist if you keep pushing this? IMHO this is a cheat, it does show the DUT limits, but has nothing to do with normal listening. Like taking 2 100 watt amps and turning them up to full volume with no source and comparing the hiss that is inaudable at normal levels. Does it matter?

If this is what he did, then it has no bearing on normal listening.

 

[Cbdb2]

If this is what he did, then it has no bearing on normal listening.

If you go to 19minutes in the post 16 video he explains this.

 

[RichB]

If you listen to the end of the fade out where its down 70db with 70db of gain above normal listening level you will probably be able to tell the difference between Cd and Hi Res. This is what Amir did. So does absolute transparency exist if you keep pushing this? IMHO this is a cheat, it does show the DUT limits, but has nothing to do with normal listening. Like taking 2 100 watt amps and turning them up to full volume with no source and comparing the hiss that is inaudable at normal levels. Does it matter?

@amirm can jump in but I do not believe that is correct. It was the sound of the keys.
If one acknowledges a difference, then this does not mean that one is correct. Perhaps for example, the difference is artifacts produced by the reconstruction filter. In that case, the proper solution is to use a different DAC or filter.

Unless one has bat hearing, any difference would have to be in the audible frequency range.
This is undeniable (for rational minds).
Something in the playback system accounts for the difference that may not point to the inherent superiority of HD-Audio.
If one simply wants the buy the best possible quality for a recording, that makes sense.
Unfortunately, that has more to do with the mastering than the format.

- Rich

 

[kemmler3D]

the trick is to identify whether they are genuinely curious, or just here to wreak havoc.

I think with a good FAQ, that will be easy enough. Genuinely curious people will read the FAQ and reply with better-informed questions from that point on. Trolls will not read the FAQ, will reply with some variation on "Stop dodging the question and trying to lecture me, egghead!" and can be safely ignored from that point on.

 

[Cbdb2]

@amirm can jump in but I do not believe that is correct. It was the sound of the keys.
If one acknowledges a difference, then this does not mean that one is correct. Perhaps for example, the difference is artifacts produced by the reconstruction filter. In that case, the proper solution is to use a different DAC or filter.

Unless one has bat hearing, any difference would have to be in the audible frequency range.
This is undeniable (for rational minds).
Something in the playback system accounts for the difference that may not point to the inherent superiority of HD-Audio.
If one simply wants the buy the best possible quality for a recording, that makes sense.
Unfortunately, that has more to do with the mastering than the format.

- Rich

Did I hear wrong? Post 16 at 19 minutes