• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required as is 20 years of participation in forums (not all true). There are daily reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

What to do about the ABX test?

Killingbeans

Major Contributor
Joined
Oct 23, 2018
Messages
3,009
Likes
5,194
Location
Bjerringbro, Denmark.
If we just stop pushing people so hard, they'll stay around and keep reading. And the chances are good that they will have a 'revelation' at some point. Sounds like a win-win to me.

Definitely. But the trick is to identify whether they are genuinely curious, or just here to wreak havoc.

A lot of new users appears to see themselves as missionaries of the shining truth of sighted listening. They have zero interest in reading anything ASR has to offer.
 

Ken1951

Addicted to Fun and Learning
Joined
Sep 28, 2020
Messages
554
Likes
1,054
Location
Blacksburg, VA
Definitely. But the trick is to identify whether they are genuinely curious, or just here to wreak havoc.

A lot of new users appears to see themselves as missionaries of the shining truth of sighted listening. They have zero interest in reading anything ASR has to offer.
Perfect example is the recent postings on the Pass Labs headphone amp. All the $100 devices you test with great measurements sound like crap. I love this amp. Why bother to test. The tested item is broken. Etc, Etc, Etc.

That's not going to change no matter what evidence is presented, what FAQs are provided. Maybe some new visitor who's lurking might be guided, but so many drive-bys convinced of their and their equipments superiority are not.
 

Thomas_A

Major Contributor
Forum Donor
Joined
Jun 20, 2019
Messages
2,298
Likes
1,615
Location
Sweden
ABX is fun! At least when there is a chance to score positive outcomes.

That should be the message. :)
 

RichB

Major Contributor
Forum Donor
Joined
May 24, 2019
Messages
1,817
Likes
2,421
Location
Massachusetts
Thanks @Blumlein 88 for this topic.

I did my first listening test to decide if I should bi-amp my Salon2s with two extra channels.
Years before, I used the AVR to do the channels but decided against it because it introduces another variable.

I settled on this testing methodology:

- Y Split a single channel
- Use two amplifier channels and a single speaker (inspired by Harmon speaker testing)
- Remove the straps and create a 12" path cable and stacking banana plugs to allow quick swapping between 1 amp and 2
- Use an iPad to allow the user to select tracks and ask for channels to be switched
- Listen at listening levels in the low to mid 80 dB at a few feet from the speaker to limit room interaction
- Level matching is not required
- Switching is done out of sight by an person behind the speakers, takes a few seconds.

This was a single blind test with the 3 listeners.
I did not record the results or tries but it was about 3 sessions for each person, and all were reliably able to pick identify the differences.
All felt there was better clarity with the bi-amp option.

As a result, the Salon2s owner and I have decided to bi-amp the mains.
Is there a technical explanation, sure the load is changed on the amplifiers and the crossovers, that are physically isolated (to reduce interference) are now electrically isolated.

The power used was low given the listening levels.
Another controversial topic, where online arm-chair posters will flay you.
This is a simple inexpensive test at home. Costs you maybe some cable, connectors and time.
Then, decide for yourself.

This proves nothing from a scientific perspective, but that was not the purpose.
It comes back to the question, what experiments can be done at home to aid in buying decisions?
This was one of those.

- Rich
 
Last edited:

RichB

Major Contributor
Forum Donor
Joined
May 24, 2019
Messages
1,817
Likes
2,421
Location
Massachusetts
ABX is fun! At least when there is a chance to score positive outcomes.

That should be the message. :)

Listening tests are tough to get right, adding an X to the selections make it worse.
Harmon does A/B listening tests for their speaker tests for good reason.
Add a third and people minds start to melt :p

- Rich
 

SIY

Master Contributor
Technical Expert
Joined
Apr 6, 2018
Messages
9,005
Likes
20,700
Location
Alfred, NY
Two common things keep popping up here:

1. Poor (fluid) specification of specifically what is being tested.

2. Conflation of the use of basic controls with ABX.
 

lashto

Addicted to Fun and Learning
Forum Donor
Joined
Mar 8, 2019
Messages
866
Likes
455
Definitely. But the trick is to identify whether they are genuinely curious, or just here to wreak havoc.

A lot of new users appears to see themselves as missionaries of the shining truth of sighted listening. They have zero interest in reading anything ASR has to offer.
true. But one can just use the "ignore" button.
The true havoc-masters can and should be managed by admins.

Much easier said then done, I know.
My ignore list is actually empty, I find that function kind of rude. And I might just be too curious for it..
 

lashto

Addicted to Fun and Learning
Forum Donor
Joined
Mar 8, 2019
Messages
866
Likes
455
ABX is fun! At least when there is a chance to score positive outcomes.

That should be the message. :)
not sure how are you going to make that gruesome stuff "fun", but +100 for the message/spirit
 

levimax

Major Contributor
Joined
Dec 28, 2018
Messages
1,768
Likes
2,562
Location
San Diego
After reading about ABX testing here I finally took the time to compare 2 amps which were quite different (tube 60 SINAD, SS 105 SINAD). This did take quite a bit of DIY setting up especially because I didn't want to switch the tube amp into "open" so instead when the tube amp was switched out it went to a dummy load. Eventually I got to where I had instant level matched switching. You can read about it all you want but actually experiencing the hopelessness (at least for me) of hearing any difference between these 2 radically different amps really gave me complete confidence about where to focus attention (speakers and room and recordings) and how much BS and bias there is regarding the electronic components. It would be nice if everyone who comes here could easily be given a chance to experience this for themselves but unfortunately that is not going to happen as the set up is quite involved. I wonder if someone with good speakers and a good microphone made level matched recordings of different amps, DAC's, cables, etc. and let people try to ABX them if that would be a practical way to get people to open their minds.
 

krabapple

Major Contributor
Joined
Apr 15, 2016
Messages
2,187
Likes
2,431
Listening tests are tough to get right, adding an X to the selections make it worse.
Harmon does A/B listening tests for their speaker tests for good reason.
Add a third and people minds start to melt :p

- Rich
it doesn't really add a third. The X is either A or B. Harman doesn't use them for speakers because ABX is a test for difference (or identity, to put it another way), not preference among things that do differ.
 

MattHooper

Major Contributor
Joined
Jan 27, 2019
Messages
4,501
Likes
7,212
After reading about ABX testing here I finally took the time to compare 2 amps which were quite different (tube 60 SINAD, SS 105 SINAD). This did take quite a bit of DIY setting up especially because I didn't want to switch the tube amp into "open" so instead when the tube amp was switched out it went to a dummy load. Eventually I got to where I had instant level matched switching. You can read about it all you want but actually experiencing the hopelessness (at least for me) of hearing any difference between these 2 radically different amps really gave me complete confidence about where to focus attention (speakers and room and recordings) and how much BS and bias there is regarding the electronic components. It would be nice if everyone who comes here could easily be given a chance to experience this for themselves but unfortunately that is not going to happen as the set up is quite involved. I wonder if someone with good speakers and a good microphone made level matched recordings of different amps, DAC's, cables, etc. and let people try to ABX them if that would be a practical way to get people to open their minds.

I mentioned your test earlier in the thread. I really appreciated your contributing that blind test report!

(Funny thing is when I blind tested my tube pre-amp I came to a slightly different conclusion than you did. Life's A Box Of Chocolates....)
 

levimax

Major Contributor
Joined
Dec 28, 2018
Messages
1,768
Likes
2,562
Location
San Diego
While it would probably not move the needle much for the skeptics I am curious about how one could "capture" the signal at the speaker terminals with an ADC so you could for instance capture the amp output at the speaker terminals for 2 different amps and see if you could hear or measure any difference between these "captures". Not sure the best way to do this without blowing things up.
 

RichB

Major Contributor
Forum Donor
Joined
May 24, 2019
Messages
1,817
Likes
2,421
Location
Massachusetts
it doesn't really add a third. The X is either A or B. Harman doesn't use them for speakers because ABX is a test for difference (or identity, to put it another way), not preference among things that do differ.

Not the Wikipedia is a trusted source but:


An ABX test is a method of comparing two choices of sensory stimuli to identify detectable differences between them. A subject is presented with two known samples (sample A, the first reference, and sample B, the second reference) followed by one unknown sample X that is randomly selected from either A or B. The subject is then required to identify X as either A or B. If X cannot be identified reliably with a low p-value in a predetermined number of trials, then the null hypothesis cannot be rejected and it cannot be proven that there is a perceptible difference between A and B.

- Rich
 

Cbdb2

Addicted to Fun and Learning
Joined
Sep 8, 2019
Messages
846
Likes
784
Location
Vancouver
No-one has claimed otherwise. I am simply interested in what level of digital resolution is needed for absolute transparency in perfect listening conditions with trained listeners.
If you listen to the end of the fade out where its down 70db with 70db of gain above normal listening level you will probably be able to tell the difference between Cd and Hi Res. This is what Amir did. So does absolute transparency exist if you keep pushing this? IMHO this is a cheat, it does show the DUT limits, but has nothing to do with normal listening. Like taking 2 100 watt amps and turning them up to full volume with no source and comparing the hiss that is inaudable at normal levels. Does it matter?
 
OP
Blumlein 88

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
17,186
Likes
29,949
While it would probably not move the needle much for the skeptics I am curious about how one could "capture" the signal at the speaker terminals with an ADC so you could for instance capture the amp output at the speaker terminals for 2 different amps and see if you could hear or measure any difference between these "captures". Not sure the best way to do this without blowing things up.
A resistive attenuation. There are some devices made for this purpose also.
 
OP
Blumlein 88

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
17,186
Likes
29,949
If you listen to the end of the fade out where its down 70db with 70db of gain above normal listening level you will probably be able to tell the difference between Cd and Hi Res. This is what Amir did. So does absolute transparency exist if you keep pushing this? IMHO this is a cheat, it does show the DUT limits, but has nothing to do with normal listening. Like taking 2 100 watt amps and turning them up to full volume with no source and comparing the hiss that is inaudable at normal levels. Does it matter?
If this is what he did, then it has no bearing on normal listening.
 

RichB

Major Contributor
Forum Donor
Joined
May 24, 2019
Messages
1,817
Likes
2,421
Location
Massachusetts
If you listen to the end of the fade out where its down 70db with 70db of gain above normal listening level you will probably be able to tell the difference between Cd and Hi Res. This is what Amir did. So does absolute transparency exist if you keep pushing this? IMHO this is a cheat, it does show the DUT limits, but has nothing to do with normal listening. Like taking 2 100 watt amps and turning them up to full volume with no source and comparing the hiss that is inaudable at normal levels. Does it matter?

@amirm can jump in but I do not believe that is correct. It was the sound of the keys.
If one acknowledges a difference, then this does not mean that one is correct. Perhaps for example, the difference is artifacts produced by the reconstruction filter. In that case, the proper solution is to use a different DAC or filter.

Unless one has bat hearing, any difference would have to be in the audible frequency range.
This is undeniable (for rational minds).
Something in the playback system accounts for the difference that may not point to the inherent superiority of HD-Audio.
If one simply wants the buy the best possible quality for a recording, that makes sense.
Unfortunately, that has more to do with the mastering than the format.

- Rich
 
Last edited:

kemmler3D

Addicted to Fun and Learning
Joined
Aug 25, 2022
Messages
665
Likes
1,266
Location
San Francisco
the trick is to identify whether they are genuinely curious, or just here to wreak havoc.
I think with a good FAQ, that will be easy enough. Genuinely curious people will read the FAQ and reply with better-informed questions from that point on. Trolls will not read the FAQ, will reply with some variation on "Stop dodging the question and trying to lecture me, egghead!" and can be safely ignored from that point on.
 

Cbdb2

Addicted to Fun and Learning
Joined
Sep 8, 2019
Messages
846
Likes
784
Location
Vancouver
@amirm can jump in but I do not believe that is correct. It was the sound of the keys.
If one acknowledges a difference, then this does not mean that one is correct. Perhaps for example, the difference is artifacts produced by the reconstruction filter. In that case, the proper solution is to use a different DAC or filter.

Unless one has bat hearing, any difference would have to be in the audible frequency range.
This is undeniable (for rational minds).
Something in the playback system accounts for the difference that may not point to the inherent superiority of HD-Audio.
If one simply wants the buy the best possible quality for a recording, that makes sense.
Unfortunately, that has more to do with the mastering than the format.

- Rich
Did I hear wrong? Post 16 at 19 minutes
 
Top Bottom