• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

What to do about the ABX test?

lashto

Major Contributor
Forum Donor
Joined
Mar 8, 2019
Messages
1,045
Likes
535
ABX is fun! At least when there is a chance to score positive outcomes.

That should be the message. :)
not sure how are you going to make that gruesome stuff "fun", but +100 for the message/spirit
 

levimax

Major Contributor
Joined
Dec 28, 2018
Messages
2,348
Likes
3,462
Location
San Diego
After reading about ABX testing here I finally took the time to compare 2 amps which were quite different (tube 60 SINAD, SS 105 SINAD). This did take quite a bit of DIY setting up especially because I didn't want to switch the tube amp into "open" so instead when the tube amp was switched out it went to a dummy load. Eventually I got to where I had instant level matched switching. You can read about it all you want but actually experiencing the hopelessness (at least for me) of hearing any difference between these 2 radically different amps really gave me complete confidence about where to focus attention (speakers and room and recordings) and how much BS and bias there is regarding the electronic components. It would be nice if everyone who comes here could easily be given a chance to experience this for themselves but unfortunately that is not going to happen as the set up is quite involved. I wonder if someone with good speakers and a good microphone made level matched recordings of different amps, DAC's, cables, etc. and let people try to ABX them if that would be a practical way to get people to open their minds.
 

krabapple

Major Contributor
Forum Donor
Joined
Apr 15, 2016
Messages
3,168
Likes
3,715
Listening tests are tough to get right, adding an X to the selections make it worse.
Harmon does A/B listening tests for their speaker tests for good reason.
Add a third and people minds start to melt :p

- Rich
it doesn't really add a third. The X is either A or B. Harman doesn't use them for speakers because ABX is a test for difference (or identity, to put it another way), not preference among things that do differ.
 

MattHooper

Master Contributor
Forum Donor
Joined
Jan 27, 2019
Messages
7,195
Likes
11,808
After reading about ABX testing here I finally took the time to compare 2 amps which were quite different (tube 60 SINAD, SS 105 SINAD). This did take quite a bit of DIY setting up especially because I didn't want to switch the tube amp into "open" so instead when the tube amp was switched out it went to a dummy load. Eventually I got to where I had instant level matched switching. You can read about it all you want but actually experiencing the hopelessness (at least for me) of hearing any difference between these 2 radically different amps really gave me complete confidence about where to focus attention (speakers and room and recordings) and how much BS and bias there is regarding the electronic components. It would be nice if everyone who comes here could easily be given a chance to experience this for themselves but unfortunately that is not going to happen as the set up is quite involved. I wonder if someone with good speakers and a good microphone made level matched recordings of different amps, DAC's, cables, etc. and let people try to ABX them if that would be a practical way to get people to open their minds.

I mentioned your test earlier in the thread. I really appreciated your contributing that blind test report!

(Funny thing is when I blind tested my tube pre-amp I came to a slightly different conclusion than you did. Life's A Box Of Chocolates....)
 

levimax

Major Contributor
Joined
Dec 28, 2018
Messages
2,348
Likes
3,462
Location
San Diego
While it would probably not move the needle much for the skeptics I am curious about how one could "capture" the signal at the speaker terminals with an ADC so you could for instance capture the amp output at the speaker terminals for 2 different amps and see if you could hear or measure any difference between these "captures". Not sure the best way to do this without blowing things up.
 

RichB

Major Contributor
Forum Donor
Joined
May 24, 2019
Messages
1,946
Likes
2,611
Location
Massachusetts
it doesn't really add a third. The X is either A or B. Harman doesn't use them for speakers because ABX is a test for difference (or identity, to put it another way), not preference among things that do differ.

Not the Wikipedia is a trusted source but:


An ABX test is a method of comparing two choices of sensory stimuli to identify detectable differences between them. A subject is presented with two known samples (sample A, the first reference, and sample B, the second reference) followed by one unknown sample X that is randomly selected from either A or B. The subject is then required to identify X as either A or B. If X cannot be identified reliably with a low p-value in a predetermined number of trials, then the null hypothesis cannot be rejected and it cannot be proven that there is a perceptible difference between A and B.

- Rich
 

Cbdb2

Major Contributor
Joined
Sep 8, 2019
Messages
1,530
Likes
1,485
Location
Vancouver
No-one has claimed otherwise. I am simply interested in what level of digital resolution is needed for absolute transparency in perfect listening conditions with trained listeners.
If you listen to the end of the fade out where its down 70db with 70db of gain above normal listening level you will probably be able to tell the difference between Cd and Hi Res. This is what Amir did. So does absolute transparency exist if you keep pushing this? IMHO this is a cheat, it does show the DUT limits, but has nothing to do with normal listening. Like taking 2 100 watt amps and turning them up to full volume with no source and comparing the hiss that is inaudable at normal levels. Does it matter?
 
OP
Blumlein 88

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,523
Likes
37,054
While it would probably not move the needle much for the skeptics I am curious about how one could "capture" the signal at the speaker terminals with an ADC so you could for instance capture the amp output at the speaker terminals for 2 different amps and see if you could hear or measure any difference between these "captures". Not sure the best way to do this without blowing things up.
A resistive attenuation. There are some devices made for this purpose also.
 
OP
Blumlein 88

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,523
Likes
37,054
If you listen to the end of the fade out where its down 70db with 70db of gain above normal listening level you will probably be able to tell the difference between Cd and Hi Res. This is what Amir did. So does absolute transparency exist if you keep pushing this? IMHO this is a cheat, it does show the DUT limits, but has nothing to do with normal listening. Like taking 2 100 watt amps and turning them up to full volume with no source and comparing the hiss that is inaudable at normal levels. Does it matter?
If this is what he did, then it has no bearing on normal listening.
 

RichB

Major Contributor
Forum Donor
Joined
May 24, 2019
Messages
1,946
Likes
2,611
Location
Massachusetts
If you listen to the end of the fade out where its down 70db with 70db of gain above normal listening level you will probably be able to tell the difference between Cd and Hi Res. This is what Amir did. So does absolute transparency exist if you keep pushing this? IMHO this is a cheat, it does show the DUT limits, but has nothing to do with normal listening. Like taking 2 100 watt amps and turning them up to full volume with no source and comparing the hiss that is inaudable at normal levels. Does it matter?

@amirm can jump in but I do not believe that is correct. It was the sound of the keys.
If one acknowledges a difference, then this does not mean that one is correct. Perhaps for example, the difference is artifacts produced by the reconstruction filter. In that case, the proper solution is to use a different DAC or filter.

Unless one has bat hearing, any difference would have to be in the audible frequency range.
This is undeniable (for rational minds).
Something in the playback system accounts for the difference that may not point to the inherent superiority of HD-Audio.
If one simply wants the buy the best possible quality for a recording, that makes sense.
Unfortunately, that has more to do with the mastering than the format.

- Rich
 
Last edited:

kemmler3D

Major Contributor
Forum Donor
Joined
Aug 25, 2022
Messages
3,013
Likes
5,614
Location
San Francisco
the trick is to identify whether they are genuinely curious, or just here to wreak havoc.
I think with a good FAQ, that will be easy enough. Genuinely curious people will read the FAQ and reply with better-informed questions from that point on. Trolls will not read the FAQ, will reply with some variation on "Stop dodging the question and trying to lecture me, egghead!" and can be safely ignored from that point on.
 

Cbdb2

Major Contributor
Joined
Sep 8, 2019
Messages
1,530
Likes
1,485
Location
Vancouver
@amirm can jump in but I do not believe that is correct. It was the sound of the keys.
If one acknowledges a difference, then this does not mean that one is correct. Perhaps for example, the difference is artifacts produced by the reconstruction filter. In that case, the proper solution is to use a different DAC or filter.

Unless one has bat hearing, any difference would have to be in the audible frequency range.
This is undeniable (for rational minds).
Something in the playback system accounts for the difference that may not point to the inherent superiority of HD-Audio.
If one simply wants the buy the best possible quality for a recording, that makes sense.
Unfortunately, that has more to do with the mastering than the format.

- Rich
Did I hear wrong? Post 16 at 19 minutes
 

GaryH

Major Contributor
Joined
May 12, 2021
Messages
1,348
Likes
1,804
Unless one has bat hearing, any difference would have to be in the audible frequency range.
This is undeniable (for rational minds).
Something in the playback system accounts for the difference that may not point to the inherent superiority of HD-Audio.
Yep, I've pointed out the invalidity of these tests before.
 
Last edited:
OP
Blumlein 88

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,523
Likes
37,054
If you go to 19minutes in the post 16 video he explains this.
Okay, this wasn't jangling keys as someone else mentioned in this case. It was a music file in Archimago's hi-res listening test. And yes, he did use IEM's and found levels fading under -70 dbFS and pumped up the volume. I do think that is cheating. Amir even says without doing this he could hear no difference. I don't see this as any different than dumping a file in a sound editor and looking for differences. It has zero to do with any normal listening. Amir doesn't consider 16 bits transparent due to this, while I'd say for normal listening it is. One could say 24 bit encoding is transparent to levels beyond the ability of any hardware and 16 bits isn't, but if 16 bits gets enough humans can never tell then what have we gained?

Just after this he shows hearing Ethan's generational test. I don't know what the difference is, but Amir said except for one track with a tale accidentally left in he couldn't hear the 8th generation files I have posted. Of course I didn't leave any dead space in files to prevent someone doing the find quiet portions and amp up the noise to hear differences thing.

What this shows is the use of ABX listening test with technical knowledge of how things work one can discern extremely small differences using ears only. Other methods of blind testing work beyond just the ABX method.
 

fpitas

Master Contributor
Forum Donor
Joined
Jul 7, 2022
Messages
9,885
Likes
14,191
Location
Northern Virginia, USA
The
true. But one can just use the "ignore" button.
The true havoc-masters can and should be managed by admins.

Much easier said then done, I know.
My ignore list is actually empty, I find that function kind of rude. And I might just be too curious for it..
And around here the mods are pretty efficient. The real bad news sorts disappear quickly enough.
 

dlaloum

Major Contributor
Joined
Oct 4, 2021
Messages
3,092
Likes
2,353
I've had this debate a couple of times. It goes like this:"the amplifiers only sound different with (insert name of B&W or Magico printer here) so obviously my speakers are more revealing"
Yep - hence we should all have speakers with impedance dips to 1 ohm, as they are more revealing...

Having said that - those 1ohm stats, are superb speakers, and my current Gallo's which I am very fond of, go down to 1.6ohm due to their CDT tweeter... it is not unusual for certain types of "exotic" drivers... - which can indeed be very revealing... so many of these types of speaker are indeed very revealing... but that isn't the reason the amps sound different on them!
 

JSmith

Master Contributor
Joined
Feb 8, 2021
Messages
5,153
Likes
13,214
Location
Algol Perseus
And around here the mods are pretty efficient.
ASR is very much a breath of fresh air compared to many other audio forums and I think most of us prefer things kept that way.

An example... a forum I would often frequent many years ago had a resident "golden ears" that members would actually send their amps (often expensive too) to for a listening "test", after which time the member would post a few pages of impressions and comments about the amp for the member who sent it and the forum in general. They were only listened to... with comparison comments, yet they were not compared AB let alone blind. In fact the amps he would "compare" the current amp to would not even be in his possession anymore. To me it was a completely useless exercise... and strangely many of the "tested" amps would end up in the FS section not long at high prices for 2nd hand after, with a referral to the "test" post as evidence of it being a fine amp. In fact many members actually relied on this list to make purchasing decisions... and the forum then had sponsors. To make it worse, the person was actually very pleasant and really believed in what they were doing after years of raking amps this way, thus due to this "experience" felt he was in a position to conduct such single "tests".

Now if anyone asked are any of these amps AB compared at least, or any kind of actual testing done on them, or pointing out the folly of same... their posts would be removed and a note sent asking the member to refrain from posting such things as it was against the spirit of the forum, regardless of seniority or join date and regardless of time consuming contributions to the forum itself. ASR is basically the bizarro world opposite of forums like that... a decent place where those attitudes are in the minority based on factual information and real published data.


JSmith
 
Top Bottom