• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

What to do about the ABX test?

Sancus

Major Contributor
Forum Donor
Joined
Nov 30, 2018
Messages
2,923
Likes
7,616
Location
Canada
So what is a next best alternative? What is a friendlier way to get the point across? How do regular ASR members pick their gear?
This probably isn't the right thread for me, because I find the endless back-and-forth about electronics to be really tiresome in general. Based on all my own testing and reading on this forum, I'm very convinced that if you take the average "decent" piece of electronics(whether it be a DAC, amplifier, whatever), you're just not going to hear any significant differences, especially on speakers. A lot of tests done to "prove" you can hear differences are done on headphones which often have much lower distortion and reduce the environmental noise as well. I'll define "decent" as SINAD of 80dB and all other tests having a "passing" result as called out in a typical ASR review. I realize that's not a very precise definition by this forum's standards, but that's sort of my point.

When we spend so much effort on arguing about miniscule differences it convinces new people that they NEED to care about whether their AVR has a SINAD of 80dB or 100dB and they should spend significant amounts of their time, money, and effort finding a "solution" to this "problem".... when their speakers usually have at least some frequency response issues and 10-100x more distortion baked in even at low levels.

I am NOT saying there are NO DIFFERENCES in a literal academic sense. I am saying the correct advice to 99.99% of the people looking for recommendations is "get a basic level of performance from your electronics and then STOP READING ABOUT IT. Instead, focus on speakers/room treatment/room correction/bass management, any of which is far more important."
 

sajgre

Active Member
Joined
Mar 6, 2021
Messages
133
Likes
232
Imo the hardest thing to explain to newcomers is that anecdotal experiences are worthless. You are not listening bass of this speaker, you are listening speaker-room interaction which means 87 variables that you can't compare, your current mood is listening with you... mids are reccesed? Maybe you are used to speakers without much bass... Speaker is analitical? Maybe distorted midbass feels warm to you. Is bass really powerful or are speakers just playing loud, is bass thin or is ls60 to small for you to believe it can play pretty low....

I'm also guilty of forgetting this, I love to read how two speaker models compare on someone's desk. Which is worthless for me because of 100 variable factors.

So how do we tell people that YES, we believe that you easily hear a difference between USB cables but please believe us that it might be in your head and we have all been there? How do we politely tell them that THEY are claiming things that are against science and so they should prove it? I have no idea :)
 

voodooless

Grand Contributor
Forum Donor
Joined
Jun 16, 2020
Messages
10,229
Likes
17,811
Location
Netherlands
It is about different approaches that might result in more useful engagement by those who doubt the measurement and blind testing approach.
Well, make it broader: how to converse with those who doubt (audio) science (and often logic) in general. That is a really hard nut to crack.

My (probably flawed) approach in general is to point out logical fallacies, and ask deeper questions about their way of thinking. Often (if possible) without directly addressing the opposing point of view. Usually, though, this doesn't really work, but leads to people ignoring the question, repeating the same thing over and over, or starting to throw ad homonyms. Seems like even asking questions is forbidden. Me throwing in the occasional sarcasm probably doesn't help either, but I can't help myself :facepalm:
 

Astoneroad

Addicted to Fun and Learning
Forum Donor
Joined
Feb 16, 2022
Messages
999
Likes
2,046
Location
a Cave in the desert
Well, make it broader: how to converse with those who doubt (audio) science (and often logic) in general. That is a really hard nut to crack.

My (probably flawed) approach in general is to point out logical fallacies, and ask deeper questions about their way of thinking. Often (if possible) without directly addressing the opposing point of view. Usually, though, this doesn't really work, but leads to people ignoring the question, repeating the same thing over and over, or starting to throw ad homonyms. Seems like even asking questions is forbidden. Me throwing in the occasional sarcasm probably doesn't help either, but I can't help myself :facepalm:
In Pirsig's book, "Zen and the Art of Motorcycle Maintenance" he discusses the concept and question of "quality". Whenever I've attempted to bring this up as a question in conversation, it has (100% of the time) not ended well, usually in an impasse, sometimes ending a "friendship". This thread has all the earmarks of my experience. I have no insights, just reflection and additional questions as to why it creates such an emotional response and challenge to otherwise thoughtful minds. I'll confess that after seeing this response repeatedly, and forecasting the same result, I've used this to exit relationships. Instead of the usual, "It's not you, it's me"... I inquire about the nature of "quality". At that point they also come to the same conclusion... that it's me and not them... lol. :cool:
 
Last edited:

FINFET

Active Member
Forum Donor
Joined
Jul 27, 2022
Messages
113
Likes
202
One thing I can think of that can be potentially more effective than ABX is to let them choose the better sound between A and B blindly multiple times. I don’t know if there’s a name for this test.

The reason is because people so strongly believe there’s a difference, that you can’t simply change their mind about this. Then let’s say, there’s a difference, but are you seeking difference? No, audiophiles are seeking “better sound”. So just choose the better sound then! But only with your eyes covered, and with at least 10 or 16 samples in each test to see if the feeling is consistent and stable.

This is by far the only method I used that actually changed one audiophile’s mind about the hobby. People so strongly believe that more expensive devices sound better (and I don’t blame them, it’s this evil industry keep educating them for decades), and they have to actually choose the cheaper producing “better sound” to believe and move their first step towards the correct direction.
 
Last edited:
OP
Blumlein 88

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,524
Likes
37,057
In Pirsig's book, "Zen and the Art of Motorcycle Maintenance" he discusses the concept and question of "quality". Whenever I've attempted to bring this up as a question in conversation, it has (100% of the time) not ended well, usually in an impasse, sometimes ending a "friendship". This thread has all the earmarks of my experience. I have no insights, just reflection and additional questions as to why it creates such an emotional response and challenge to otherwise thoughtful minds.
I've done the same thing without it being problematic. Sometimes it didn't make the difference I hoped for. I remember Pirsig describing how he once asked an Indian "what kind of dog is that?" to which the Indian replied, "That....is a good dog". A description of a quality of the dog.
 

Astoneroad

Addicted to Fun and Learning
Forum Donor
Joined
Feb 16, 2022
Messages
999
Likes
2,046
Location
a Cave in the desert
I've done the same thing without it being problematic. Sometimes it didn't make the difference I hoped for. I remember Pirsig describing how he once asked an Indian "what kind of dog is that?" to which the Indian replied, "That....is a good dog". A description of a quality of the dog.
Yes, but that one is easy... since there is no other kind of dog... other than a Good one!! ;)
 

voodooless

Grand Contributor
Forum Donor
Joined
Jun 16, 2020
Messages
10,229
Likes
17,811
Location
Netherlands
One thing I can think of that can be potentially more effective than forcing people to do ABX, is to let them choose the better sound between A and B blindly, but do that many times. I don’t know if there’s a name for this test.
I think the bigger issue here is that people don't like to be told how to do things. They heard the difference: "Who are you to tell me I'm wrong?". Usually, there is a blatant refusal to do any more testing or acknowledge in any way that what they have done is flawed. To have them do any additional testing whatsoever is already a win.
 

FINFET

Active Member
Forum Donor
Joined
Jul 27, 2022
Messages
113
Likes
202
I think the bigger issue here is that people don't like to be told how to do things. They heard the difference: "Who are you to tell me I'm wrong?". Usually, there is a blatant refusal to do any more testing or acknowledge in any way that what they have done is flawed. To have them do any additional testing whatsoever is already a win.
Agree. Op's idea to provide a friendlier extra test can work on some less stubborn minds. I think the which-is-better test can be less offensive and less intense or frustrated to do. People may choose confidently in such a test while in ABX they will compare again and again try so hard to hear the difference that soon it becomes tedious and boring. It is also a friendly approach like "you said you hear a difference, and I believe you. Just consistently educate me about which one is better"
 

voodooless

Grand Contributor
Forum Donor
Joined
Jun 16, 2020
Messages
10,229
Likes
17,811
Location
Netherlands
Maybe this has some good tips:


And no, I don't think those people aren't exactly in a cult per se, though I think some might exhibit several of those traits. But I do think many of the points made can still be applied very well here.
 

restorer-john

Grand Contributor
Joined
Mar 1, 2018
Messages
12,588
Likes
38,288
Location
Gold Coast, Queensland, Australia
One should realize that taking an AB(X) test is only for the one taking it. Not admissible as evidence.

Exactly. I made this point only a few days ago. It means absolutely nothing to anyone else, except the person taking the test. It's not the equivalent of some "preference curve"...
 

Sokel

Master Contributor
Joined
Sep 8, 2021
Messages
5,850
Likes
5,790
Exactly. I made this point only a few days ago. It means absolutely nothing to anyone else, except the person taking the test. It's not the equivalent of some "preference curve"...
Except if that person has some kind of established (proved ((how?)) or not) authority.
Followers will take it from there.
 

restorer-john

Grand Contributor
Joined
Mar 1, 2018
Messages
12,588
Likes
38,288
Location
Gold Coast, Queensland, Australia
I have post the most number of ABX tests here and I don't do anything of the kind. Nor would your scheme work because foobar ABX randomizes what you are listening to so prior knowledge doesn't help you. Why don't you try replicating the tests I have passed and see how far you get with cheating.

Just how many actual individual and unique ABX tests have you posted here on ASR? Seriously, not the same ancient key jangling sample from a decade ago...

Amir, the files are local. You can analyse the files to any extent you want prior to running those files through a Foobar ABX. You pick two files, and ABX them. That's how it works. We all do it and have done for years. Most of us do it honestly, many don't.

It's easy to listen for aberrations, issues, noise, hiss, clicks, level differences, distortions, balance differences etc and key yourself into what you should listen for to accurately determine the differences and achieve a flawless score. What does that achieve? Absolutely nothing.
 

Shadrach

Addicted to Fun and Learning
Joined
Feb 24, 2019
Messages
664
Likes
948
Exactly. I made this point only a few days ago. It means absolutely nothing to anyone else, except the person taking the test. It's not the equivalent of some "preference curve"...
Not strictly true this.
One ABX test may not prove anything to anyone bar the subject but an aggregate of near identical ABX tests does mean something to others.
if 100,000 people take a test and no difference was the outcome then that does mean something and what that means is the majority could not percieve a difference under the test conditions. One can argue until one is blue in the face that there is a difference but if the majority didn't identify it then for the majority a difference didn't exist.
 

Rednaxela

Major Contributor
Joined
Mar 30, 2022
Messages
2,055
Likes
2,676
Location
NL
The whole ABX thing isn't really about ABX testing at all. There would be no need to suggest such a test if people would actually accept established science. So really, I think you should be trying to solve a different problem.
Nail on the head.

When I feel the urge to ask for an ABX test it is ultimately an expression of a desire to defend progress.

Armchair perhaps but I'm hardly ever interested in the actual test or its results.
 
OP
Blumlein 88

Blumlein 88

Grand Contributor
Forum Donor
Joined
Feb 23, 2016
Messages
20,524
Likes
37,057
Just how many actual individual and unique ABX tests have you posted here on ASR? Seriously, not the same ancient key jangling sample from a decade ago...
I once posted an updated key jangling file. I found I could ABX it with reasonable ease using Arnie's files. When I did my own downsampling however with a better software I could not. I recall Amir either couldn't detect it or said it was far more difficult than Arnie's conversion which he also could correctly pick. Congruence like that is more than nothing, but is not definitive proof or equal to correctly done serious tests of groups of listeners.
 

solderdude

Grand Contributor
Joined
Jul 21, 2018
Messages
15,891
Likes
35,912
Location
The Neitherlands
Not strictly true this.
One ABX test may not prove anything to anyone bar the subject but an aggregate of near identical ABX tests does mean something to others.
if 100,000 people take a test and no difference was the outcome then that does mean something and what that means is the majority could not percieve a difference under the test conditions. One can argue until one is blue in the face that there is a difference but if the majority didn't identify it then for the majority a difference didn't exist.

Suppose a difference can be found in the upper treble (so young ears only) or requires a very good headphone/speakers and or training and 10 people take 'the test' and 9 fail where there is 1 who was trained, had the right gear and could reliably detect the difference. Does that one test prove audibility or are the other 9 truly showing no audibility ?
 

voodooless

Grand Contributor
Forum Donor
Joined
Jun 16, 2020
Messages
10,229
Likes
17,811
Location
Netherlands
Exactly. I made this point only a few days ago. It means absolutely nothing to anyone else, except the person taking the test. It's not the equivalent of some "preference curve"...
That is not totally correct. If the statement is made: "nobody can hear the difference", and an ABX test shows somebody can (with the caveat that there is a statistical significance at play here that needs to be "good enough" (open for debate)), this is valuable information and might warrant further investigation.
 

Galliardist

Major Contributor
Joined
Jun 26, 2021
Messages
2,558
Likes
3,273
Location
Sydney. NSW, Australia
I think one thing we definitely need is a pinned article on what happens in the brain when we listen to music. I see that Amir has an article about preference already that we should also refer to. One aspect of this is that we are arguing that being fooled in sighted tests is actually the general case, and that is a hard thing to make stick.
 

FrantzM

Major Contributor
Forum Donor
Joined
Mar 12, 2016
Messages
4,339
Likes
7,739
This probably isn't the right thread for me, because I find the endless back-and-forth about electronics to be really tiresome in general. Based on all my own testing and reading on this forum, I'm very convinced that if you take the average "decent" piece of electronics(whether it be a DAC, amplifier, whatever), you're just not going to hear any significant differences, especially on speakers. A lot of tests done to "prove" you can hear differences are done on headphones which often have much lower distortion and reduce the environmental noise as well. I'll define "decent" as SINAD of 80dB and all other tests having a "passing" result as called out in a typical ASR review. I realize that's not a very precise definition by this forum's standards, but that's sort of my point.

When we spend so much effort on arguing about miniscule differences it convinces new people that they NEED to care about whether their AVR has a SINAD of 80dB or 100dB and they should spend significant amounts of their time, money, and effort finding a "solution" to this "problem".... when their speakers usually have at least some frequency response issues and 10-100x more distortion baked in even at low levels.

I am NOT saying there are NO DIFFERENCES in a literal academic sense. I am saying the correct advice to 99.99% of the people looking for recommendations is "get a basic level of performance from your electronics and then STOP READING ABOUT IT. Instead, focus on speakers/room treatment/room correction/bass management, any of which is far more important."
emphasis is mine. I was writing a lengthy reply when I saw this..soooo...

+1.25 to the power 10,000

Since 1 raised to any power remains 1, I decided 1.25:
+ 1.25 to the power 10,000
:D


Peace.
 
Top Bottom