What to do about the ABX test?

[Galliardist]

Suppose a difference can be found in the upper treble (so young ears only) or requires a very good headphone/speakers and or training and 10 people take 'the test' and 9 fail where there is 1 who was trained, had the right gear and could reliably detect the differencfe. Does that one test prove audibility or are the other 9 truly showing no audibility ?

It proves audibility in the circumstances given.
Sorry for stating the obvious.

 

[solderdude]

It proves audibility in the circumstances given.
Sorry for stating the obvious.

That's exactly the point. It only says something about the individual that does the test, not about audibility per se.

 

[Galliardist]

This probably isn't the right thread for me, because I find the endless back-and-forth about electronics to be really tiresome in general. Based on all my own testing and reading on this forum, I'm very convinced that if you take the average "decent" piece of electronics(whether it be a DAC, amplifier, whatever), you're just not going to hear any significant differences, especially on speakers. A lot of tests done to "prove" you can hear differences are done on headphones which often have much lower distortion and reduce the environmental noise as well. I'll define "decent" as SINAD of 80dB and all other tests having a "passing" result as called out in a typical ASR review. I realize that's not a very precise definition by this forum's standards, but that's sort of my point.

When we spend so much effort on arguing about miniscule differences it convinces new people that they NEED to care about whether their AVR has a SINAD of 80dB or 100dB and they should spend significant amounts of their time, money, and effort finding a "solution" to this "problem".... when their speakers usually have at least some frequency response issues and 10-100x more distortion baked in even at low levels.

I am NOT saying there are NO DIFFERENCES in a literal academic sense. I am saying the correct advice to 99.99% of the people looking for recommendations is "get a basic level of performance from your electronics and then STOP READING ABOUT IT. Instead, focus on speakers/room treatment/room correction/bass management, any of which is far more important."

No, this is EXACTLY the right thread for you. The central myth that underlies the others is that everybody's hearing is different and so everybody needs a different system and has to spend endless amounts of time dissecting and improving every single aspect of their equipment in ever weirder and more wonderful ways. One of the central subjective tenets is "everything's important", is it not.

We need to shoot that one down in flames.

Most people would be happy with most systems if we didn't have that dumb idea playing for ever. It still exists here, as you point out, in the idea that better SINAD is vital to getting the one true sound, when it really reflects on the quality of the engineering above a relatively low threshold. In fact, the most important part of this argument is what you are doing in your head, knowingly or not.

What's important to me right now is that I am listening to a glorious interpretation of Handel's harpsichord music and I can follow the voices, parts and timing of the playing easily. I was able to find it easily and play it without problems. To tell the truth, if you have a good enough sound to really appreciate music without worrying about either the equipment any further or the techniques any further, why worry?

There's probably a truth that all those people listening to music on their phones and their Bluetooth mono speakers are enjoying their music way more than we audiophiles are. A sense of proportion, that's what we all need.

 

[Galliardist]

That's exactly the point. It only says something about the individual that does the test, not about audibility per se.

Only if that one individual is unique and has the best hearing of all people alive, which is statistically very unlikely. But we need a larger and well conducted test to know what percentage of the wider population that one person represents.

If we need to know that enough to make the test worth conducting, of course.

 

[Sokel]

One aspect of this is that we are arguing that being fooled in sighted tests is actually the general case, and that is a hard thing to make stick.

More over,problem is the main arguments rise when expensive devices come to play,some low high-end (=10K$) and up.
The urge to beat cheaper devices is a lot less even thought there's always cases of blind fanbois who would even defend a broken device or take a single measurement to a single level to raise as a flag even though the rest of them are not so nice.
What is needed is honesty,no one would ask anyone to test his preference or his deliberate taste (even in "flawed" devices) or most of everything the aesthetics or the depth of his pockets!

Not everyone is a fool,I'm all for measurements for example but no ugly (to me) device would sit on my rack.And I defend that any day.But beautiful does not sound better,I know.

 

[solderdude]

Only if that one individual is unique and has the best hearing of all people alive, which is statistically very unlikely. But we need a larger and well conducted test to know what percentage of the wider population that one person represents.

If we need to know that enough to make the test worth conducting, of course.

No the test result is only valid for that individual under his circumstances, assuming the test was done properly. It says nothing about other people nor about audibility in general.

 

[voodooless]

That's exactly the point. It only says something about the individual that does the test, not about audibility per se.

In the general context that this argument is used in such a discussion, this is perfectly fine: they make a claim that they can hear a difference, and we ask for proof of that. Only the result of the individual is relevant for this.

In any case, this topic should be about communications skills, not ABX tests.

 

[Cote Dazur]

What is a more effective way to engage people who don’t understand things about what can and cannot be heard without chiming in over and over “hey, do an ABX test or it didn’t happen”?

I believe one of the issue for many is not the test itself, even though it is not simple or easy to execute, as many have stated here.
One big chalenge is understanding first that our earring is flawed by the ability our brain has to filter and interpret the sound, making it possible for suggestions to totally alter what we really hear. So suggesting an ABX might be more relevant if it comes with reading links explaining scientifically how differences/improvement we believe hearing may or may not be there at all.
The other challenge when suggesting ABX is patience, people need time to grasp the errors of their ways, undo years of believing in voodoo spending money on snake oil, but we are all, here, proof it can be done.

 

[Ken1951]

Those visitors "Believe". The probability of changing this approaches zero. ABX or any other testing will be immediately dismissed. ASR folks don't listen to music, don't "Believe" their ears, only look at measurements, only test at 1K, etc, etc, etc, ad nauseam. IMHO about 99.99% of these visitors are here only to preach and receive their Gold Star of Banishment From ASR. But maybe I'm just too old and cynical.

 

[Multicore]

Regardless what other strategies you might employ, you can also communicate values.

The kind of taunting that "ABX or it didn't happen" represents is a basic an ancient feature of schismogenesis, the creation and strengthening of the in-group vs.the others, or the us team vs. the them team. Others have described the same thing under different names but I like schismogenesis (from Bateson): the creation of schism.

Does ASR want to be such an identity group? Not me. I'm not much of a joiner and never was. And if that's the general feeling then it's up to the more senior ASRers to gently discourage those that take it upon themselves to defend the valor and honor of the ASR clan and explain that making a show of doing so doesn't elevate them.

 

[pkane]

Seems increased commentary in recent weeks about ABX tests. Much of it stemming from people who come to ASR to set us straight about trusting our ears. I do agree with some who have said that calls for ABX or it didn’t happen have become almost like a club to beat people over the head with, and nearly cultish in how some new posters have the call rain down upon them. Not that I haven’t been guilty of it myself.

Some comments by @restorer-john have caused me to think about this situation. We stand little chance of convincing, or engaging in meaningful discussion with people with this approach. Like restorer-john I think there is a lot more talk of it than participation in or use of ABX listening tests among most posters. For most audiophiles it is impractical for most situations.

Some who don’t like ABX tests complain they are stressful. Only if you feel challenged by it or think you’ll suffer loss of face. After you have done it a couple or three times it isn’t stressful. It is major league TEDIOUS and BORING. Most of us do them with Foobar ABX or similar software. That isn’t very useful for amps and not at all for speakers.

So what is a next best alternative? What is a friendlier way to get the point across? How do regular ASR members pick their gear?

Blind tests are the best most discriminating method. I find I can detect with 100% reliability some very small differences when using two segments of 5 seconds or less and rapid switching. OTOH, some of those I score 50/50 if segments are 15 or 30 seconds long. I have found anything I only hear using the very short segments which both can fit inside my Echoic memory are so small they have zero relevance to normal music listening. So on one hand if you cannot hear something using short rapid switching listening tests it is a pretty sure bet you cannot hear it. On the other if the difference isn’t large enough to hear with 30 second segments it isn’t big enough to matter for music listening.

I believe the #1 thing to emphasize with any comparative listening is you must match levels precisely. Set a comfortable listening level and measure voltage of test tones at speaker terminals so each component matches within 1%. You cannot do any useful listening comparisons without this step. This one thing even in sighted listening can cause people to experience the disappearance or large reduction in differences they thought they were hearing.

The #2 thing to make clear is that fairly small deviations in frequency response are audible. So checking that might eliminate any need to go further for differences you hear. There are some simple ways to test this.

So what other things can we do or that some of you do that is useful? What is a more effective way to engage people who don’t understand things about what can and cannot be heard without chiming in over and over “hey, do an ABX test or it didn’t happen”?

ABX is a relatively hard test because one is required to identify the device. Most of us are not trained to do this well, and untrained audiophiles are probably the worst because they are trained ‘wrong’.

A simpler blind AB preference test is less stressful because every self-respecting audiophile has done dozens, if not hundreds of sighted AB comparisons and is very familiar with the process. The setup and statistical analysis would need to be a bit different to determine audibility, but obviously, in order to have a valid preference the two configurations must, at the very least, have audible differences.

 

[LightninBoy]

I also cringe a bit when the ABX challenges come out. The ensuing back and forth always reminds me of this scene (but without the happy ending)

And then there was a thread sometime back when the forum attempted to define some guidelines around blind testing. What an embarrassing cluster that was.

But the bottom line is this, the poster making the claims of "obvious differences" will not be believed unless the methodology is documented, explained, and recorded to a standard that few of us would be willing to subject ourselves to.

So I suggest this strategy. When responding to these claims, the goal of the responses should not be to tell the poster how to convince the forum. Instead the goal should be to educate the poster on how easy it is to fool themselves with typically sighted comparisons. And provide some *basic* steps they could take to make their comparisons more useful to *them*. Sure, few will follow the advice, but at least those who do won't have the false expectation of proving it to the forum.

 

[voodooless]

Regardless what other strategies you might employ, you can also communicate values.

The kind of taunting that "ABX or it didn't happen" represents is a basic an ancient feature of schismogenesis, the creation and strengthening of the in-group vs.the others, or the us team vs. the them team. Others have described the same thing under different names but I like schismogenesis (from Bateson): the creation of schism.

Does ASR want to be such an identity group? Not me. I'm not much of a joiner and never was. And if that's the general feeling then it's up to the more senior ASRers to gently discourage those that take it upon themselves to defend the valor and honor of the ASR clan and explain that making a show of doing so doesn't elevate them.

That may be a noble effort, but ultimately will not bring any new knowledge to users. So what actually should be done to communicate in a way that is understandable and engaging?

 

[voodooless]

So I suggest this strategy. When responding to these claims, the goal of the responses should not be to tell the poster how to convince the forum. Instead the goal should be to educate the poster on how easy it is to fool themselves with typically sighted comparisons. And provide some *basic* steps they could take to make their comparisons more useful to *them*. Sure, few will follow the advice, but at least those who do won't have the false expectation of proving it to the forum.

The question then is what the goal is? Do you only want to reach those few? What about the rest? Would it not be more effective to reach those?

In general, I think reaching “the few” is never the issue.

 

[pma]

So what actually should be done to communicate in a way that is understandable and engaging?

IME personal meetings are much more useful and revealing then social network discussions. Social network discussions have limited impact.

 

[voodooless]

IME personal meetings are much more useful and revealing then social network discussions. Social network discussions have limited impact.

Fully agree, but hardly a practical solution, is it?

Guy: “In hear differences in speaker cables!”
ASR: “Just you wait, we’ll come and visit to set you straight!”

 

[LightninBoy]

The question then is what the goal is? Do you only want to reach those few? What about the rest? Would it not be more effective to reach those?

In general, I think reaching “the few” is never the issue.

The goal is to reach as many as you can, which I believe will end up being "a few" whether you gently guide them or brow beat them with rigid ABX methodologies. But the former "few" will be greater than the latter "few".

 

[pma]

Fully agree, but hardly a practical solution, is it?

It used to be a practical solution. Since 2002 till about 2015, we had numerous meetings of audio fans here usually 2-3 days at some hotel, we hired a conference room for listening sessions. However, with the expansion of social networks, this activity ceased, unfortunately. Now we have what we have. Social networks open for everyone, with all its pros and cons.

 

[pkane]

The question then is what the goal is? Do you only want to reach those few? What about the rest? Would it not be more effective to reach those?

In general, I think reaching “the few” is never the issue.

Part of the problem is the desire for immediate satisfaction of having someone convinced in a few posts or a quick and easy win in an online argument. That doesn’t work. A slow and constant drip of information, suggestions, and a thoughtful, civilized discussion, slow prodding in the right direction is much more likely to convince someone in the end than a dismissive sarcastic remark. But it is just not the way online communities work, unfortunately.

 

[AdamG]

Seems increased commentary in recent weeks about ABX tests. Much of it stemming from people who come to ASR to set us straight about trusting our ears. I do agree with some who have said that calls for ABX or it didn’t happen have become almost like a club to beat people over the head with, and nearly cultish in how some new posters have the call rain down upon them. Not that I haven’t been guilty of it myself.

Some comments by @restorer-john have caused me to think about this situation. We stand little chance of convincing, or engaging in meaningful discussion with people with this approach. Like restorer-john I think there is a lot more talk of it than participation in or use of ABX listening tests among most posters. For most audiophiles it is impractical for most situations.

Some who don’t like ABX tests complain they are stressful. Only if you feel challenged by it or think you’ll suffer loss of face. After you have done it a couple or three times it isn’t stressful. It is major league TEDIOUS and BORING. Most of us do them with Foobar ABX or similar software. That isn’t very useful for amps and not at all for speakers.

So what is a next best alternative? What is a friendlier way to get the point across? How do regular ASR members pick their gear?

Blind tests are the best most discriminating method. I find I can detect with 100% reliability some very small differences when using two segments of 5 seconds or less and rapid switching. OTOH, some of those I score 50/50 if segments are 15 or 30 seconds long. I have found anything I only hear using the very short segments which both can fit inside my Echoic memory are so small they have zero relevance to normal music listening. So on one hand if you cannot hear something using short rapid switching listening tests it is a pretty sure bet you cannot hear it. On the other if the difference isn’t large enough to hear with 30 second segments it isn’t big enough to matter for music listening.

I believe the #1 thing to emphasize with any comparative listening is you must match levels precisely. Set a comfortable listening level and measure voltage of test tones at speaker terminals so each component matches within 1%. You cannot do any useful listening comparisons without this step. This one thing even in sighted listening can cause people to experience the disappearance or large reduction in differences they thought they were hearing.

The #2 thing to make clear is that fairly small deviations in frequency response are audible. So checking that might eliminate any need to go further for differences you hear. There are some simple ways to test this.

So what other things can we do or that some of you do that is useful? What is a more effective way to engage people who don’t understand things about what can and cannot be heard without chiming in over and over “hey, do an ABX test or it didn’t happen”?

Great topic to discuss. Thanks for starting this thread @Blumlein 88. As Moderators we are left with picking up the pieces and trying to determine the item (new member) is worth gluing back together. Maybe my word choices here could be better but understand that we are the final arbitrations. Are they all trolls? You would be surprised to see just how many Reports we get from these new visitors. We get pressured by the mob mentality just like anyone else. We have been directed by Amir to be more lenient and to give more rope to new members. We try as much as possible to keep this in mind and give them multiple opportunities. We kindly ask that you also keep this in mind as well. Eventually we get frustrated by the work created by mounting reports that this guy is a Troll and is just Trolling ASR…Striking a balance of competing interests. Sometimes it may seem like the Moderators are not doing their jobs. How much harm can it cause to let the new member have a few more chances ? Try to temper your instinct to hit the Report button and give them the opportunity to absorb the science. Changing a person’s perception is a very difficult endeavor.

Asking for a more receptive atmosphere would be a positive. However, we also realize that many of our Senior Members have become jaded by years and years of experiencing the same arguments over and over. So, exactly how do we change the culture here? We are a unique segment of the Audio world. How do we balance these opposing views of Science and Objectivity with Feelings and Subjective Emotional perceptions? Isn’t that the fight the Scientific Community has been fighting for like ever ?

We didn’t start this fight and I doubt we can ever win it either. But,,,talking about it and offering possible solutions, options and different approaches is a valuable exercise. Focused reflection of our individual actions and as a group can create slivers of opportunity to change how we treat and respond to our newcomers. Some of which end up becoming incredibly important and valuable contributing members of ASR.

We are swiftly closing in on 40,000 Members and 1.4 million posts. Let’s keep talking about this and know that we are reading every single post in this thread and learning with you. This is your forum more than it is ours. It is you and your fellow ASR members who really control the Rudder of this Ship. We ultimately go where you steer us. We follow where you lead. These are just my personal thoughts and wish to encourage introspection and reflective thinking.