What to do about the ABX test?

[xnor]

I have done single blinded tests at home, there is A and B and no X in the methodology.
In each listening session A or B may be either component and the user selects how long to listen and when to switch.
At the end, provides a preference if any and a description of the differences.

- Rich

Sure there is an X. There's also a Y. What you describe is a test with X and Y.
Those get randomly assigned either A or B at the start of each round.
ABX tools may also offer a Y. Y is simply the other option. In those tools, you can also choose only to listen to X and Y.

The point of these tools/protocols should not be to make you suffer or to force you into something (which is a common invalid excuse that golden ear audiophiles make), but to help you detect even the tiniest of differences in a way that eliminates the possibility of you tricking yourself.

Whether you make your choice because you hear a difference, have a preference or experience a tingling sensation in your pinky does not matter.
The point is that, statistically, if your choices align with random chance then we're inclined to say that you cannot hear a difference.

 

[RichB]

From the OP:

So what other things can we do or that some of you do that is useful? What is a more effective way to engage people who don’t understand things about what can and cannot be heard without chiming in over and over “hey, do an ABX test or it didn’t happen”?

Sure there is an X. There's also a Y. What you describe is a test with X and Y.
Those get randomly assigned either A or B at the start of each round.
ABX tools may also offer a Y. Y is simply the other option. In those tools, you can also choose only to listen to X and Y.

The point of these tools/protocols should not be to make you suffer or to force you into something (which is a common invalid excuse that golden ear audiophiles make), but to help you detect even the tiniest of differences in a way that eliminates the possibility of you tricking yourself.

Whether you make your choice because you hear a difference, have a preference or experience a tingling sensation in your pinky does not matter.
The point is that, statistically, if your choices align with random chance then we're inclined to say that you cannot hear a difference.

Level matched A/B or Y/X comparisons can be a useful tool for people at home to try to evaluate and compare components.
Level matching using a voltage is better than using SPL in room.

SBTs (Single Blind Tests) are a good method at home since these component tests often requires manual intervention.
Encouragement is to suggest and support rigor, discouragement is “hey, do an ABX test or it didn’t happen”.

Let's say a user buys a set of $2,000 speaker cables. Speaker cables can absolutely change the response of a speakers.
Especially, ones with dubious designs. A rigorous experiment is to setup a SBT using two-amp channels from a split source.
Another person could switch the speaker cables out of view (SBT) upon request, such the listener does not know which cables are attached.

Possible outcomes could be:

- The listener is unable to reliably determine the cables in use
- The listener reliably determined the cables and picks the cheaper cable
- The listener reliably determined the cables and picks the more expensive cable

This gets interesting if the product in question is still within the return window.
The user may decide that they don't really sound much different but loves how they look on their new cable lifters.

I encourage such experiments (with rigor) and reading impressions. It is clear what these are listening sessions. They are not proof that one thing is superior to another on an objective basis.

I think many of us have been on the upgrade train, searching for that last bit of (affordable) performance, so have a visceral response to impression posts that threaten our wallets (again) , but this is not always productive nor helpful.

- Rich

 

[lashto]

That is what this thread is about. Different strategies. It is not an anti-ABX thread or one questioning its veracity. It is about different approaches that might result in more useful engagement by those who doubt the measurement and blind testing approach.

Here's a more concrete suggestion..

ABX/DBT with two different devices is by far the most complex & hard. We cannot really expect/request an 'amateur' to do one. Or to do it right.

It is however easier to compare recordings with some ABX plugin. @amirm can use one latest-and-greatest ADC and record the sound of all tested devices. And members will try the ABX fun.
One of those super revealing recrdings could be used: Arnie's keys-jingle or audience applause or...
Still not exactly easy but should be much easier/simpler. And I'm putting a lot more work in other people's laps

Otherwise, I only have the old "keep an open mind" .. wish that one was not so over-used and thoroughly-abused.

 

[xnor]

Level matching using a voltage is better than using a SPL in room.

I've also seen people fall into the trap of comparing amps with different output impedances and matching the unloaded output voltages.

Imo, in such situations its more practical to just measure the output impedances, look at the headphone/speaker impedance curves and do a quick calculation of the max frequency response deviations. Because why bother with the physical setup when you can tell straight from the numbers that amp B will result in a, let's say, 3 dB deviation?

If one still thinks that amp B has other sonic "qualities" and wants to test that, then I'd create an EQ curve to eliminate the FR differences.
So instead of matching loaded output voltages at a single frequency you match across all audible frequencies.

Let's say a user buys a set of $2,000 speaker cables. Speaker cables can absolutely change the response of a speakers.
Especially, ones with dubious designs. A rigorous experiment is to setup a SBT using two-amp channels from a split source.

Here I'd again look at the specs first and run the numbers. You need to measure the voltage drop/resistance for level matching anyway.
A cable can be modeled as a passive filter and its effects on FR can be calculated.

Good manufacturers will provide the specs, so you don't even need the equipment to measure them yourself.
The thing is this: once you understand any of this then you also understand that such cables are a scam / waste of money.

I encourage such experiments (with rigor) and reading impressions. It is clear what these are listening sessions. They are not proof that one thing is superior to another on an objective basis.

While I agree that people should do blind tests I think that such tests are mainly useful for the person doing the test, because it tests their setup and hearing limits.
Objective measurement data is more universally useful.

 

[RichB]

I've also seen people fall into the trap of comparing amps with different output impedances and matching the unloaded output voltages.

Imo, in such situations its more practical to just measure the output impedances, look at the headphone/speaker impedance curves and do a quick calculation of the max frequency response deviations. Because why bother with the physical setup when you can tell straight from the numbers that amp B will result in a, let's say, 3 dB deviation?

If one still thinks that amp B has other sonic "qualities" and wants to test that, then I'd create an EQ curve to eliminate the FR differences.
So instead of matching loaded output voltages at a single frequency you match across all audible frequencies.

I've always matched amps levels using 200Hz, 500Hz, 1000Hz, and 2000Hz when driving speakers but settled on 1kHz.
That will be included in future posts.

Certainly, output impedance will provide some insight into maximum deviations but that of course will vary based on the actual load.
A well designed A/B comparison can help the user determine difference detection and preference.
People who love their tube preamp may not like a Benchmark SS LA4.
Still their observations.

For SS amps, output impedance is not sufficient to characterize an amplifier driving loads.
Here are two amp measurements into load with frequency variation based on the output impedance.

AHB2: https://www.stereophile.com/content/benchmark-media-systems-ahb2-power-amplifier-measurements

The output impedance, including a 6' speaker cable, was a low 0.09 ohm at 20Hz and 1kHz, rising slightly to 0.22 ohm at 20kHz. As a result, the modification of the Benchmark amplifier's frequency response due to the interaction between this impedance and that of our standard simulated loudspeaker was just ±0.1dB (fig.1, gray trace).

Parasound: https://www.stereophile.com/content/parasound-halo-21-power-amplifier-measurements

The Parasound's output impedance was a very low 0.077 ohm at 20Hz and 1kHz, increasingly slightly to 0.1 ohm at 20kHz. (These figures include the series impedance of 6' of loudspeaker cable.) The modulation of the amplifier's frequency response, due to the Ohm's law interaction between this source impedance and the impedance of our standard simulated loudspeaker, was therefore minuscule, at ±0.1dB (fig.1, gray trace).

The AHB2 has higher output impedance but maintains .2 dB reduction (from 8 Ohms), dropping to .25 at 10kHz.
The A21+ with lower output impedance maintains .5 dB reduction (from 8 Ohms) up to 10kHz.

Is .25 versus .5 dB difference noticeable, perhaps I don't know, I also don't know if this test represents the worst case an amp driving reactive loads.
Are there measurements for distortion driving reactive loads representative of all speakers for 20Hz to 20kHz?
Personally, I look at all measurements that I can find but also know this may not be the whole story but lends a great deal of confidence.

- Rich

 

[dlaloum]

I've also seen people fall into the trap of comparing amps with different output impedances and matching the unloaded output voltages.

Imo, in such situations its more practical to just measure the output impedances, look at the headphone/speaker impedance curves and do a quick calculation of the max frequency response deviations. Because why bother with the physical setup when you can tell straight from the numbers that amp B will result in a, let's say, 3 dB deviation?

If one still thinks that amp B has other sonic "qualities" and wants to test that, then I'd create an EQ curve to eliminate the FR differences.
So instead of matching loaded output voltages at a single frequency you match across all audible frequencies.


Here I'd again look at the specs first and run the numbers. You need to measure the voltage drop/resistance for level matching anyway.
A cable can be modeled as a passive filter and its effects on FR can be calculated.

Good manufacturers will provide the specs, so you don't even need the equipment to measure them yourself.
The thing is this: once you understand any of this then you also understand that such cables are a scam / waste of money.

In a world moving ever more towards computerised measurement and RoomEQ, a slight FR variation between two amps or speaker cables becomes irrelevant, as the Room/Speaker EQ is capable of compensating for these...

With connections prior to the stage where automated EQ can be applied, it may well be more relevant - but even then only applicable to analogue.... so in today's world, we are pretty much down to turntables, or vintage sources such as radio / cassette....

It has become (mostly) irrelevant - the standard connection methods are all becoming digital... HDMI, SPDIF, Ethernet
The move to digital connection to speakers (active speakers) - is well on its way too.

We can discuss the relative merits of different breeds of horses and types of cart, but we are usually driving cars...

 

[kongwee]

All the ABX in AES are just read purpose. You can't listen for a difference is not a big deal at all. It is ok you can't differential a 8361 and M9.

 

[xnor]

For SS amps, output impedance is not sufficient to characterize an amplifier driving loads.
Here are two amp measurements into load with frequency variation based on the output impedance.

The relevant bit of information regarding my first point is the deviation of the dark gray line from each other when comparing both graphs. After all, you're not switching from 8 ohm speakers to 2 ohm speakers when trying to compare these amps, you're using the same speakers.

Regarding your point, besides some curves in these graphs looking like the were manually inserted (though this might be due to lossy compression), I don't know how they measured the output impedance.
If from 8 to 2 ohms there's a 0.75 dB drop then the (effective) output impedance is about 0.25 Ohm.
For a 0.25 dB drop it is about 0.08 Ohm.

 

[levimax]

See below some actual FR measurements I took when doing an ABX test of a tube amp with ~0.8 ohm output impedance and a SS amp with vanishing low output impedance and then the SS amp with a 0.9 ohm resistor in series. The FR behaved as expected and the 0.9 ohm resistor almost perfectly "converted" the FR of the SS amp to that of the tube amp. Even though the FR differences were measurable due to the output impedance I was not able to reliably ABX them, maybe someone could. I guess if you had pathological cases like a 5 ohm output impedance SET and a speaker with 1 ohm minimum you could get some large and hard to predict FR changes but for most amps and speakers the FR differences due to output impedance differences are small and easily compensated for if need be.

 

[RichB]

The relevant bit of information regarding my first point is the deviation of the dark gray line from each other when comparing both graphs. After all, you're not switching from 8 ohm speakers to 2 ohm speakers when trying to compare these amps, you're using the same speakers.

Regarding your point, besides some curves in these graphs looking like the were manually inserted (though this might be due to lossy compression), I don't know how they measured the output impedance.
If from 8 to 2 ohms there's a 0.75 dB drop then the (effective) output impedance is about 0.25 Ohm.
For a 0.25 dB drop it is about 0.08 Ohm.

The black lines relate to your point on output impedance.
The drop in output at 4 Ohms and 2 Ohms shows the amp with lower output impedance having a greater drop.
Since most speakers are reactive loads, the effective load placed on the amplifier can have an effect. It may even trigger limiting when the voltage.
Nothing in the output impedance measurements of the above amplifiers explains the drop in output at 2 Ohms.
In fact, the frequency variation in the black line is far less that the load.
There is also no reason to believe that a combination of load and phase could not produce greater deviations in linearity.

This is an interesting article:

Heavy Load: How Loudspeakers Torture Amplifiers​

Heavy Load: How Loudspeakers Torture Amplifiers

Why, in loudspeaker reviews, is impedance measured (assuming that the magazine in question bothers to measure anything)? Generally, for one principal reason only: to establish whether the speaker presents an "easy" or a "difficult" load to its partnering amplifier. In the design context, much...

www.stereophile.com

Output impedance influences linearity with resistive loads but EPDR can be used to get a better understanding of the load presented by a speaker.

EPDR is simply the resistive load that would give rise to the same peak device dissipation as the speaker itself. Adopting the EPDR view, the red traces of figs.5–7 become those of figs.8–10.

Taken together, these figures confirm that the orders of EPDR identified in figs. 8–10 are of real, practical significance when playing music signals: speakers really can make these high demands of amplifier output-device dissipation in normal use. If the amplifier's protection is invoked as a result, then its output will be clipped, even though the speaker's voltage and current demands may be within its capability.

This article is useful if you want to understand why the B&W 802 "has a reputation for being an amplifier ball-breaker."
Output impedance of an amplifier is an important metric, but is insufficient to describe an amplifiers performance driving reactive loads.

With respect to amplifiers, we have objectivists that can read these charts, see that even with simple tests, deviations can delve into the audible range so must rely on ABX to explain why it does not matter. I'd expect a call for more and better measurements.

- Rich