Audio by Van Alstine ABX Comparator Review

[markk]

....

But of course he doesn't need to have consciously noticed the difference in relay clicks in order to have it influence his testing.

It's somewhat of an intellectual stretch to have someone subconsciously pick the "correct" DAC 110 out of 112 times based on some golden ear superpower, but not have said listener be consciously aware of the difference in the click and how it might influence his choice.

What we know is the ABX comparator is not truly blind and so the results should be taken with a great deal of suspicion. But yes, suspicion as to one's inward motives can be hidden behind a veil of plausible deniability I suppose.

 

[RandomEar]

It's in 0.1 dB increments, which should be good enough.

Nominally, yes. If you divide the level differences found in Amir's measurements (see this post for the numbers) by the number of steps, you actually arrive at 0.13 dB/step. So the worst possible deviation - if carefully matched - would be 0.065 dB.

However, this assumes that all DACs also do have an equally flat FR. Otherwise, deviations at one end of the spectrum could make the level matching worse in the most sensitive region of our hearing around 2-5 kHz, if the device used for level-matching averages the SPL over the full audible range. This is where recordings of the DUTs would have been very valuable... again...

 

[pjug]

Deleted. Never mind, I misunderstood something.

 

[Bob101]

Why hasn't anybody made a device like this that actually measures well? How hard can it be these days?
It could once and for all silence the 'I can't explain but I hear a difference' crowd

Until then: someone should start a business selling audiophile blindfolds. In many varieties of course: smooth, gritty, airy, freezedried, whatever... All equally ridiculously expensive of course.

 

[amirm]

Surely you of all people have access to a proper microphone…

What do you mean? You see me recoding live sounds in my review work???

 

[mdunjic]

So Josh was lying. What a surprise.

He was probably clueless and unaware

 

[amirm]

People who trust Josh and his test will trust the results.
Others that would have liked to see measurements simply continue to think the result is a fluke.

No. I did not take a position like that. I wanted to find out what really went on so asked to purchase the device and got the run around. Now that I have the box, it is clear that if one wanted to cheat, it was trivial to do so. It didn't require hooking up an analyzer and such to see the difference. Mere listening to the sound of the relay, which you could back in time and then forward to compare, would let you identify which source is which.

 

[kemmler3D]

The reviewer claim that the sound is "uniform" shows either extremely poor ability to listen for differences, or dishonest reporting of the device characteristic.

We could use this as some kind of control and dismiss the listener as not being qualified to enter the next phase of detecting differences in DACs

Michael_jackson_eating_popcorn.gif

In seriousness I would add it's probably worthwhile to get a good recording of the click sounds, but it sounds like the short takeaway is "differences in relay clicks were louder than any known difference between DACs".

I think it is possible that Josh heard the click differences, consciously ignored them, but subconsciously used those to trigger a "placebo"-driven heard difference in the DACs.

Or at least that's a way to explain what happened without any need to question his honestly or intentions.

 

[amirm]

Can you record the sounds emanating from the box when switching A>B, B->A and A>A and B>B and post those ?

Before you put me through that work, I have been listening to relay clicks more than 50 years. I have always heard the difference in the form of "tick" vs "tock." As a fellow engineer, you think a relay make or break can sound identical when it is clearly audible from many feet away? Remember I post the article from Bost Audio Society hearing the same effect.

Mechanically and acoustically, it is going to sound different. The contacts are different, and mechanical forces on the relay are different. So it is going to sound different. The differences could be small but heck, here we have someone saying he is hearing differences in number of atoms moving from one place to another. How could he not?

The only defense against this would have been him stating that the relays did sound different but he did this and that to avoid that impacting the results. But he said the opposite.

I realize we have the means here to tell sources apart and not proof of whether such means was used. We have to apply this information to the unbelievable results he got. And the fact that the differences he claimed he heard were the type typically claimed by subjectivists.

And then we the massive signature of the ABX box imprinted on the testing. I don't care that it is similar to both inputs. What is important is that the effect is so severe that it would by far dominate any impression of fidelity or lack thereof. As a minimum, he should have commented that the box ruined the fidelity of tracks he was listening to.

 

[voodooless]

So, how are they coping over at the other forum?

 

[amirm]

So, how are they coping over at the other forum?

I just looked. The form owner says this:

"Has anyone without his ego on the line looked at the results and compared them to the measurements Josh made, and rendered an opinion? Also, was the supposed distortion below the level of audibility, which I’m sure was established decades ago?"

So he worries about threshold of audibility here but not when it comes to differences between two DACs?

 

[pjug]

I just looked. The form owner says this:

"Has anyone without his ego on the line looked at the results and compared them to the measurements Josh made, and rendered an opinion? Also, was the supposed distortion below the level of audibility, which I’m sure was established decades ago?"

So he worries about threshold of audibility here but not when it comes to differences between two DACs?

It is the same with their rejection of doing the test with Foobar2000. The extra ADC and DAC are a problem that don't let you really hear the comparison. But added distortion and noise and rolloff from a comparator box doesn't matter.

 

[pma]

This is a review and measurements of the Audio by Van Alstine (AVA) ABX Comparator. It is discontinued but I purchased it for US $1,300.

The result is that bad that it did not need AP to measure it. (And from the components used for volume control we expected how it would measure ). Even a cheap Focusrite 2i2 would have been enough to measure level of distortion components and level of mains hum components. That said, it is clear that Josh could have done measurements himself. He did not and he was:

1. reluctant to measure the box, though he was using Multitone software,
2. reluctant to make recordings from DAC outputs.

These facts and his style of communication re lending/selling the box make me feel he was aware what it was, at least to some extent, and he knew well why he did not give any usable data.

 

[solderdude]

No. I did not take a position like that. I wanted to find out what really went on so asked to purchase the device and got the run around. Now that I have the box, it is clear that if one wanted to cheat, it was trivial to do so. It didn't require hooking up an analyzer and such to see the difference. Mere listening to the sound of the relay, which you could back in time and then forward to compare, would let you identify which source is which.

I understood that the box masked the typical relay sound by always flipping 2 relays in the opposite 'direction' at the same time and thus no difference in sound.
Is this not the case in this box ?
If so then the relay sound can certainly create a tell.

This is why I suggested to record the sound of the relays in the mentioned direction and offer it as 'proof' on how it is possible to cheat knowingly or unknowingly.
Posting the relay sound will silence the folks that claim there was no tell and the ABX test was valid.

The only possible technical explanations are:
a: relay sound.
b: different roll-off due to cable capacitance and varying output R.
The latter is unlikely so the most obvious culprit would be relay clicks which is very easy to prove by simply holding a mic close to the box and operate it.

It saves a lot of time as the box does not even have to be tested again.

 

[pjug]

The result is that bad that it did not need AP to measure it. Even a cheap Focusrite 2i2 would have been enough to measure level of distortion components and level of mains hum components. That said, it is clear that Josh could have done measurements himself. He did not and he was:

1. reluctant to measure the box, though he was using Multitone software,
2. reluctant to make recordings from DAC outputs.

These facts and his style of communication re lending/selling the box make me feel he was aware what it was, at least to some extent, and he knew well why he did not give any usable data.

He also knew the history of relay clicks being a problem for the comparator. He just brushed it aside since he had a newer version of the box, seemingly unwilling to check this for himself.

 

[pma]

This is why I suggested to record the sound of the relays in the mentioned direction and offer it as 'proof' on how it is possible to cheat knowingly or unknowingly.

I do not think it would make any sense to push Amir to make one more and questionable measurement, without any defined method how to evaluate it.

 

[solderdude]

It was merely a request to post this and remove all doubt to readers incl. those in the other forum.
Seemed so trivial and easy to do.

Camera on the device. mic on top of it and operate. This could have saved Amir from doing all the measurements if the relay click sound was not as advertised and is an easy tell.

Ah well... it was just a suggestion to remove all doubt.

Have built plenty of stuff with relays and can easily hear if one is energized or falls off.
I was under the impression that the trick of the ABX box was that always 2 relays do the switching where one is energized and a second one falls off. Always the same sound.
I also was under the impression that there was another relay that muted the signal during the switching to even mask the timing difference (energize is usually faster than falling off).

 

[shuppatsu]

I wanted to find out what really went on so asked to purchase the device and got the run around. Now that I have the box, it is clear that if one wanted to cheat, it was trivial to do so.

Unlike the Goldensound filter imbroglio, none of this was recorded. If Josh wanted to cheat he wouldn't have listen to relay clicks. He could have reported whatever results he wanted to. Which is maybe what he did! I like to think not, but who knows?

And then we the massive signature of the ABX box imprinted on the testing. I don't care that it is similar to both inputs. What is important is that the effect is so severe that it would by far dominate any impression of fidelity or lack thereof.

From where you and I sit, this is pretty damning, though as @solderdude stated wholly predictable by the components used in the device. But as you well know from the "Do measurements mean everything or nothing?" megathread, a substantial portion of subjectivists don't think that distortion measures the differences they hear. But if the goal is simply to assure people with ASR priors that they can continue believing what they already do, then mission accomplished?

 

[pjug]

The result is that bad that it did not need AP to measure it. (And from the components used for volume control we expected how it would measure ). Even a cheap Focusrite 2i2 would have been enough to measure level of distortion components and level of mains hum components. That said, it is clear that Josh could have done measurements himself. He did not and he was:

1. reluctant to measure the box, though he was using Multitone software,
2. reluctant to make recordings from DAC outputs.

These facts and his style of communication re lending/selling the box make me feel he was aware what it was, at least to some extent, and he knew well why he did not give any usable data.

I think the AS owner should catch some of the heat for all this too, not just Josh. When people were trying to get Josh to provide more to help get at what Josh was hearing, Chris was lashing out against them and chalking it up to imagined agendas. I think Chris was much worse in all this than Josh was.

 

[voodooless]

It would be fun, in true mythbuster fashion, to replicate the original results.

@amirm replicating a 16/16 test with the box surely would be the final nail in the coffin.

Yes, I realize it’s more time invested, but possibly some other member would be willing to give it a go