• Welcome to ASR. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Audio by Van Alstine ABX Comparator Review

Frank Van Alstine rejected the results of ABX tests for a long time because he claimed that they were flawed.

He had evidently observed that some unspecified amplifiers displayed poor behavior (he described it as ringing) at the output when presented with a complex load. He then said that the poor behavior was mirrored at the _input_ of these amplifiers. This led him to say that ABX tests were flawed due to having a common input source that could be affected by one (or both I suppose) of the amplifiers under test. He further turned-up his nose at supposed grounding problems with ABX comparators.

I took this all as an attempt to explain away the fact that his products sounded no different than many others (and of lower cost) when DBT/ABX was applied.

It's interesting that, when putting his brand on an ABX comparator, he would do it on a device with such obvious flaws. :)

(Many years ago I thought that he had some interesting things to say and made some sense. Some of his products also seemed to be pretty sturdy and reliable. In addition, he would put his circuits in old Dynaco and Hafler chassis. I found that to be useful because I had some old Dynaco gear that nobody locally wanted to attempt to repair.)

Ok, back more on topic:

It's interesting that so much in this thread has been about some guy named Josh. Amir tested an ABX box. He did not test a Josh. Maybe Josh-related stuff should go in another thread?

Yes, I'm certainly one to talk - with my own off-topic writing here. "…it is good to have measured myself, to recognize my limitations." (Susan Hough)
 
Last edited:
Even if the AVA box had input- and output-buffers (which would have helped in certain aspects) the too high distortion of the AD5262 would be its downfall.
The AD5262 also has its own limitation in FR (40kHz -0.1dB, see below). And a buffer would not help that either but at least not make it worse.
Arguably it would be considered wide enough (100kHz -0.5dB) if it had a buffer.
A stepped DACT would not be able to have enough 0.1dB steps to be of any practical use unless one only needs 2 or 3dB trimming only.
A misunderstanding again. I did not mean that buffering would help to make AD5262 good and I did not want to use the stepped DACT attenuator to design an AB box. It was only shown that level independent frequency response is doable, even with the 10k pot. My posts are rather hints to use reader's own thinking. I do not intend to repeat elementary school basics. The way to make a proper design is a buffered relay attenuator. Constant I/O impedances best. Make it at about 2kohm for noise reason.
 
So for me it is clear that the test results are at least suspect.
They can only be proven to be correct after the test is repeated by recording the sound of the different DACs, level- and time-match those recordings and Josh could succesfully ABX them (out of 20) with the recordings verified.
Could 'we' (Amir or someone willing to further test the ABX box) not do this?:

1) obtain at least two of the DAC models Josh 'heard a difference between' (I'm using air-quotes there)(And we'll assume unit-to-unit variance of a DAC model is not significant)
2) record their sound, level- and time-matched, both alone and when connected to the AVA ABX box, using the test signal (music) that Josh used?
3) optional: do an ABX test as well

Or are we missing some data that would preclude all that?
 
Could 'we' (Amir or someone willing to further test the ABX box) not do this?:

1) obtain at least two of the DAC models Josh 'heard a difference between' (I'm using air-quotes there)
2) record their sound, level- and time-matched, both alone and when connected to the AVA ABX box, using the test signal (music) that Josh used?
3) optional: do an ABX test as well

Or are we missing some data that would preclude all that?
It would be a great, and likely definitive, test but would take a lot of effort on @amirm 's part. You would not need to use the same test signal, however. If the measurements showed significant differences, as most everyone here feels to be true, then no need for further testing (ABX or otherwise) but it would close the loop.

So far, the consensus seems to be that the ABX switch is flawed and thus test results inconclusive at best. Without your experiment or similar it is unlikely the two sides will ever agree. And if you do the test, the outcome will probably be the same as far as the two camps are concerned, but it would tie it off for us (assuming the difference is measurable). The relay clicks are already pretty obvious, however.
 
Could 'we' (Amir or someone willing to further test the ABX box) not do this?:

1) obtain at least two of the DAC models Josh 'heard a difference between' (I'm using air-quotes there)(And we'll assume unit-to-unit variance of a DAC model is not significant)
2) record their sound, level- and time-matched, both alone and when connected to the AVA ABX box, using the test signal (music) that Josh used?
3) optional: do an ABX test as well

Or are we missing some data that would preclude all that?
Amir already did that some pages ago and his comment was that he heard day and night difference.
Hence the revival of the thread.
 
I'd be more curious to see if the d*mn box can be sold again, so that some of us get some $ back :-D ... maybe someone asking Amir to do additional tests want to buy the box and perform the tests themselves? :-D
 
Could 'we' (Amir or someone willing to further test the ABX box) not do this?:

1) obtain at least two of the DAC models Josh 'heard a difference between' (I'm using air-quotes there)(And we'll assume unit-to-unit variance of a DAC model is not significant)
2) record their sound, level- and time-matched, both alone and when connected to the AVA ABX box, using the test signal (music) that Josh used?
3) optional: do an ABX test as well

Or are we missing some data that would preclude all that?
Pointless. Why? What purpose?
 
I'd be more curious to see if the d*mn box can be sold again, so that some of us get some $ back :-D ... maybe someone asking Amir to do additional tests want to buy the box and perform the tests themselves? :-D
I am kind of OK keeping it to compare speakers in the future. Not for any kind of blind testing but for level matching AB to hear tonality differences.
 
Thanks to the measurements from Amir, combined with some additional info from Erin, I was able to figure out how the actual level trimming circuit is designed.
Thanks for the detailed analysis. Certainly above and beyond! :)

Do you still need me to measure those static values or is that the data you now have from Erin?
 
Some odd commentary by Josh:

"Since then, Amir has begun to cast doubt on the fidelity of the ADC loopback method, too, and it’s clear that the only ABX test that will ultimately be accepted by folks at ASR is one that Amir administers in person. I have no desire to participate in such a spectacle."

Why would it be a spectacle? He is able to so easily to tell DACs apart. If I were him with such ability, I would be happy to have it proven yet again, this time in person with a witness. This tells me he has clear doubt that would be able to hear the claimed differences.

BTW, I have not offered to do this but it think someone local to him could be made available. We could even eliminate the AVA box since again, he is able to so clearly tell DACs apart. He said in his article that the difference was so clear that he could come back to it way later and still hear the difference:

"Once I noticed one clear difference that allowed me to identify the sources, it became very difficult to hear past it to other potential differences. I tried to get around this by leaving weeks between tests of the same pair of DACs and not reviewing my notes. That worked once or twice. Eventually, though, it became impossible not to remember at least one known difference."

Ergo, we can play the same music for him. Switch cables without him seeing. Then repeat. Switching doesn't need to be instant.

"As I wrote earlier, I bought the box because I thought I could distinguish level-matched DACs using my passive XLR switch, and I seemed to pass a test with a helper. But I knew that wasn’t a true ABX test, and I had my own doubts, which is why I bought the Van Alstine box. It’s also why I decided to do a few of the loopback tests."

I would have accepted those "helper" tests with much more ease than anything he has done with this box, and the conduct he has shown since. There is nothing wrong with a proctored test like that if the helper did not produce any "tells." I don't know why he thought a highly distorting box like AVA would be more acceptable to us when the first thing it does is remove transparency that the source DACs worked so hard to produce.

Really, he should have just asked us for feedback at the start. Plenty of people have by posting in threads, asking for feedback, then sharing the results and being open to questions. It is not like he is an expert in any of this.

It is still not too late. Let's have some local audiophiles go to his house and a) see if he can pass a simple AB test with cable swapping and b) have said audiophiles also take the test and see how they compare. He says he has DACs that didn't need level matching. Let's have that be confirmed with multimeter reading.

None of this is hard and is much less work, and more definitive than us continuing on the path of analyzing the AVA comparator.

"It makes no difference to me whether you, Amir, or anyone at ASR believes my results. I wasn’t trying to convince you of anything or ask you to change your behavior."

It is not just for us damn it. :( It is for all of audio science. We need to know if you can indeed hear these differences with such ease so that we can raise our standards and get to the point of transparency. Why in the heck do you think I am spending so much energy and resources on this? Show an ounce of curiosity with respect to getting the truth for heaven's sake. Repeat the test, and let's see if you can get there.
 
Amir already did that some pages ago and his comment was that he heard day and night difference.
Hence the revival of the thread.
Thanks for providing a embarrassing (for me) example of what I railed against: 'forgetting (or not reading) what happened here a week ago'.

Perhaps you or he can help me through Amir's post there.

Amir's listening signal chain, as best I can glean, was
AudioPrecision (AP) analyzer/signal generator optical out ---> DAC optical in---> DAC RCA out ---> AVA RCA in -->AVA RCA Out --> A90 headphone amp--> 'random IEM'

The measuring signal chain would attach the measuring device to either the same AVA RCA out or A90 amp's headphone out.

AIUI, the AVA ABX box has a built-in level matching feature, using trims and a numerical display. Amir used the trim control on AVA to "get them to match to about 0.07 dB which was achieved with trim level of 90 vs 99" (99 on the AVA being 'max level' i.e. no trim). He then listened to white noise via a headphone amp and IEMs. He heard a whacking big night and day difference beween A and B right away! Then with a 1 kHz tone, still different: the 99 trim sounds louder than the 90. Same result if done blind: it seems AVA level matching sucks!

Then: "I try many different methods to measure the output of the AVA from AP to digital voltmeter. They all show that fraction of dB difference [presumably, ~0.07dB]. No way am I hear[ing] that slight difference so clearly." [Which is all little hard for nontechnician me to parse. Did it mean: I tried many different measuring devices, ranging from an AP analyzer to a digital voltmeter (DMM)? Or was it, I tried measuring many different parameters, using an AP and a DMM? Or something else meant by methods? But this is a side-issue]

Amir then wonders if the culprit was the AVA box's output impedence changing with trim setting. This could have audible effects but be objectively masked by the AP's and a digital multimeter's own large default impedences (100 k Ohm and 1M+ Ohm , respectively). And voila, when he lowers that to 600 Ohms, he now can see (shown in a graph of output impedence vs frequency) that there's a considerably higher impedence for the AVA output at a 90 trim (~8.6 k ohm) than the one with 'no' (99) trim (~5.3 k ohm). It also damningly shows that output impedence varies with *frequency*, and differently so, between trims, with the 90 trim having much stronger impedence dropoff at higher frequencies that the 99 trim does.

tl;dr: using AVA's level matching could actually cause an audible difference, especially in the high frequencies. (NB the graph does not measure 'audibility' in dB per se)

The A90 headphone amp input has a 10 k Ohm impedence, so there's likely to be 'big time interaction' between it and the AVA box, too. It won't be fully masking the output impedience difference. But what is its actual effect?

When Amir sets AVA trims based on the A90 headphone amp's output, they're now set to 99 vs 96, and the A and B outputs are now 'audibly very close'. I presume this judgement used the 1 kHz tone, sighted. Amir knows that 'level matching' two outputs that vary in 'EQ' -- e.g. at high frequencies -- as well as in overall level isn't precisely possible without an equalizer. He notes that he needs to go back and try the white noise signal to see if the 'high frequency impact' specifically has lessened with amp-output-based level matching. (Don't know if he did that, and no relevant measurements shown at this point in his post -- yet.)

He then compares this all to what Josh did to level-match:

-- Josh first level matched via AVA trims using a 1 kHz tone and pink (not white) noise and checked/measured the results at the AVA RCA output with a digital multimeter --> Amir says Josh's voltmeter would have an input impedence >1 M ohm, and we now know this would badly mislead the user for 'level matching'.

-- Josh then checks the AVA's level matching at his headphone amp's output, by feeding his Benchmark HPA4 amp's headphone output into a E1DA Cosmos ADC and then digital out into REW using the same signals (1 kHz and pink noise). [NB, this seems rather over complicated, to me, compared to what Amir did.]

-- Josh even repeats the latter with the output of a different amp, the Benchmark LA4

The impedence of one or both of these amps is 50 K ohm, according to Amir, enough to still have a mismatch.

Given Amir's findings, Josh at this point should have noticed that the AVA-'level matched' DACs he confirmed at AVA output with a multimeter were in fact NOT level matched at amp output, as measured by REW. But Josh did not report that. Josh (Amir's quoting) only said that measuring at the amp out "ensured the levels were equal at the output I’d be hearing", with no reference to the previous protocol's result. Nor measurements to demonstrate any of it (at least, none reported by Amir)

At this point Amir shows his AP's FR measurement (RMS level vs signal frequency) of the Topping D70 and Matrix MQA DACs, measured at AVA output (not headphone output), when AVA trims are set to 90 and 99, i.e., as he first heard the 'night and day' diffs on his IEMs at the top of his post. He says it shows "nearly 0.2 dB difference due to filter differences".
It does show that differences, but I find this graph hard to interpret.

1791922716329.png


I see four colored lines (curves), two thick, two thin, DAC A being red, B purple. (There's also a mysteriously vertical purple line at 20 kHz). I'm not sure which lines are 'data' and which are 'cursors', and I don't see 'delta' (black). The y axis is labelled in 0.5 dB increments -- quite small increments. In thick lines I see there is a consistently very small difference - I would estimate this as quite less than 0.2 dB, closer to the 0.07 dB 'fraction of a difference" -- between A and B across most of the frequency range, growing larger (and crossing) above ~11 kHz. The thin lines seem to be separated by ~0.2bB and are virtually parallel...with a tiny change high up? I think these parallel lines would be the showing the 0.2dB difference Amir refers to? I feel stupid, I don't get the relationship between the thick and thin lines.

In any case, this would seem to be a graph indicating what would have been fed to Amir's headphone amp in his initial listening.
What has me scratching my stubble is, I'm not grokking where a 'night and day' difference heard 'so clearly', comes from, unless its that >11kHz diff, or something his amp did to the signal. Is a 0.2dB diff from 10Hz to 11 kHz otherwise really night and day? I don't know. Haven't tried doing it. The difference does include the frequency range humans are most sensitive to.

Is this proposed as an explainer for what Amir heard at the start (and by extension for Josh's ridiculous 'successes'?)? But it's more or less what Amir would have measured at the start, yes?(Or not?) Why was it so confounding then as an explainer, if not now?


(and just to dot an 'i': Did Josh test a Topping D70s vs Matrix MQA DAC? I'll assume yes, though it wasn't explicit in Amir's post)
 
Last edited:
Thanks for the detailed analysis. Certainly above and beyond! :)

Do you still need me to measure those static values or is that the data you now have from Erin?
No not really needed anymore. Erin provided me with the data from his box (thnx Erin). I assume it is a similar box internal in design.
It allowed me to determine the resistor values. It's those resistors that bugged me as to why there was a -1.6dB drop at 99 trim and why your output resistance measurement made little sense. Now they do.
 
Last edited:
It does show that differences, but I find this graph hard to interpret.
AFAICT, thick curves are data, thin lines (horizontal and vertical) are cursors. The exact position of the cursors is displayed in the lower right corner. X is 19.7682 kHz for both, that's the vertical line. Y is -0.512 (red) and -0.330 (purple), these are the horizontal lines. There's no line for delta, only the calculated value is displayed. It's 0.181 instead of 0.182, probably because of rounding. There isn't any "change high up" for the thin lines. If it looks like it, then maybe it's just an optical illusion in the presence of the data curves?
topping_d70_vs_matrix_mqa_zoom.png
 
Thanks. I was indeed reallystretching tentatively about a possible 'change high up' in the cursors..they looked parallel.

0.181 rounds to 0.2, same as the number Amir refers to.

What the cursors are, why they are there, in relation to the data curves -- still don't know.
 
What the cursors are, why they are there, in relation to the data curves -- still don't know.
They show the differential at nominal 20 kHz to determine audibility potential due to combination of DAC roll off added to that of AVA.
 
20 kHz looks maybe like the point of maximum difference in the 10 Hz-20 kHz range. Is it, and is that why it was chosen for the cursors?
 
20 kHz looks maybe like the point of maximum difference in the 10 Hz-20 kHz range. Is it, and is that why it was chosen for the cursors?
No, it is a frequency response chart so the cursors on purpose were put at 20 kHz nominal. Normally there would be another at 20 Hz but I borrowed that to show the difference between the two graphs at 20 kHz.
 
I want to make sure this simple point is understood as Josh keeps saying it isn't so.

New setup. Signal generator is Audio Precision APx555 with output level of 2 volts and impedance of 100 Ohm. AP is not used from here on.

All measurements are performed using Keysight U1237A multimeter. I used this instead of AP because its input impedance is very high (10 Megaohm). Here is the data:

MeasurementTrim 99Trim 90
Input voltage2.00 V2.00 V
Unloaded output1.76 V1.58 V
Loaded output (50 kΩ)1.59 V1.37 V
Loading attenuation−0.88 dB−1.24 dB
Calculated source impedance5,346 Ω7,664 Ω

Translating, I first measured the output of AVA with trim levels left at 99. The box itself causes a voltage drop from 2 volts to 1.76 which is fine (though should have been documented and noted).

I then added an output resistor with nominal value of 50K Ohm to emulate the input impedance of HPA4 headphone amp Josh used for listening tests. Further voltage drop was observed to the tune of 0.88 dB. This indicates that the output impedance of AVA is fairly high. Again, not a problem by itself (though yucky from engineering point of view).

The problem comes into picture when I perform the same series of measurements, this time with trim level set to 90 which I needed to do to match the levels of Topping D70s against Matrix MQA. We now see a 1.24 dB difference between no load on the output of AVA, vs loading it down with 50K Ohm resistor. This is of course different than 0.88dB we measured for trim 99. In other words, the output impedance of the AVA is trim dependent and highly so.

Computed source impedance is shown at the bottom which shows stark trim level dependency.

Put another way, using the trim control to match levels now introduces a 0.36 dB error (0.,22 volt) in Josh's setup (input to HPA4). This should have been easily observable by him, as was to me in my testing.

This should have been a clear aha moment for him and clearly and crisply noted in the article. That it was so unimportant to him as to not get a mention, makes me doubt anything he says regarding level matching.
 
Below the schematic as is present inside the AVA box. That is excluding the relays used for routing the input signals. These will not change anything.
AVA box schematic.png
I have the box open but don't see either one of those 3K Ohm resistors. There is a single resistor visible which is half a megaohm to ground. How did you ascertain that there were those two resistors in line?

I do measure 23K or thereabouts but I think that is what the AD IC is presenting, not because of any external components.
 
Back
Top Bottom