• Welcome to ASR. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Audio by Van Alstine ABX Comparator Review

More words of wisdom from owner of AS Forum:

"I feel very comfortable with Josh's article and his conclusions, that are only valid for him. "

The what? Just for him? He did all that work only to not be valid for you or others there?

Ah, wait, you mean that blind tests are not really useful so you should be free to continue to use uncontrolled testing to promote whatever you like. Got it.
This last statement of Chris' is exactly the kind of thing I was thinking of in my earlier comment. It's not a technical claim/opinion per se, but there's no way in the world that Chris actually thinks about or can discuss Josh's conclusions as if they are only valid for Josh. It's a typically intellectually lazy statement meant to position himself as "moderate and reasonable," with no thought to the actual meaning of what he's claiming. (And by "meant to position himself" I don't even necessarily mean intentionally or even entirely consciously.) In my view and experience, there's really no useful dialogue (or even useful debate or argument) to be had with someone who takes that approach to discussion.
 
Last edited:
I respect Chris a bit because I went at him pretty hard during the dCS lawsuit threat thing, and he didn't ban me or threaten me even though I had virtually no post history and was clearly not of the A/S culture. And that's pretty much it.

I respect Josh because TVBO is a great series. He doesn't exactly cover himself in glory here.
 
Chris’ only concern is his revenue stream.
Keith
 
Josh writes:

"As Amir now confirms, I contacted him before testing about the voltage differences I was seeing, and he advised matching levels at the box’s output. I didn’t take that advice. Someone else I contacted correctly identified the impedance effect and the solutions, so the issue was solved before I ran the tests."

He should have checked his message exchange with me before telling that story. This is what he wrote me when I said to measure the output of AVA to eliminate other variables:

"Thank you for the advice. I've received other advice to measure with the DMM from the 1/8th output of the HPA4, since that would be higher voltage. Would that be better than measuring from the Van Alstine's RCA output?"

Getting higher output out of HPA4 has nothing to do with output impedance of AVA being too high and hence, causing level mismatch.

Even if he was told about this effect, he should have mentioned it in the write up. As that would be critical information for anyone trying to replicate his results.

Bottom line, nothing he says is reliable at this point.
 
The procedure used by Josh as written down in the article.

Now it’s time to level-match. After consulting with several experts, I settled on a three-stage process. First, I played test signals — a 1 kHz sine wave and broadband CTA-2034 pink noise — and measured the Comparator’s RCA outputs with a digital multimeter.
Next, I moved the measurement downstream to the point of listening. I connected the Comparator’s RCA outputs to the RCA inputs of the Benchmark HPA4 headphone amp I’d use for testing, fed the amp’s headphone output into my E1DA Cosmos ADC, and rechecked each DAC’s level in REW using the same signals. Matching through the actual amp, rather than at the Comparator alone, ensured the levels were equal at the output I’d be hearing.

Finally, I defined each system and checked the levels a third time using the amp-and-Cosmos method, toggling between the “A” and “B” systems. This final check guarded against any error introduced in the system definition process.

Only then did I enter Formal Blind Test Mode and begin actual ABX testing.
Seems like the right thing to do as the AVA box would always be loaded correctly and evenly (50k).
And even if absolute values are off they would be the same for A and B.

Now a question could be (which was not disclosed) is how tightly he matched at 1kHz (the tolerance).
This depends on the display setting in REW and can be as good as 0.01dB (around 0.1%)

This accuracy level was not mentioned in the article, he only mentioned around 0.1dB which could mean the readout was not set to 2 decimals.
If so he could be +/- 0.05dB off (assuming 1kHz sinewave) so 0.05dB to 0.15dB could be displayed as '0.1dB'.
Worst case would be 0.05dB on A and 0.15dB on B so 0.1dB could be the difference... assuming the display only showed 1 decimal.
 
The procedure used by Josh as written down in the article.
Which I quoted earlier. As I mentioned it doesn't at all observe that the two methods showed differing results and why. The fact that the AVA is broken in this manner is big news. No way is it something you read between the lines.

And again, see how his message to me indicates that the reason to measure the output of the HPA4 is because it would be louder. Not because it would be more accurate.
 
Which I quoted earlier. As I mentioned it doesn't at all observe that the two methods showed differing results and why. The fact that the AVA is broken in this manner is big news. No way is it something you read between the lines.

And again, see how his message to me indicates that the reason to measure the output of the HPA4 is because it would be louder. Not because it would be more accurate.
Did Josh record the level match data with each test? I don't recall seeing a table confirming that each test's levels were measured as XYZ and how the measurement was taken. But I may have missed that.

If it's not recorded for each test then it's another case of 'take my word for it,' which is fine for an informal, personal test. Not fine for something presented as authoritative. And it's this lack of transparency that has you running round in circles trying to figure out exactly what happened... But only Josh knows, if he even remembers such details.
 
The fact that the AVA is broken in this manner is big news. No way is it something you read between the lines.
Indeed.

The most incriminating thing Josh posted afterwards is the null between A and B when there is a difference in trimming.
It shows the pk-metric to worsen substantially (bordering on being audible) when there is a difference between trim levels.
This is the case in all comparisons between DACs except for the 2 mentioned ones where no trimming was used.
This happened to be the one with the worst score (13/16). In the other cases 15/16 and 16/16 suggests the trimming may have led to the devices being told apart.
This too is not proven but likely given the various test results.
Also the delay between the inputted sample value and outputted analog value could have generated a tell ... if long enough and the crosstalk of -40dB into consideration.
The AD5262 has -64dB but.... 2 are used and if each one handled a stereo input that would not matter and the PCB layout/wiring could be the reason.
If, however, 1 chip was used to attenuate L for both inputs and the other for R for both inputs that would change things. This would be unlikely scenario.

In any case ... the AVA level trimming can certainly (complete box, not only the chips) is certainly the weakest link and performs MUCH worse than any of the tested DACs.

One more question regarding the 20k input resistance you measured.
Was this 20ksomething or closer to say ... 23k for instance ?
 
Last edited:
No. There is no mention of any measured voltages, and evidence to demonstrate it. Heck, we don't even have the actual result of the listening trials! The AVA comparator does not compute success rate. You are supposed to keep a manual log and then one by one compare what the box says was the real input vs a log you maintained manually during the test! It is ALL take my word for it.
 
One more question regarding the 20k input resistance you measured.
Was this 20ksomething or closer to say ... 23k for instance ?
I will have to go back and remeasure. That was from memory.
 
I will have to go back and remeasure. That was from memory.

That would be interesting info. (more on that later on)
Just an ohm meter will do.
The device must be on.

These resistance values also:

  1. output A resistance (tip-sleeve) in 99 setting nothing connected to the RCA in (A selected)
  2. output A resistance (tip-sleeve) in 90 setting nothing connected to the RCA in (A selected)

  3. output A resistance (tip-sleeve) in 99 setting with shorted RCA-input A (A selected)
  4. output A resistance (tip-sleeve) in 90 setting with shorted RCA-input A (A selected)

  5. resistance between input A-pin and output pin (A selected) in setting 99
  6. resistance between input A-pin and output pin (A selected) in setting 90
 
Last edited:
I'm reluctant to spend another minute reading any more about the AVA ABX box, let alone writing any more about it, but I think there's a point here.

It's not armed conflict level important, but audible DAC difference even anything near what Josh reported, if actually, truly, specifically, true, would throw decades of psychoacoustics into extreme doubt. At the very least open up a whole new field of inquiry within it.

I don't think decades of psychoacoustics research have been thrown into doubt at all,
Josh hasn't done that just because Josh says he's done it (which he doesn't). That can only be done with collective agreement.
There was a small but finite possibility that Josh had made an advance in audio science, but once Amir's test results were out, it was pretty clear that he hadn't.
Granted, his particular test rig MAY have been able to reproduce the variances in FR & PR that he was truly hearing, but the test results prevent collective agreement.
We're left with three separate issues.
(1) Are DAC differences audible?
(2) Can the AVA ABX distinguish those differences?
(3) Do people just want to argue?
We know the answer to the third., and as you say, the second has been answered, but everyone keeps arguing about it for reasons I can't fathom.
What this episode has shown, however, is there are better ways of answering the first question.
 
(1) Are DAC differences audible?
Some are, some are not, some may be audible to person A but not to B.
(2) Can the AVA ABX distinguish those differences?
the AVA box can not distinguish ANYTHING. Only the operator can. The box can certainly mask certain differences and create differences.

We don't know... simply because DAC differences and AVA box alterations might be present leading to a perceivable difference but cannot be told apart other than... there is 'an audible difference'.
It may or may not be the DAC and may or may not have been caused by the AVA box or may or may not have been a mix.

This might be caused by:
  • level matching (not as rigorous as believed)
  • difference in filters (but remember 96kHz to not near Nyquist)
  • difference in time delay between DACs
  • differences caused by the 'amplitude trimmer' (which have been confirmed by Josh and Amir)
  • a slight difference in roll-off between trimming
  • maybe even phase differences
  • and (heaven forbid) relay click sound differences.
  • maybe not listed reasons
Lets rule out faking results by Josh.
I don't believe someone doing all the effort/time/money would be dishonest on purpose.
Making some errors he did not know were there ... sure could be possible.
(3) Do people just want to argue?
Of course.
That's why both threads are this long.
 
Last edited:
The procedure used by Josh as written down in the article.



Seems like the right thing to do as the AVA box would always be loaded correctly and evenly (50k).
And even if absolute values are off they would be the same for A and B.

Now a question could be (which was not disclosed) is how tightly he matched at 1kHz (the tolerance).
This depends on the display setting in REW and can be as good as 0.01dB (around 0.1%)

This accuracy level was not mentioned in the article, he only mentioned around 0.1dB which could mean the readout was not set to 2 decimals.
If so he could be +/- 0.05dB off (assuming 1kHz sinewave) so 0.05dB to 0.15dB could be displayed as '0.1dB'.
Worst case would be 0.05dB on A and 0.15dB on B so 0.1dB could be the difference... assuming the display only showed 1 decimal.
The way I understand the quotes from the article and as far as I can remember the discussion in the original thread, it does not actually say how levels were matched in the end. Because it mentions two test signals (1 kHz & pink noise) and both were supposedly used to match levels at the comparator output and at the HPA4 output. However, due to the slightly different treble roll-off at different attenuation levels (even into the HPA4, although plots by Josh supposedly show that to be greatly reduced) there should be a difference between the matching at 1 kHz vs. the matching using pink noise. It would be a small one, but should be easily measurable. If that was indeed the case, which signal did Josh end up using for level-matching?

Further on, I also do not know what Josh refers to with his final check after "system definition". Did he then proceed to change the volume again, even though it should have been matched before? And, of course, the lack of any documentation concerning the matching results he achieved isn't helping either.
 
Why not simply play same song through three different dacs, save the DAC output via good ADC (as all the content to begin with did come through ADC's in studios we would not lose any information) then simply compare the three recordings and see what differences we get?

What's wrong with that kind of test?

Would we get a complete null if we ADC twice the same DAC?
 
Lets rule out faking results by Josh.
Let's not make it about Josh at all.

It's about transparency (both of the ABX box and the methodology). The ABX is so corrupt it's not fit for this purpose and it's pretty clear to me (at least) that such a heap of junk could only produce a 110/112 ABX test result if there's a fundamental flaw in the methodology. But the methodology was not fully documented and every time Amir has found an issue with the box that could account for it, the methodology seems to get updated post hoc to discount it.

Trying to figure out how the result was achieved requires re-doing the test with all the issues that can be fixed, fixed. The noise/distortion is now known about and can't be removed, the level matching, it seems, ought to be correctable. Repeating the test on this box under strict, controlled, supervised, documented, conditions will then give a result worth scrutinizing.

My bet is that the real reason for the result (including them being accurate, as mind-boggling unlikely as that is) has already been discovered, but the lack of transparency of the methodology used, which seems to depend on Josh's memory (of all 112 tests!) more than anything else, is lacking.
 
The ABX is so corrupt it's not fit for this purpose and it's pretty clear to me (at least) that such a heap of junk could only produce a 110/112 ABX test result if there's a fundamental flaw in the methodology. But
Exactly. Still some, even technically knowledgeable members here try to find excuses for the test and believe in a good will of the tester.
 
I don't think decades of psychoacoustics research have been thrown into doubt at all,
Josh hasn't done that just because Josh says he's done it (which he doesn't). That can only be done with collective agreement.
There was a small but finite possibility that Josh had made an advance in audio science, but once Amir's test results were out, it was pretty clear that he hadn't.
Granted, his particular test rig MAY have been able to reproduce the variances in FR & PR that he was truly hearing, but the test results prevent collective agreement.
We're left with three separate issues.
(1) Are DAC differences audible?
(2) Can the AVA ABX distinguish those differences?
(3) Do people just want to argue?
We know the answer to the third., and as you say, the second has been answered, but everyone keeps arguing about it for reasons I can't fathom.
What this episode has shown, however, is there are better ways of answering the first question.
I wrote actually, truly, specifically, true precisely to forswear the need for responses such as yours.

Oh well, I tried. ¯\_(ツ)_/¯
 
I respect Chris a bit because I went at him pretty hard during the dCS lawsuit threat thing, and he didn't ban me or threaten me even though I had virtually no post history and was clearly not of the A/S culture. And that's pretty much it.

I respect Josh because TVBO is a great series. He doesn't exactly cover himself in glory here.

He was on the side of the angels re: MQA, but this "It's his truth" stuff is just ludicrous cod-postmodernism.
 
Back
Top Bottom