Audio by Van Alstine ABX Comparator Review

[pma]

100% this. Here's the high level list of cues and mitigations I'm using for the design in the reference ABX thread

Unintended cues in an ABX test

Your approach is very sophisticated and I wish you luck in doing a final product. Not intending to be negative, but just sharing my experience with the unit that I briefly described here. Though it was not an optimum design and needed an operator to assist during the test, we never ever found a difference between DACs under test. Need to mention that the tests were performed with speakers, not with headphones. The partcipants noted they were NOT able to hear a difference based on rotary switch and relays operation. The relays were on DPDT principle and always 2 of them simultaneously switched. Please note that the channels were selected in a bottom unit by means of a rotary switch (thus operator needed).
Important note - switching was never done when playing music. The selected sample was played during path A, B or X but not switched during playing. The X path had to be selected by random sequence of activation of relays, thus only the operator knew what was the X selection.

 

[tmtomh]

This. The review is misleading.
Josh said he listened to the clicks while music was playing and couldn't hear a difference.
Amir says the distortion added by the box is equal for both units.
The box may be unsuitable for testing DACs, but the comments about Josh as a liar and an unsuitable listener are not correct conclusions.
He also passed the additional AB test that was demanded here.

Hey there - for some reason I didn't notice your posts until this one. Nice to see you here. As you probably have noticed, I don't post at AS (AudiophileStyle) anymore and haven't logged in (let alone posted) for years, since Chris created the "Objective-Fi" ghetto and then immediately let that subforum be turned into a mockery of what he claimed it was supposed to be. The one part of AS that I have still read as a guest visitor is Josh's The Best Version Of series.

I agree with you that calling Josh a liar is an example of the needlessly overheated and ad hominem rhetoric that has popped up in this thread in some comments (not to mention, it's a claim lacking in evidence). By the same token, it's important to note that Amir has written in one or more comments in this thread that he could hear the relays and the difference while listening with IEMs. So I don't think the relay question is settled by a long shot.

Regardless, I don't view Josh's or anyone else's ability to pass an ABX test as suspect in and of itself, but the specific 98% result of all his runs I do find extraordinary and worthy of questioning and further investigation. Investigation does not have to be suspicion and certainly should not descend into character assassination, but I think Amir's review raises issues that cannot be so easily dispensed with as your comment suggests.

 

[MattHooper]

Even ignoring that, the measurements show dramatic level of distortion and to some extent, noise, impacting source DACs. We are talking whopping 50 dB extra distortion!

Damn it!

The golden ears looking for ways to dismiss ABX testing boxes saying “ you can’t put anything like an ABX box in the path… it will lower transparency and could impact what you hear” were RIGHT all along !!

;-)

 

[solderdude]

This. The review is misleading.

Misleading is a big word.

Josh said he listened to the clicks while music was playing and couldn't hear a difference.

That is unlikely IF there is an audible difference as reported by Amir. Why would one not be able to distinguish differences in relay sound but pick up on cues in audio ?
That said ... Amir does have a very good microphone in his Klippel NFS with an excellent pre-amp
I understand he does not want to play ball similar to Josh who also did not want to play ball and hand over easy to obtain additional 'evidence' on request.

Amir says the distortion added by the box is equal for both units.

It is.
But the point is signal fidelity is actually compromised and has a high amount of higher order harmonics. Much more so than any DAC around.
0.017 % in music is not audible to me (suggest to take some of Pavels tests) and the transducers used by Josh are likely worse and generally mostly lower order.
A recording of the AB signal by Josh would have shown this and many other remaining questions.
That would have been instrumental. Alas Josh did not want to play. We can't blame Amir for this and the criticism is valid.
Sure ... those recordings would not have answered the 'relay sound' question. For this we need to trust Amir.

The box may be unsuitable for testing DACs, but the comments about Josh as a liar and an unsuitable listener are not correct conclusions.

Agreed... but DACs were tested using the box.
Calling Josh a liar is akin to stating Amir's review is misleading.

In both cases not all desirable 'evidence' has been presented and have to rely on what was written and presented.

In short:
People trusting Josh will still believe Josh and people trusting Amir also have seen enough.
Nothing has changed, questions remain and life goes on.

Stalemate continuous.

 

[pma]

Amir has written in one or more comments in this thread that he could hear the relays and the difference

RELAY CLICKS RECORD

Especially for you and @solderdude .

The first click is OFF, followed by ON and so on.

 

[Sokel]

As a side-note, Amir HAS a REALLY good mic, it just happens to be on the Klippel, he could just throw this thing on it

(in case that the emoticon is not enough, I'm joking)

Seriously now, and as this review is amongst the most interesting apart from the usual ego rants, would it be the worst idea to hook couple of DACs to it (or the specific ones, as Amir owns them) and observe the impedance effects on them?

Just an FR plot at couple of attenuator positions.

That would rule out another factor of doubt.

 

[pma]

There is no ASR without an ABX, so attached are ON and OFF records to make your abx homework .

 

[Sokel]

There is no ASR without an ABX, so attached are ON and OFF records to make your abx homework .

Fatal flaw as it seems:




(I could go on and make 100 trials, the tell is too large to miss)

 

[pma]

The sound was now recorded from my first and primitive A/B box, dated 2001. 4 relays operating.

 

[solderdude]

RELAY CLICKS RECORD

Especially for you and @solderdude .

The first click is OFF, followed by ON and so on.

View attachment 562624 View attachment 562626

Thanks Pavel, you saved me the trouble ... I planned to record the sound of a, similar as used, relay myself.
The ones used in your ABX box seem smaller and usually have a lower amplitude and higher pitched 'click'.
Larger relays have a lower 'clunck' alike sound and are used in the VA box as the same relays are also used to switch speakers.

The idea around the AB-box (which I read somewhere that was supposed to be implemented) is that the A-B switching relays existed from 2 relays operating at the same time but in reverse direction. So while one relay is energized (and makes a different sound) the other relay makes a different sound. Regardless if A or B is selected the resulting 'relay clicks' would always sound the same and give no tell.
Also while A to A and B to B there would be a similar 'equal sound relay click' audible.

From Amir's remarks this, however, does not seem to be implemented.
This would mean one can still tell what 'A' and 'B' is without even listening to the music and despite the indicators being off and could be a tell even when listening to music when the click is just barely audible during listening.

 

[SSS]

Seems that I was wrong. The discussion carries on. Now there is the search for the best hi-end clicking relay sound. LOL.
Seriously, the problem of relay clicks is easily solved by using reed relay and put them into a acoustic damping box. You will hear nothing when a relay is actuated.

BUT, wow, at every relay there is then a magnetic field where the sound signal need to pass and there may be ferrous material in the leads. So the signal passing relay needs to be in the non magnetic state (no coil current). And relay contacts can cause distortion, this needs to be measured. So, better use FET switches?

 

[antcollinet]

But I don't think those flaws of this comparator unit can be dismissed so easily,

I think of it more simply. There is no doubt that the noise and distortion characteristics of the comparator massively overwhelm those of the devices tested.

Which means all that can be heard is the comparator imperfections – not the DACs.

So to explain the massively unlikely result – if it came from audible characteristics – then there must be a sufficient difference between channels to be audible. Noise, distortion, or frequency response.

If it came from tells other than the sound – well, there are very few to choose from. Relay clicks, LEDs, improper level matching. That is about it. Other than deliberate cheating.

One thing is absolutely certain – it did not come from any sound differences from the DACs tested.

 

[antcollinet]

The (very scientific) principle of parsimony would encourage you to assume a mistake or some kind of clumsiness in his methods before assuming dishonesty.

Ah yes. I like : "Never ascribe to malice that which can better be explained by incompetence." Makes me smile every time.


EDIT : Ninjad - quite some time ago, by @shuppatsu

 

[solderdude]

The LEDs are off during the actual test so can not be a tell.
Remains the relay clicks (most likely given Amirs findings), frequency response differences (may even be caused by the AB-box as those are measurable), improper level matching (seems to be done properly though but no confirmation by recording the output)

I would just state: one thing is certain and that is that the listening test is suspect by lack of supporting evidence.
And even if those were present the 'clicks' or the AB box could still be responsible for the suspect result (even when unintentional/without realizing)

What the actual reason is for the unexpected results remains unclear and will never be clear either. Suspicions remain.

 

[antcollinet]

you started.

Seems not. You are regularly making unpleasant personal attacks. It is why you are normally on ignore. Not sure how you became visible in this thread.

Delusional entitlement don't deserve an explanation.

 

[antcollinet]

That said ... Amir does have a very good microphone in his Klippel NFS with an excellent pre-amp
I understand he does not want to play ball similar to Josh who also did not want to play ball and hand over easy to obtain additional 'evidence' on request.

Wow. Ad hominem aside, do you think he should be expected to dismantle or re-wire his expensive Klippel, potentially impacting calibration, to satisfy your curiosity.

 

[mcdn]

Your approach is very sophisticated and I wish you luck in doing a final product. Not intending to be negative, but just sharing my experience with the unit that I briefly described here. Though it was not an optimum design and needed an operator to assist during the test, we never ever found a difference between DACs under test. Need to mention that the tests were performed with speakers, not with headphones. The partcipants noted they were NOT able to hear a difference based on rotary switch and relays operation. The relays were on DPDT principle and always 2 of them simultaneously switched. Please note that the channels were selected in a bottom unit by means of a rotary switch (thus operator needed).
Important note - switching was never done when playing music. The selected sample was played during path A, B or X but not switched during playing. The X path had to be selected by random sequence of activation of relays, thus only the operator knew what was the X selection.

I agree with you completely - the best ABX device gives null results most of the time for DACs. Using speakers does make it a bit unfair as you're now mainly listening to the speakers not the DACs, but your point is still sound.

The new device has a lot more functionality than just switching, which makes it more useful for a range of tasks:

• automatic level matching of line level DUTs to +-0.1dB
• automatic level matching of amplifier DUTs to +- 0.1dB
• user assisted matching of speaker DUTs (obviously limited to the user's SPL measuring equipment)
• automatic polarity checking
• automatic drift detection
• automatic distortion checking (only to -80dB, but it will catch any serious audible issues before testing)
• timestamp generation so captures can be auto-cued for later checking
• self-calibration using an internal signal generator
• digital input switching and level matching
• standardised protocols for a range of tests
• saving and uploading of results
• etc etc...

 

[mcdn]

Seems not. You are regularly making unpleasant personal attacks. It is why you are normally on ignore. Not sure how you became visible in this thread.

@antcollinet @PristineSound can you both take this argument somewhere else?

 

[antcollinet]

@antcollinet @PristineSound can you both take this argument somewhere else?

When a member is making personal attacks against others, a single post pointing this out in defence of them - publicly in the thread - is appropriate.

 

[mcdn]

When a member is making personal attacks against others, a single post pointing this out in defence of them - publicly in the thread - is appropriate.

Sorry @antcollinet , got confused with the upthread noise with @Chrispy