Blind test: we have a volunteer!!!

[firedog]

I have sent him email twice about it and he has never answered.

They answer the phone. You often get Frank when you phone.

 

[bboris77]

The only problem with using the Magnius/Modius for this experiment is that they are backordered.

 

[KSTR]

In case this project actually succeeds and has a positive result, I'd volunteer for a detailed analysis as to what the true differences are with a ultra-high resolution subtractive measurement using actual music signal, concentrating on distortion and other non-linear effects. That is, it would factor out any subtle frequency response differences as their effect on the results can be checked independently using one and the same DAC (the side result is exactly this frequency response difference).
The other way around it is also possible, determine subtle frequency response differences first and eliminate them to avoid clues, that would require a) me having both devices here before and after the test, and b) the listening setup to be used having convolving facilities.

 

[bboris77]

Another variable that cannot be forgotten in this type of blind test are the headphones being used as well as the exact output volume/power level. Anyone doing the post-mortem analysis must use the exact same setup.

P.S. it would also be really fascinating to see a blind test of 2 identical amps, one in high gain, one in low gain. While I don’t remember whether @GoldenOne has ever made any statements about negative feedback, it has been a long-standing claim by various subjective reviewers that it affects the sound in an adverse fashion.

 

[solderdude]

Another variable that cannot be forgotten in this type of blind test are the headphones being used as well as the exact output volume/power level.

For a level matched comparison of DAC's the only concern is having a good enough headphone (and he has those) and an amplifier with low distortion (which he also has).

When comparing amps with different gain (or when switching between gains) one has to carefully 'match' the levels. After that I don't think one can reliable tell amps apart. Subjective reviewers are highly unlikely to do this within 0.1dB and that's where the 'reports' come from. Incorrectly performed tests.

It may be worth noting I haven't seen @GoldenOne post anything here about the 'upcoming test' and would be interested to know if he would show the entire process and outcome on his YT channel.

 

[bboris77]

For a level matched comparison of DAC's the only concern is having a good enough headphone (and he has those) and an amplifier with low distortion (which he also has).

When comparing amps with different gain (or when switching between gains) one has to carefully 'match' the levels. After that I don't think one can reliable tell amps apart. Subjective reviewers are highly unlikely to do this within 0.1dB and that's where the 'reports' come from. Incorrectly performed tests.

It may be worth noting I haven't seen @GoldenOne post anything here about the 'upcoming test' and would be interested to know if he would show the entire process and outcome on his YT channel.

Yes, for sure, the headphones matter less if he is doing a DAC comparison. What I had in mind was a comparison between two different amplifiers, say one discrete and one op-amp based.

 

[diddley]

For a level matched comparison of DAC's the only concern is having a good enough headphone (and he has those) and an amplifier with low distortion (which he also has).

When comparing amps with different gain (or when switching between gains) one has to carefully 'match' the levels. After that I don't think one can reliable tell amps apart. Subjective reviewers are highly unlikely to do this within 0.1dB and that's where the 'reports' come from. Incorrectly performed tests.

It may be worth noting I haven't seen @GoldenOne post anything here about the 'upcoming test' and would be interested to know if he would show the entire process and outcome on his YT channel.

He should make it a live streaming event.

 

[Pio2001]

May I suggest two things to consider, please?

- 1) Both channels should be measured properly with a True-RMS voltmeter or an oscilloscope and posted in the dedicated thread/test/review; playback signal should be mono, not stereo (or channel inputs should be reversed and output voltages should be remeasured for perfect accuracy between the two channels).
- 2) A/B or A/B/X tests should also be conducted in mono, not just in stereo.

Thank you!

Hi,
Any audio interface can compare levels with an accuracy much better than 0.1 dB.
That is more difficult with an oscilloscope or a voltmeter.

What is very difficult is not to measure the levels, but to align them within 0.1 dB, as many volume settings only operate with steps of 0.5 dB.
What's even more difficult is to align separately the left and right channels level. And this is mandatory !

As for mono or stereo, the test must follow the claim under test : if the claim is that one device sounds different from the other with a given signal, then it must be verified with this signal.
If the claim is that an amplifier sounds bad with any music, then the listener can use any music to pass the test.
What would be incorrect would be to test the claim that two amplifiers sound different with a given music A, using another music B.

In the context of audio sources / amplifiers, having a positive result in a double blind listening test is very rare. Therefore the test protocol should favorise the possibility of success : the listener should be able to perform training sessions, choose the most appropriate music, choose the most suitable listening volume, have no time limit, and be able to give a final answer to a given trial only when he is certain.

It is also possible to tell the results to the listener (right or wrong) after each trial, if he wishes, as long as this information is not used to change the total number of trials (except of course if the max number of mistakes is already reached).

 

[Shazb0t]

I'm with @solderdude. There needs to be a way that the test is verified as legitimate. He has all the motivation to game the test that one would need.

 

[DeepFried]

Oh this is exciting

Having done a very basic blind A/B myself in the past and found it impossible to tell the difference between my PC headphone out and a Fiio dac/amp, even when I was sure beforehand that I could tell the difference, I wonder if this will be another case of our brains playing tricks on us. honestly I still think I can hear a difference between the two even though I proved to myself that I can't!

 

[KSTR]

I would think the only way to have an invulnerable result here is building a microcontroller/RPi-based 5-button ABX-box with a sealed compartment containing the two DACs, level matched etc. Only three connections from this black box to the world: Mains input, Digital input (SPDIF coaxial or AES3 as that can be easily split), analog out (XLR or RCA) to headphone/speaker amp.
The advantage of this that the testing person has all freedoms, is not subject to "test stress" from being observed (either by a person or a webcam), etc.

Not exactly easy but not impossible in a joint forum effort. Details matter, for example any relays must be located in a sound-proofed section so they don't give clues. Hidden clues of various sorts are the main focus, this must be avoided at all cost.

There would some requirements for the DACs:
- must power up by applying mains (otherwise the controller would also need "to press the power/standby button" in some way).
- must have Digital Input other than USB. The test set's input could be USB though when an extra interface is used to distribute the USB signal to two DAI outputs simultaneously.

 

[Randy Bessinger]

I have sent him email twice about it and he has never answered.

I wrote him a couple months ago about that unit and also never heard back. Seems kind of retired?.

Didn’t QSC also used to sell a unit? I have one of the old units David Clark, Arnie and the crew used years ago to blind test cables etc. It is in my basement.

 

[Roland]

Surely there have been a number of these blind tests carried out already to provide evidence to support the basic premise of ASR; that there are no audible differences between amps or dacs that measure the same. Otherwise where is the “science”?

 

[jtwrace]

Some info on the ABX

https://www.audiocircle.com/index.php?topic=123310.0

We have refined the design of our ABX component comparator and also made it much more affordable.

It now is direct coupled in to out (no capacitors in the signal path, no active solid state components, just carefully designed pop free relays and instant quiet switching with silicon volume controls for exact level matching.

It allows direct comparisons of two preamplifiers, two power amps, and three set of speaker systems either separately or at the same time. It even allows comparisons of interconnect cables.

Here are the details:

The price is $999

ABX Switch, rev 2, by Audio by Van Alstine 01-26-14

The ABX Switch allows seamless switching of up to two sources, two amps, and three sets of speakers.
Controlled by universal remote, usually from the listening position, the ABX switch has the capability of:
• Precise volume level matching, within 0.1 db, of any selected system.
• Infrared universal remote control.
• Blind test mode (2 of them) can force evaluation strictly by how it sounds.
• Works with or without a subwoofer.
• Programmable power up state and adjustable display brightness.
• Persistent volume settings power up the same as they were when last powered down.
• Multiple systems can be set up permanently and switched in/out instantly with remote control.
• Low noise
• Built in “Y” splitter can be used to drive two units with one source
• Allows selection of up to two speakers at the same time
• Mute entire system with a single button
• Gold plated RCA connectors for line level signal connections
• Gold plated 5-way dual banana connectors for speaker level signal connections
• DC coupled (no capacitors in the signal path).

If you are interested, contact me and I can e-mail you the full operation manual including photos inside and out.

Interesting things you can do with our ABX comparator:

If you want to make a quickie and easy blind test of two power amplifiers, for example. Attach them both to the ABX box, use the ABX box level adjustments to match signal levels from each, then simply hit the botton that turns off all the indicator lamps on the ABX box. Switch back and forth rapidly between the two amps under test a few times on the same program material and you will soon loose track of which amp is which. Then do more careful listening and when you are certain of which amp you are listening to, turn the indicator lamps back on and see if you are right. Repeat process several times. Can you really pick out what you are hearing? Once you loose the ability to hear with your eyes, all bets are off.

Of course you can do the more formal built in test procedure for the most accurate and foolproof results too.

Do you want to see if your favorite wonderful good sounding capacitor is really really better. For this you will need two identical samples of the component you are playing capacitor swaps with. Put all the good sounding parts in one sample, keep the other one stock, and run the ABX test between them. How wonderful are your capacitors now?

You can even put interconnect cables to the acid test by using two identical preamplfiers.

There are many ear opening things the ABX box can help you understand.

If you are a speaker manufacturer or audio electronics manufacturer you definitely need our ABX box. You probably will find it is your most important test too.

Frank

E-mail me at [email protected] with your e-mail address and I can send you a .pdf file of the user manual complete with photos and hookup diagrams.

 

[JSmith]

Otherwise where is the “science”?

It's all in the coat hangers mate... coat hangers.

But that was speaker cables... I'm sure transparent level matched DAC's will be really easy to ABX. /s



JSmith

 

[JSmith]

I'm all prepared;





JSmith

 

[Zensō]

I'm all prepared;





JSmith

Sorry, I was in the process of editing my post and messed up your reply. I’ll repost it.

 

[Zensō]

Surely, if we have to go to such great lengths to ensure a legitimate result, the differences must be completely inconsequential, and the characteristics described in the GO review must be wholly imagined (I suppose this is why Amir is willing to gamble $1K). That said, I do enjoy a bit of theater, which is mostly what we have here.

 

[JSmith]

Sorry, I was in the process of editing my post and messed up your reply.

No worries, I thought you'd chosen to remove the post so I thought I should remove the quote from mine... now it's all in a right state.

gamble $1K

If it is a gamble then it must be some of the best odd's ever.

I do enjoy a bit of theatre

Hence how I'm all prepared as before.



JSmith

 

[Zensō]

No worries, I thought you'd chosen to remove the post so I thought I should remove the quote from mine... now it's all in a right state.

If it is a gamble then it must be some of the best odd's ever.

Hence how I'm all prepared as before.



JSmith

Yeah, I need to dig out the popcorn maker.