How Should I A/B Test Headphone Amps?

[hutt132]

Update: Test results in this post https://www.audiosciencereview.com/...d-i-a-b-test-headphone-amps.8186/#post-203680

Update 2: Some more setup comparisons https://www.audiosciencereview.com/...b-test-headphone-amps.8186/page-2#post-247035

I’m wanting to A/B test two headphone amps from a single DAC, with as little delay as possible between switching.

The two ways I thought about doing this are:
1) Use a switcher between the DAC and the amps and have a 2 male to 1 female 3.5mm splitter plugged into the output of both amps. This way my headphone is plugged into both amps, but only one amp is getting a signal passed to it via the switcher.

2) Use RCA splitters from the DAC to feed both amps and have a 3.5mm switcher on the output of both amps. This way the amps are both amplifying the signal to their output, but the outputs to the headphone are switched between them.

Would there not be a difference between these two setups?
Could a 3.5mm splitter cable damage the outputs of both amps, even if only one is being passed an audio signal?

I was looking at either the Nobsound Little Bear MC1022 or Nobsound Little Bear MC22 for these two setups, but am taking suggestions for a similar priced alternative.

 

[amirm]

I have had a few amps get upset when you switch away the load form it. I think the Dragonfly was one. It would shut down.

That aside, I have been doing #2 with a simple AB switcher.

You would need to match levels which is not hard when the two have low output impedance. If one is tube with high impedance for example, the impact on level becomes frequency dependent and matching levels becomes very hard.

 

[amirm]

I have to think about it but #1 may damage one of the amps. The switching on their inputs may cause a spike that will then push that signal into the other amp's output.

 

[StevenEleven]

That was the first post for @hutt132 ! And he wants to do an A/B test! How cool is that! We must welcome him! We must!

 

[hutt132]

I have to think about it but #1 may damage one of the amps. The switching on their inputs may cause a spike that will then push that signal into the other amp's output.

I had the same worry about power going into the other amps output, but wasn't sure if it would matter in this case since it's made to fluctuate power.
I'll be comparing the Schiit Jotunheim, JDS Atom, JDS O2, Massdrop O2, and Schiit Vali 2.

That was the first post for @hutt132 ! And he wants to do an A/B test! How cool is that! We must welcome him! We must!

Been a lurker on forums for years got tired of reading reviews from people that like to compare equipment by unplugging and plugging back in their headphones and then guess if there was a difference (we have extremely short auditory memory). Or even trying to compare equipment that they've heard many months back and writing a review on it.
The only way for me to actually be able to tell what is different is to directly compare them myself with as little delay as possible when switching instead of reading reviews by people who had a few seconds of delay when switching components.

 

[amirm]

He is a good man indeed!

 

[solderdude]

I’m wanting to A/B test two headphone amps from a single DAC, with as little delay as possible between switching.

The two ways I thought about doing this are:
1) Use a switcher between the DAC and the amps and have a 2 male to 1 female 3.5mm splitter plugged into the output of both amps. This way my headphone is plugged into both amps, but only one amp is getting a signal passed to it via the switcher.

Won't work because the output resistance of amp B will load amp A as well. As that can be as low as 0.1 Ohm you will effectively be 'shorting' the other amp.

2) Use RCA splitters from the DAC to feed both amps and have a 3.5mm switcher on the output of both amps. This way the amps are both amplifying the signal to their output, but the outputs to the headphone are switched between them.

This is the only way it will work.
If you want to do it properly you would have to use a switch that also disconnects the grounds from the outputs so a switch with 3 'switches' in it.
DIY is your friend here.

I was looking at either the Nobsound Little Bear MC1022 or Nobsound Little Bear MC22 for these two setups, but am taking suggestions for a similar priced alternative.

Could work, set the volume control to max setting in this case it must never be lower than max when switching outputs with this.
Better would be to use a switch for speakers.

Use the computer volume control to adjust the listening volume.
Only works for single ended (6.3 and 3.5mm TRS) headphone amps !

You will also need an AC voltmeter (or ADC in a PC or something) and a file with a 400Hz sinewave or there about.
That AC voltmeter must not be a cheapy with 200VAC as the lowest setting. It must be one that has 2VAC or even 200mV AC settings.

Then you must measure the measure and ensure the output voltage of both amps is exactly the same on both amps separately (by using its own volume control) WITH the headphones you will be testing with. Not with 2 different loads or no load.
For this you need to make a small extension cord to fit between the headphone out and headphone itself so you can measure at that point.

I have performed quite a few tests this way, very educational but requires some rigor and soldering of wires and switches.

 

[maxxevv]

I would suggest to be completely fair and transparent the following:

i) Level match the two amps by tuning to a volume that you are comfortable listening to and yet achieve the necessary fidelity that you need to compare them.

ii) Note the volume pot setting with a tape marking.

iii) Play a 60 hz tone while keeping the volume on the first amp that you marked. Measure the output AC voltage value with say a 30 Ohm resistor (if your headphone of choice is closer to 300 Ohm, use a equivalent 300 Ohm resistor instead)

iv) Connect up the second amp, do the same AC voltage measurement with the same load, but adjust volume pot until you get to the same voltage reading. Mark the voltage pot setting with tape .

v) Connect up the amps with a RCA switcher like the one you linked. And the amps output to your single headphone with the jack switcher you listed too. Ideally, use passive switchers.

vi) Switch between the two amps via the switchers. Ideally, have someone do it for you blind, so that you will not know which one ( only know that its "A" or "B") you're listening to, just noting down whether its amp "A" or amp "B" on paper. Then compare the notes.

The above will prevent any inadvertent damage to amps or weird behaviour. Also, the volume via measured output matching is important to prevent skewing of opinions that volume differences can induce.

 

[solderdude]

Note that when using actual headphones as a load the 60Hz is not the best idea for matching levels when output resistances of amps are high.
around 400Hz (440Hz is also O.K.) is safer in that case as most dynamic headphones can have a substantial peak around 60Hz.

When using a resistive load instead the 60Hz or 100Hz or there about is fine.

 

[hutt132]

If you want to do it properly you would have to use a switch that also disconnects the grounds from the outputs so a switch with 3 'switches' in it.
DIY is your friend here.
Could work, set the volume control to max setting in this case it must never be lower than max when switching outputs with this.
Better would be to use a switch for speakers.

Use the computer volume control to adjust the listening volume.
Only works for single ended (6.3 and 3.5mm TRS) headphone amps !

You will also need an AC voltmeter (or ADC in a PC or something) and a file with a 400Hz sinewave or there about.
That AC voltmeter must not be a cheapy with 200VAC as the lowest setting. It must be one that has 2VAC or even 200mV AC settings.

Then you must measure the measure and ensure the output voltage of both amps is exactly the same on both amps separately (by using its own volume control) WITH the headphones you will be testing with. Not with 2 different loads or no load.
For this you need to make a small extension cord to fit between the headphone out and headphone itself so you can measure at that point.

Thank you for the advise!
I have a Fluke 117 multimeter that I can use for measuring the voltage, and a 3.5mm 2 male to 1 female splitter where I can use the other end to take a reading while the headphones are plugged in.

Do you know of any 3.5 or 6.3 switchers that you would recommend that also switch the ground? I honestly just don’t want to diy it right now.

 

[Bob-23]

"After using it for a few seconds my attitude to audio listening tests changed forever. If you are game for a challenge then try it yourself. It will cost you under $50 to build. It may be the best or worst fifty bucks you ever spent depending on your attitude." (Phil Allison)

For an instantaneous and absolutely seamless switching I built an AB-Box with relays (construction by Phil Allison, edited by Rod Elliott on his great Elliott Sound Products-/ESP-site):

https://sound-au.com/absw.htm

It switches two poles only, i.e. doesn't switch ground, but it would be no problem to switch ground, too. I could not prove any bleeding from one channel into the other. It needs an external power supply though. It does what it should.

When used for headamp comparison only, it can be built for much less than fifty bucks, cheaper relays will do the job, too.

 

[solderdude]

Do you know of any 3.5 or 6.3 switchers that you would recommend that also switch the ground? I honestly just don’t want to diy it right now.

Haven't looked, I would craft something myself with switches like this.

Look for 4PDT switches.
(4-Pole Dual Throw)

You can even switch balanced outputs with it if needed.

 

[hutt132]

Thank you everyone! I got the black Nobsound Little Bear MC102 and a 2 pack of 1ft 3.5mm cables.
Hopefully I won't hear any noise issues with this switcher and it'll get the job done. I'll try to find some time this weekend with my friends to have an amp showdown and share pictures.

 

[hutt132]

Got the 3.5mm switcher today, had a few of my audiophile buddies come over, and proceeded to question our reality for the rest of the day.

I'll start off by describing my A/B testing setup. I played flac to my Modi Multibit via toslink, which fed two headphone amps at the same time with RCA splitters. Both amps' outputs fed a MC102 3.5mm switcher that switched between each output to feed my HD58X headphones. I used a Fluke 117 multimeter to level match each amp to 0.08v while playing a 400Hz test tone with the headphones connected.

A variety of genres were tested such a pop, electronic, country, and classical. I sat with my eyes closed while my friend clicked the button to switch the inputs to eliminate bias, so I had no idea which amp was which and I was not picking up visual queues from his body language.

Amplifiers tested were the Schiit Jotunheim, JDS Atom, JDS O2, Massdrop O2, and Schiit Vali 2 (with 6BZ7 tube and Amperex ECC88/6DJ8 Orange Globe Logo O-Getter tube).

Comparison Results Prior to the A/B Setup
My initial impressions with each amp when I was comparing them by only unplugging my headphones from one amp and plugging them into the other as fast as possible were:
Atom sounded more bright/trebly than the rest
JDS O2 sounded more full/slightly more textured than the Jotunheim
Vali 2 sounded more textured than the Jotunheim
Orange Globe tube sounded better than the 6BZ7

Results After the A/B Setup
All the amplifiers sounded the exact same. Tube amp and all. Same quality. Same soundstage. I could not tell when my friend was pressing the button besides when there was a very faint pop sometimes when the signal was switched. I had to open my eyes to make sure he was pressing the button to switch them because I didn't believe it. My friends had the same results, they couldn't hear any differences. I even unplugged one cable from each amp to ensure the switcher was actually working.

While writing this I still can't seem to believe it. I don't know how I was so sure I perceived a difference when I compared the amps by manually plugging my headphones into each. The tube amp is what shocked me the most. There was zero perceived difference between every amp tested.

 

[THW]

personally I think blind-tests are excellent reality checks regarding our actual hearing ability.

had that awakening regarding how my hearing isn’t actually so golden when I did a blind test between 320 kbps MP3 and FLAC.

 

[RayDunzl]

While writing this I still can't seem to believe it.

Assuming the buddies tried their ears at differentiating the units, did they reach the same conclusion?

---

Oh, sorry, I see they did, upon reexamining your post...

Doh.

 

[amirm]

I even unplugged one cable from each amp to ensure the switcher was actually working.

Despite doing this kind of testing for a long time, I still resort to this when the results don't agree with my assumptions. I am shocked half the time....

 

[amirm]

All the amplifiers sounded the exact same. Tube amp and all. Same quality. Same soundstage.

I have made the same comment constantly in my reviews yet folks keep talking about soundstage this and that.

 

[solderdude]

It takes a lot of trouble, extra people helping (and seeing one fail in what they think could hear so clearly) and rigor to perform such sobering tests.
It is the main reason why golden-eared folks don't like these tests.
It seems to 'take away' their hearing abilities (which are immediately back when continuing sighted) or makes everything sound the same because of reason A, B, C, D right up to Z.
It's like taking the blue pill

The background of my avatar isn't just blue-ish purple (mostly blue with a hint of red) without a reason.
Kowalski also isn't just a random choice (+ I love that type of humor)
It is 'sporting' a prototype headphone (Gimp-shopped in there) that, sadly enough, never made it to the market for reasons known to me but have to remain undisclosed. That's how this business works alas.

 

[StevenEleven]

Got the 3.5mm switcher today, had a few of my audiophile buddies come over, and proceeded to question our reality for the rest of the day.

I'll start off by describing my A/B testing setup. I played flac to my Modi Multibit via toslink, which fed two headphone amps at the same time with RCA splitters. Both amps' outputs fed a MC102 3.5mm switcher that switched between each output to feed my HD58X headphones. I used a Fluke 117 multimeter to level match each amp to 0.08v while playing a 400Hz test tone with the headphones connected.

A variety of genres were tested such a pop, electronic, country, and classical. I sat with my eyes closed while my friend clicked the button to switch the inputs to eliminate bias, so I had no idea which amp was which and I was not picking up visual queues from his body language.

Amplifiers tested were the Schiit Jotunheim, JDS Atom, JDS O2, Massdrop O2, and Schiit Vali 2 (with 6BZ7 tube and Amperex ECC88/6DJ8 Orange Globe Logo O-Getter tube).

Comparison Results Prior to the A/B Setup
My initial impressions with each amp when I was comparing them by only unplugging my headphones from one amp and plugging them into the other as fast as possible were:
Atom sounded more bright/trebly than the rest
JDS O2 sounded more full/slightly more textured than the Jotunheim
Vali 2 sounded more textured than the Jotunheim
Orange Globe tube sounded better than the 6BZ7

Results After the A/B Setup
All the amplifiers sounded the exact same. Tube amp and all. Same quality. Same soundstage. I could not tell when my friend was pressing the button besides when there was a very faint pop sometimes when the signal was switched. I had to open my eyes to make sure he was pressing the button to switch them because I didn't believe it. My friends had the same results, they couldn't hear any differences. I even unplugged one cable from each amp to ensure the switcher was actually working.

While writing this I still can't seem to believe it. I don't know how I was so sure I perceived a difference when I compared the amps by manually plugging my headphones into each. The tube amp is what shocked me the most. There was zero perceived difference between every amp tested.

View attachment 30155

Total awesomeness. My initial introduction to small audible differences was using three headed cassette decks where you could hear the differences switching back and forth in real time while you changed the bias and tried to get the recording to sound exactly like the LP. Well guess what—it never did. So I learned how to make a judgment call as to when it was close enough.

And then I had the classic MP3 vs. original in Foobar ABX experience and some listening tests on Hydrogen Audio. To the extent I could ever hear the difference, if I ever could, it was just dull work listening for artifacts. Some people really do get good at it but even if I could I was not going to put in the drudgery to become one of them or to prove that I could.

And so then when I read of ABX tests where people did not hear differences between amps I was just like, yeah, based on my other experiences I’ll take your word for it and just enjoy my music. When someone shows an ABX test where they could tell a difference, that’s okay too—they’ll usually tell you the differences were extremely minor, or there is some kind of impedance mismatch between the amp and the speaker or headphone, or the amp was clipping, etc.

Anyway, very cool Great stuff. Welcome to the world of learning how to get to truly better sound without getting distracted by stuff that makes no difference in terms of what you hear.

Still, I think it’s best not to over-generalize—there will unusual or outlier cases—weird circumstances or some kind of noise that sneaks in on one device or early clipping or impedance mismatches, But if everything’s working as intended it will at a minimum be exceptionally difficult to hear any difference, and the differences won’t be worth worrying about, and quite likely you won’t hear any difference at all.

Speakers and headphones. . . totally different story. And a whole new world of listening skills and learning and critical thinking you can develop.