Blind test: we have a volunteer!!!

[Raindog123]

The Magni 3+ would likely be easier to distinguish from the Magnius compared to the Atom.

Magni 3 (including Heresy) does not have differential inputs.

 

[MrPeabody]

That's exactly why an AB test should be conducted in mono too, not just in stereo, to remove the spatiality. This will ease things up in perceiving details retrieving differences or bass differences etc.

Don't you think that Amir and GO can work out the parameters without you insisting on things that neither of them cares about?

 

[GoldenOne]

That's exactly why an AB test should be conducted in mono too, not just in stereo, to remove the spatiality. This will ease things up in perceiving details retrieving differences or bass differences etc.

As @JohnYang1997 said this should be valid in stereo.

It may be an interesting test to see if it's easier or harder to discern a difference when in mono if the first test in stereo passes, but until then it makes sense to keep things applicable to real world use and use stereo.

Firstly because as said, that's how we listen to stuff. I said I thought I could hear differences in stereo. That is the assumption we're testing.

Secondly because if you're trying to test the assumption that things sound identical, you should give every potential opportunity to assist finding a difference. If your assumption that the two things sound identical is correct then it won't matter and only makes the result more conclusive.

But if the assumption is wrong then the scientific approach is to approach it in an unbiased manner. Deliberately designing the test so that even if there is a difference it will be harder to discern is just approaching things in a biased manner.

The results need to be able to stand on their own, be conclusive, and applicable to real world use. No matter which way the results turn out.

 

[amirm]

If not, I'd probably be OK to use the atom but honestly haven't spent much time with it so would need to check if I can actually hear a difference between it and magnius first.

I lost track of all the things you said was wrong with Magnius. You are saying the atom has all the same faults? How could that be?

 

[GoldenOne]

I lost track of all the things you said was wrong with Magnius. You are saying the atom has all the same faults? How could that be?

No I'm saying I've hardly listened to the atom and not at all with any full size cans so I don't know what it sounds like. And therefore I can't say anything about it yet.

Youre correct, it would be incredibly unlikely to have all the same issues I perceived hence Im probably happy to use that. But I don't want to commit to anything yet given as I've not actually listened to it

 

[richard12511]

As @JohnYang1997 said this should be valid in stereo.

It may be an interesting test to see if it's easier or harder to discern a difference when in mono if the first test in stereo passes, but until then it makes sense to keep things applicable to real world use and use stereo.

Firstly because as said, that's how we listen to stuff. I said I thought I could hear differences in stereo. That is the assumption we're testing.

Secondly because if you're trying to test the assumption that things sound identical, you should give every potential opportunity to assist finding a difference. If your assumption that the two things sound identical is correct then it won't matter and only makes the result more conclusive.

But if the assumption is wrong then the scientific approach is to approach it in an unbiased manner. Deliberately designing the test so that even if there is a difference it will be harder to discern is just approaching things in a biased manner.

The results need to be able to stand on their own, be conclusive, and applicable to real world use. No matter which way the results turn out.

I've tested the mono vs stereo thing with speakers several times, and imo it's much easier to hear tonality differences in mono. I think this test will be much easier to pass in mono, but I also understand the reason for wanting to go stereo(since that is how we listen).

Only time I think stereo might be easier is when you have two speakers with equal frequency response but much different dispersion widths. In that case, the imaging/soundstage is likely going to be the biggest difference, which is probably better suited for stereo. Given that this is an electronics test, soundstage/imaging won't be affected(since these devices can't change dispersion patterns).

You could always try both

 

[amirm]

- I'll do the test at a friend of mine in London's house.

This is a showstopper I am afraid. Whoever it is, is a proxy for me, not for you. With my money on the line, we need to have someone on the side of objectivity so that he can be on the look out for any issues that come up in person.

I know we have other members in London. We need to see one is kind enough to do this work. I am happy to pay $100 for their time and effort.

 

[amirm]

No I'm saying I've hardly listened to the atom and not at all with any full size cans so I don't know what it sounds like. And therefore I can't say anything about it yet.

I am really lost then. I thought all of these amps had different sounds to you. Now you are saying that some could sound the same? Anyway, can't you listen to Atom tonight and report to us tomorrow?

 

[Raindog123]

London - 05:06am

 

[GoldenOne]

I am really lost then. I thought all of these amps had different sounds to you. Now you are saying that some could sound the same? Anyway, can't you listen to Atom tonight and report to us tomorrow?

I can report tomorrow whether I think the atom sounds different to the stuff I have here. And as soon as the Magnius arrives I'll confirm if it sounds different to me to that as well.

And yes. Some amps do sound either indistinguishable or incredibly similar to the point where I'm sure i wouldn't be able to reliably tell them apart.

I never claimed that EVERYTHING sounds different and that's also not what we're looking to test.

 

[GoldenOne]

London - 05:06am

Hey we're judging audio not my sleep schedule

 

[GoldenOne]

This is a showstopper I am afraid. Whoever it is, is a proxy for me, not for you. With my money on the line, we need to have someone on the side of objectivity so that he can be on the look out for any issues that come up in person.

I know we have other members in London. We need to see one is kind enough to do this work. I am happy to pay $100 for their time and effort.

That's fine
Happy to have someone of your choosing host if they're comfortable to do so, or just come and observe/verify at my friends. Beer and food provided

 

[amirm]

I can report tomorrow whether I think the atom sounds different to the stuff I have here. And as soon as the Magnius arrives I'll confirm if it sounds different to me to that as well.

So you have located one? Please confirm and I was trying to find one as well.

 

[GoldenOne]

So you have located one? Please confirm and I was trying to find one as well.

Viewer of mine has offered to send me theirs but its coming from Canada so may be a while

 

[trl]

The test would still be valid in stereo, right?

I don't see why not. Thing is to have the AB test done in both stereo and mono, well...at least few seconds in mono and the rest of tests in stereo.
I expect that the mono AB compare will better retrieve details differences, due to eliminating the scene caused by the stereophony.

In mono is also way easier to catch volume channel differences, mostly caused by the ribbon potentiometer.

 

[trl]

Don't you think that Amir and GO can work out the parameters without you insisting on things that neither of them cares about?

I usually like to make my point clear, so I do insist if I feel that more arguments need to get put on the table.
There's no need to be rude nor impolite, especially today!

 

[amirm]

I don't see why not. Thing is to have the AB test done in both stereo and mono, well...at least few seconds in mono and the rest of tests in stereo.
I expect that the mono AB compare will better retrieve details differences, due to eliminating the scene caused by the stereophony.

In mono is also way easier to catch volume channel differences, mostly caused by the ribbon potentiometer.

Let's let him test in stereo just like he did in this video review. Hate to change that, him failing the test, and have him blame it on this factor.

 

[solderdude]

I can report tomorrow whether I think the atom sounds different to the stuff I have here. And as soon as the Magnius arrives I'll confirm if it sounds different to me to that as well.

And yes. Some amps do sound either indistinguishable or incredibly similar to the point where I'm sure i wouldn't be able to reliably tell them apart.

I never claimed that EVERYTHING sounds different and that's also not what we're looking to test.

The A90 is a poor choice to use as a comparison. Not a poor amp but not the wisest choice for design reasons as both use the exact same output device (TPA) and seem to let most of the gain be handled by another opamp. Also both seem to use the same 'trickery' to get a low output R.
Chances are that the pure resistive load of the Susvara will be a very easy load when some 'trickery' is used to prevent the TPA from oscillating on complex loads. The Susvara (nor HE1000, nor Arya) are anything but a complex load.

When there really are audible differences between amps I would expect them to be in the business end (output stage) combined with loads.
The PCB design of Schiit would have to be really Schiit to differ that much, which I don't think it is, but given the 20dB difference in crosstalk for 32ohm between SE and balanced it does raise some concern about the common/return path. Not that -70dB crosstalk is audible but just an observation.

I was about to comment about the Atom and AHB2 to be similar in performance yet not the Magnius.

Atom is SE in and not balanced out. This would require SE out for Magnius and SE inputs.

IMO either compare balanced or SE and not SE to balanced.
SE out on Magnius (acc. to specs) I would consider bordering on audible. High Gain, SE Output: Less than -72dB at 2V RMS into 32 ohms, CCIF

More thought needs to go in amp selection. You might not be doing yourself a favor comparing to very similar devices if you want $1000.- to go to charity

 

[Pio2001]

Hi,
I am happy to see this blind test being prepared.

I have myself setup a double blind ABX test between headphone amplifiers. Maybe it can help if I talk about the problems we encoutered during our test, and the solution we used.

- Between each run I will leave the room, and my friend will use a coin flip app (which will be shown on camera 2) to choose which way around to connect the two amps. (he will always unplug and replug each run even if its the same config as previous run just to ensure the time between runs is even and there is no possible giveaway that nothing was changed)

I prefer that you don't leave the room otherwise folks would wonder if you are watching the camera stream where the coin flip is occurring. There are a ton of random number generators online. Your proctor could use that to select the next iteration.

You need to simply sit away from him while keeping the headphone on all the time.

Here, during our old test, we had two problems.
First, while sitting in the room, we could perfectly hear what the operator was doing with the connections. We just had the headphones plugged into one amp or the other. But the noise was different. Plugging inside amp1 made "clunk", while plugging inside amp2 made "klang".
Second, while wearing headphones, we could hear the connection inside amp1 thanks to a faint static noise "click" in the driver, while plugging the headphones inside amp2 made no noise.

The solution that we found was this :
The listener was constantly sitting, turning his back to the setup. He could ask to listen to what he wanted (play, pause, play, restart playback, please etc).
Both amps output ports were plugged. One with the headphones, the other with a dummy plug.
Headphones cable was running below a heavy book, so the that listener who is wearing the headphones can't feel the cable moving left or right.

The listener could ask to listen to A, to B or to X at will. For this purpose, the listener would
-Tell "Amplifier A, please"
-Put down the headphones on his lap (to avoid hearing the click)
-Cover his ears with his hands and count aloud "ONE TWO THREE FOUR FIVE" ! We found that this was enough to prevent him to hear the noise of the cables and plugs making "klunk" or "klang". During this time, the operator had to unplug headphones and the dummy plug from Amp1 and Amp2, then plug them back in the required configuration (here, headphone in Amp A, dummy plug in Amp B). The dummy plug is just an extra precaution, as the listener can't hear what's happening anyway while covering his ears and speaking aloud.
-Put back headphones onto his head.

EDIT : I forgot, the operator has to keep the headphone's jack and the dummy jack in hand while switching, because (if we don't cover our ears) we can clearly hear the noise of them being put of the table, and thus recognize which one is being manipulated by ear alone.

I want to make sure we run each test so that it stands alone on its own. That is, you pick one clip and one clip only and experiment is repeated until p< 0.05. If you then want to repeat the test with another clip you can.

Beware that repetition of a given test spoils the p value !

Having taken part in several real-life ABX tests, I know that the number of listeners is difficult to predict. There is always someone wanting to take part at the last moment.
The problem is that the more people take the test, the more likely you can get p < 0.05 at least once among all the experimental data.

It is also true if several headphones are involved (the test fails with headphones A, but succeeds with headphones B) or with several musical tracks (the test fails with track A, but succeeds with track B).
And people are naturally inclined to try another solution to pass the test if it turns out that it is more difficult than expected.

The p value should hold for all the trials of one listener, all headphones and all musical content.
Dealing with several listeners is very difficult : the more listeners are involved, the higher the threshold becomes in order to reach a fixed p value.

 

[Pio2001]

Also, you still haven't adressed a major problem : how to align separately the left and right gain of each amplifier within 0.1 dB ?