I'd like to thank you for your earnest replies here, despite the hostile tone present. I'd like to reiterate that I'm not trying to argue, and I apologise for having a hostile tone in my initial message. I have some questions:
Are these null tests being only done for DACs? Are you planning to demonstrate the elements you hear in your review in amp null testing/measurements too?
I apologise too. Hostility is not at all my intention.
I would love to do the same for amps, but have not done so yet simply because I don't have the equipment necessary to do so. A dac can just be connected to the ADC and off you go pretty much. But for an amp you need an appropriate dummy load, and also that then raises questions of amplifier performance with a static vs a true mechanical/reactive load.
I also just have had access to many more dacs than I have amps so that's also a big factor.
If I am in future able to do so then yes I would love to do a similar thing with amplifiers, but at present I don't have the equipment needed.
The review seems a bit contradictory to your desire to find common ground between a subjective and objective viewpoint. My understanding is that the common ground would literally be proof that the subjective element is measurable and audible, against the null hypothesis of it being placebo. It would be illogical to assume there is some mythical element to audio that is immeasurable.
I completely agree. And that would be my end goal, to hopefully have an answer (most likely from someone much more knowledgeable than myself) as to how various subjective aspects are determined from an objective standpoint.
I'm sure much of that we can do now. Its fairly common that devices with more 2nd order harmonics sound subjectively 'warmer' than devices with equal SINAD but predominantly 3rd order harmonics for example. So there is likely plenty of stuff that can be explained with relative ease.
But then i'm sure there are also plenty of things that cannot be explained. Rob Watts from chord for example has said previously that in their own internal testing, they've found that things such as running noise shapers to -300dB accuracy rather than a level such as -150dB that would otherwise be considered "inaudible" regardless, still provides a benefit. How much of this is true I don't know, but it was an interesting remark nonetheless.
Or things like the MScaler. He also says that there is no on-paper reason as to why the MScaler should improve soundstage perception, and yet it does. He himself says he doesn't know why that is, but it does.
I tried this myself, and also even had a couple family members try a blind test, and with ten runs with each of them, there was not a single instance in which any of them did not correctly pick which one had the MScaler enabled.
So I'm sure there are also a lot of things that either cannot be measured with current equipment/technology, or the reasons for why certain differences are perceived may be unexplained for quite some time.
But this goes in two ways. Its from a scientific standpoint equally as incorrect to proclaim something DOES make a difference with 0 evidence as to proclaim it DOESN'T make a difference with 0 evidence. Until such time as proper evidence is provided its all speculation. But one can also make the argument that if you CAN have better, even if you think it might not make a difference, then why not?
To that end, why didn't you include amp null-testing and blind listening tests in the review?
The lack of null test is because as mentioned above I don't have the required dummy load to test it against something else (I also have doubts about static vs mechanical load behaviour on amps).
Do you think your subjective opinion on the Magnius' sound signature will be reinforced with blind listening tests/null-testing?
Yes. And once I am able I'll do that. Blind testing of amps is trickier than DACs though because I have to leave the room for each run (because even just the sound of the connector could tell you which amp is which if you were blindfolded etc). There is greater delay etc.
With DACs its much easier. I volume match the dacs, have my housemate randomise the cables (cover the back with a blanket too) and off I go.
Though the reason i've not included that in the video is quite frankly it doesn't prove anything. Anyone who feels that the dacs sound the same will simply claim i've faked it, I didn't volume match properly, something something. And I'd rather not make a 1-2hr video with uninterrupted footage of me ABX'ing DACs.
It won't be included in the reviews, because for people seeking subjective opinion they'll have no interest in it, and for those wanting absolute objective proof it wouldn't be conclusive or thorough enough no matter what I did.
I will make a standalone video at somepoint though filming the process uninterrupted.
Your review uses many words like "fatiguing", "pushing dynamics forward", "texture", "lower-end timbre", "separation" (in the context of DAC/AMPs)... But to this day I have not seen evidence that these words are physically quantifiable in any way. How would you prove any of the subjective terms here actually exist, separate from placebo, in null testing?
It doesn't. The null test doesn't give any information about how differences relate to subjective findings. It just demonstrates that differences exist.
That's all.
Determining what causes subjective differences is a different challenge entirely and honestly I don't know if it could realistically be done without quite extensive testing with a large number of participants. Which would need the backing of someone with plenty of cash to spend.
Us folk on a forum can only do so much.
I'm sure some companies/manufacturers probably have some hard evidence on such things. But if they do it'd be in their interest to keep that to themselves rather than sharing it with competitors.
Furthermore you state a repeated frustration that ASR's measurements are incomplete. Do you have any academic evidence to show what other measurements may be in need of consideration? I'm particularly interested in the assertion that 'real' music presents a dimension not measured by @amirm's current test suite. Would you be willing to do at least the multitone test in your null testing?
And I stand by that statement. As i've said here and elsewhere. Objective information is very important. But when test conditions are not consistent, the measurements done are not consistent (the fact that the measurements included vary from product to product is questionable), and when there is 0 room for criticism, it makes further discussion impossible.
I'm not going to say any more there because I don't want to tangent onto an argument when there's no need though.
Yep, I'll include a multitone test too. (It will be excluded from a few simply because I don't have the dacs anymore, but for anything I have access to at the moment i'll include)
