Input != output is always measurable, there is nothing that excludes transient effects.
I'm not convinced that simple, static (periodic) harmonic testing (SINAD / THD) identifies 'one-off' transient induced distortions, "AM vs. FM distortions" (John Curl's terminology), feedback-generated transient phase modulation effects (very possible related to ICs with massive loop gain, like 140dB loop gain in some cases), very-high-order-dominant harmonic series (which would make the harmonic generator app earlier in this conversation a very instructive exercise), conventional distortion measurements at lower-than-full-scale operating levels (
i.e., real listening levels far below FS -- we know that THD ratios often worsen at lower levels), the dynamic (vs. static) impact of power supply topology on AB perceptible timbre under a wide range of operating conditions (another AB test we do extensively -- some power supply topologies have detectable artifacts that don't show up in static THD testing -- sometimes as simple as a regulator*), filter performance ('ringing' and time smear fx) and its impact on sonic quality (missed entirely by common THD metrics), and likely many others.
Beyond simple-static THD testing, there are a range of possible distortion vectors, some well-known, some obscure and not widely studied and in some cases (
e.g., old Gerzon papers) bizarre and impenetrable -- which could contribute to the empirical reality that some "higher simple-THD" devices sound more transparent than certain "vanishingly low THD" products that have AB-detectable timbre or spatial issues. The IC test paper that Amir shared even noted this: certain ICs with higher single-reading-THD were deemed more accurate than some with lower THD. We have experienced this over and over and over in 30 years of AB characterization, as has anyone else deeply involved in high quality audio design. Simply ask around -- ask any long-experienced audio design engineer if lower simple-THD readings always correlates with "improved perceptual transparency." (not talking 1% to 0.00001%, but common audio comparisons today).
And .... sigh .... this circles back to my concern with ASR forum. Namely, (1) lower THD does not necessarily mean "better", and (2) -120dB SINAD is not a scientifically irrefutable "guarantee of transparency." The site's owners, if they claim to be scientists, should admit both realities, and make the simple adjustments. They are doing the audio and scientific community a disservice by editorializing personal opinions over fundamental peer-review.
Real science. No personal biases or presumptive aspersions. Professional. And way more fun than "my religion v. your religion" which, frankly we get too much of in the "real world" today and is causing grave psychic damage to our nation and world.
* A year ago, we brought in around ten different SMPS +12V "line lumps" for an AB listening trial with a new product -- in fact the product we're announcing at NAMM next week. More than half of the supplies tested were 8/10'ed rejected. Audible issues. Subtle, yes, but audible. Most people probably wouldn't care or hear the subtleties. We care. And, no, none of these power supplies caused any change to THD, IM, noise, slew, FR, PH, etc.. And, to address all those who say "you didn't really hear any difference" -- I would say, you may be very correct. I don't pretend to be gospel. Let's hope others take the same high road. Thanks.