In my experience I have tested dozens of stereo power amps and preamps over the past few years, solid state and tubes based, from $500 to over $15,000 models (and not mine). I also help some local builders to fine-tune their handcrafted tube amplifiers.
I perform both classic measurements (THD, IMD, DR and many others) and listening tests, the latter usually with more people. The environment is always the same, well treated acoustically, the levels are controlled, even if not with high precision. Listening is not blind for logistical reasons, but it also true that we often start tests without particular expectations of what we are listening to.
On the basis of this personal experience, I can say that:
- Expensive devices are not always synonymous of an excellent sound; however, well sounding ones are very often expensive.
- Devices that result in very good measurements do not always turn out to be better for listening than those with less good measurements.
On the first point I think there is enough agreement from common experience.
The second point I think is the main source of discrepancy, as there is no agreement on what "good" measures and "good" ear sound mean. This because in between there is the psychoacoustics: how we, as human, ear sounds.
An interesting (on line) book that can help to better understand the relationship between these is "Premium Home Theater - Design & Construction" by Earl Geddes & Lidia Lee, where it is reported verbatim about the distortion of audio components (page 58):
- There is virtually no correlation between Total Harmonic Distortion (THD) or Intermodulation Distortion (IMD) measurements of a system and the subjective impression of the sound quality of that system. The correlations were weak, but most shockingly they were negative—according to these tests people liked THD distortion. This is actually somewhat true in general that people prefer some forms of distortion to no distortion.
- Signal based distortion measurements (THD, IMD, MTD), which are based on a purely mathematical formula which does not take into account the characteristics of human hearing, do not hold out much hope for ever being an accurate measure of subjective impression.
- Measures need to be based on the actual nonlinear characteristic of the system and scaled to account for the human hearing system.
- THD, IMD, MTD are symptoms of the root problem — the system nonlinearity. These symptoms manifest themselves in different ways depending on the type of nonlinearity present in the system which causes them to be system dependent. This system dependency prevents these symptoms from being valid guides to the audibility of the underlying distortion.
- Sharp changes in the linearity transfer function, things like hard clipping and crossover distortion, have the most detrimental effect on sound quality. Gradual changes such as soft clipping have a minimal effect.
The above statements, the result of decades of work, imply that the measurements that are still carried out today are certainly a guide for understanding many aspects of the audio component behaviour and then useful for detecting macroscopic defects (e.g., if it is noisy there is very little to question). But the measurements alone, once "minimum" quality values are achieved, are unfortunatelly not sufficient to judge how that audio component behaves when listening for parameters such as spaciousness, location and timbre (in other words, the listening pleasure that we all seek), on which a definitive point will probably never be placed.
