I am always baffled and amused by the deliberate overlapping of disjoint measures we use in audio. On the one side we have "objectivists", who are determined to achieve absolute fidelity, be it in a microphone transducing acoustic energy into voltage/current, an amplifier, which gives you more of the same thing, or a loudspeaker, transducing voltage/current into sound pressure (etc.). On the other side we have subjectivists who primarily care about how the final sound "sounds" to them. The problem is that the measures of "goodness" in the two cases are drastically different, both in kind and in the way one does the measuring. They do have a tiny overlap, but they are mostly orthogonal.
On the "objectivist" side the primary goal is fidelity which can be measured in numerous ways, some of which are easy and common, others not as easy and more rare. The benefit is that anybody doing them can replicate what the others have done with a very high probability of obtaining the same results. This is all part of the scientific process and the backbone of technological progress. Part of these evaluations are also subjective evaluations. However, in order to satisfy the repeatability condition, given that they are based on statistical analyses, it necessitates double blind testing on a large population with a focused set of objectives. We all know about people and their biases. Removal of those is part of those requirements, too.
On the "subjectivist" side the primary goal is to find out how much "satisfaction" one gets from listening to reproduced sound under some predefined conditions. I call it "pleasure", other terms can be applied with equal success. Since most of the time those are centered around a specific reproduction chain in a specific acoustic environment, at a specific time, anything can throw it off, making it impossible to be made repeatable to the level which satisfies scientific rigor. I am not implying that it is the wrong thing to do, just that it is different and not nearly as useful to others, if at all.
Here are a couple of specific experiences touching on both of these sides of the coin. I recently built a room for enjoying sound reproduction, equipped it with the best equipment I was willing to pay for, and was extremely happy with the sound. I did a lot of measurements and adjustments over time. However, one thing I noticed was that the same track of music could elicit a very different emotional response from me at different times. Sometimes it would amaze me how faithful and realistic the reproduction was, and at others I would be disappointed, expecting the same elation as some of the previous times. All I can come up is that I changed, as nothing else had! It made me doubt (even more) my ability to subjectively evaluate sound reproduction "quality" (in quotes, as it means different things to different people - it is imperative that we define the metric when stating that something is "better").
At another time, several years earlier, I was using a Behringer digital crossover which I found to be a terrifically useful piece of equipment. I had doubts about it's ability in the sense of fidelity (ie. SINAD etc.) but no real evidence. I was not equipped to measure it, but I could conduct a subjective test. As close to double blind as we could mke it, with changes between stimuli talking only several seconds. I collected half a dozen audiofile friends, came up with a varied list of good recordings, generated a random list to use in the experiments, and decided to just look for the ability to determine if there was any perceived difference in sound, depending on whether the device was in the signal path or not. (four types of stimuli: yes-yes, yes-no, no-yes, and no-no. Question was: are the segments the same or not. It took us some time to find the optimal length for the stimuli. Very short, so that you can remember it - not enough information. Very long, and you could not remember the first version. About 20 seconds worked for us. Ultimately, the listeners' choice (same or different) was completely random. And post-experiment comments mostly centered on: They sounded the same, but we KNEW that some were going to be different so we took that option every once in a while.)
The lesson I learned is that most electronics is so good these days that the burden for achieving good reproduction (both faithful to the original performance at the place of the microphone (objectivist) and satisfying my "pleasure" parts of the brain (subjectivist) - admittedly those get triggered in my brain if the objectivist targets are met
fall on the energy trunsduction components (speakers) and the acoustics of the listening room.
I always measure, and always listen. It is rare that the two disagree, and if they do, after some investigation, I usually learn something new, useful and non-contradictory. Perhaps if we all tried to put ourselves into the other camp's shoes, we could stop bickering. But more than anything else, please define the metric before defining the performance!-) It would go a long way towards promoting harmony