With what argument? That outdoor speakers were meant to be mounted and used differently, and what you are accustomed to from CEA2034 interpretation is only calibrated to one specific scenario (indoors)? How is it more fair to interpret measurements differently/incorrectly vs the scenario in which the speakers are actually used?
If the goal is to correlate the Olive score with subjective impressions, those impressions should be made under the same general conditions of the double blind tests the preference formula was devised from (indoors, not flush against a wall). A speaker sounding good yet scoring poorly when used way outside this scope does not invalidate Olive's model; it is in fact an
invalid test of the model. If the goal however is to give one person's uncontrolled, sighted subjective impression of how a particular speaker sounds in its intended set-up, that's fine, but it's nothing more than that.
If I understand Amir correctly, he’s saying that his subjective reviews will always be honest. If it sounds great and measures poorly, it’s better for him to report this honestly than to lie (i.e. to bias his subjective review with any particular expectation from the measurements), which it seems some of you are asking.
No-one's asking for that. In fact, the opposite was suggested - to listen to the speaker before looking at the measurements, in order to
remove possible measurement expectation bias, which is just one of the many cognitive biases sighted listening tests are subject to, that Dr. Olive and his team eliminated by doing them double-blind.
It’s interesting to me the range of reactions when people’s beloved preference ratings don’t match subjective outcome: some (like me) are excited and energized, because this means there is potential for the objective science to improve. Others are dismayed or uncomfortable with the idea that science may need to adapt to new observations — preferring instead to grasp to comfortable old ways of modeling reality — with such extreme velocity (almost like a defense mechanism or immune response) that some demand that direct observational evidence (subjective listening tests) be changed to match their expectations, not vice versa!
Again, no-one's said that. (Did you mean ferocity by the way, not velocity? If so, I haven't seen anyone being 'ferocious' in this thread at all - just calm, rational arguments, at least up until this one of yours.)
I regularly see both mindsets in science and engineering (where I work), and I can promise you: the former mindset is the only one that yields genuine innovation and progress.
This is not a question of engineering, it's a question of science. Engineering is concerned with getting something to 'just work' given a set of objectives, specifications and constraints. Science is concerned with uncovering truth and actual causal mechanisms via a robust method proven over centuries to weed out low quality and circumstantial evidence that can mislead to false conclusions.
And in scientific philosophy, the latter mindset (unreasonable attachment to the status quo) is the blatantly incorrect mindset to hold here. It is a deeply unscientific worldview: that the existing status quo must be preserved at almost all costs, no matter what new evidence says (short of demanding a body of new evidence of a size that is completely unreasonable to demand of new science).
So much irony. When many of you read the old triumphant (and sometimes not) stories of progress from scientists hundreds of years ago, who were given an unfairly hard time by the traditionalists of their era (Galileo, even Einstein, and many others of course), I bet you laugh at the foolishness of those too stubborn and set in their ways to accept the possibility of scientific change and progress prior to the accumulation of undeniably overwhelming evidence conclusively establishing the new theory as the new canon. You probably rightly see those old fashioned traditionalists as being too stubborn and set in their ways to even entertain the possibility of a better model of reality. You probably realize that their demands for extreme levels of proof before even considering the new theory (if not outright rejection or change ever being possible) are unreasonable for a new theory, and accomplishes nothing but obstruction of the new science and the obsolescence of those unwilling to go with the flow of new science.
Yet I am not sure if you realize that you are exactly like all of those people throughout history who resisted scientific change. And no, you don’t get to retort “I’m open to change, I just need to see an equal weight of evidence as the old theory had”. If that were the criteria for new science, then nothing would ever change — because old ideas and old models will always have the benefit of age contributing to their greater set of evidence. It is unreasonable to demand that any new way of modeling reality be supported by more evidence than prior theories which have enjoyed the luxury of time to cement itself in ways new science can never possibly compare until it is no longer new science and becomes the new status quo.
Your depiction of both people's attitudes in this thread and the scientific process is not remotely accurate. Yes, we do get to use the phrase I've bolded, because that is one of the basic tenets of science - in order to invalidate a theory, at least the same standard of methodology must be used to the theory under question. If this wasn't the case, all the sciences would be an utter mess of speculative theories holding the same weight as those with mountains of hard methodological evidence behind them. In medicine, drugs would be approved left right and center without being properly trialed in double-blind tests to rule out irrelevant cofactors such as cognitive bias / placebo; homeopathy would be classed as science instead of pseudoscience; the standard model of particle physics would include hundreds of speculative elementary particles instead of the seventeen that have been confirmed to exist etc. etc. In fact experimental particle physics is rife with examples of evidence that has prima facie looked strong for the existence of new particles, reaching a confidence level of 3-sigma (99.7%), yet subsequently found to be mere statistical blips. This is precisely why they even do
blind statistical analysis and require 5-sigma confidence for evidence to qualify as a scientific discovery. Even
6-sigma events have turned out to be false due to systematic errors, which was realised in part due to the mountain of 'status quo' evidence contrary to the result. As this erroneous result shows, poor data does
not necessarily cumulatively add to the body of scientific knowledge; in fact it can be damaging in that it will lead us to false conclusions, if left unchecked and not subject to strict scientific standards. And the likely reason they initially overlooked this relatively basic systematic error involving faulty cabling? They got carried away with the prospect of making the discovery of the century that would have turned their field on its head and disproved Einstein. Humility is one of the greatest assets in science, and hubris one of the most dangerous vices. For every Galileo and Einstein, there are countless people holding speculative ideas that have not borne confirmation through scientific evidence - your argument suffers from severe selection bias. Of course, I'm not saying audio science should necessarily be held to the same high standards as particle physics, but a scientific methodology of the same standard as the theory being questioned should at least be attempted if any claims of the latter being invalidated are to be made. This does not mean casual observations necessarily have zero utility, but that utility can at best only be suggestive of avenues to properly investigate scientifically, and is in no way science itself.
So as far as who is "doing science", I'd say you should look inward before ironically criticizing the operator of AudioScienceReview.com of not doing science.
