Your incredulity is understandable and would be justified if the study was a medicine study made on random people with totally different biology and lifestyle judging on 1 parameter at 1 moment, withouthere you need a lot of participants for it to be relevant because of all the unintended parameters which will generate high statistical noise.
Here it's a different kind of study, first because of the panel selection, then the number of factors tested, the nature of the factors themselves, the repetition of the tests that can become quickly very high, all of this happening in an environment that is higly controlled: all of that increases significantly the statistical relevance of such studies, even with a small panel.
To make an analogy it's a bit like you were able to test 200 medicines on 16 people with the exact same biology and lifestyle in a single day, it's a bit of a gross analogy but that's the best I got.
More broadly, whatever the subject, the difference between subjectivity and objectivity is made by the number of parameters you'll manage to be conscious of and you'll try to control (and you'll always be partly subjective because you can't be conscious of all the parameters).
Here in these studies the control is very high compared to medicine studies, which have little control for practical reasons.
(Sorry for my broken english, hope you got it).
What's even more remarkable, in my opinion, is that these were audio professionals, for whom sensitivity to lateral energy might have been assumed to be greater than the general population of listeners.
From Toole, second edition of Sound Reproduction: "
Why do recording and mixing engineers prefer to listen with reduced lateral reflections (higher IACC)? Perhaps they need to hear things that recreational listeners don’t. This is a popular explanation, and it sounds reasonable, but experiments reported in Section 6.2 indicate that we humans have a remarkable ability to hear what is in a recording in spite of room reflections—lots of them. But there is an alternative explanation, based on the observation that some listeners can become sensitized to these sounds and hear them in an exaggerated form. Ando et al. (2000) found that musicians judge reflections to be about seven times greater than ordinary listeners, meaning that they derive a satisfying amount of spaciousness from reflections at a much lower sound level than ordinary folk: “Musicians prefer weaker amplitudes than listeners do.” It is logical to think that this might apply to recording professionals as well, perhaps even more so, because they create artificial reflections electronically and manipulate them at will while listening to the effects. There can be no better opportunity for training and/or adaptation. In fact, it is entirely reasonable to think that acousticians who spend much of their lives moving around in rooms while listening to revealing test signals can become sensitized to aspects of sound fi elds that ordinary listeners blithely ignore. This is a caution to all of us who work in the field of audio and acoustics. Our preferences may reflect accumulated biases and therefore may not be the same as those of our customers."
So, on the one hand,
@Karmacoma seems to be suggesting that the background of this
pilot (suggesting that it hasn't been followed up in terms of verification or falsifiability, which presumably are important to the scientific method, especially when consider the rate at which the results if published initial studies may not be verifiable in follow-up) study population should replace larger N, in a similar way that Harman uses a smaller pool of trained listeners instead of a much larger untrained group for many investigations. I actually have no objection to this. I previously brought up "absolute" or "perfect" pitch in terms of what seems to be a spectrum of ability to identify tone frequency without external reference as an example of how testing with smaller numbers could lead to fallacious conclusions.. It seems as though the prevalence is much higher in trained musicians compared with the general population, so while it would be trivial to run an investigation and "prove" that perfect pitch doesn't exist using a few handfuls of general population listeners, it would be a much different matter in musicians.
On the other hand, given the characteristics of recording professionals described by Toole above, it might actually be MORE surprising that an even greater proportion of this study population did not prefer attenuated lateral energy from diffusion or absorption over reflection. However, perhaps it might be noted that this preference may have been expressed with respect to a specific cognitive task, as opposed to, for example., preference for recreational listening. But yes, the error bars are huge. Wait until you really consider the error bars for the so-called Harman Harman curve for speakers in rooms!
However, I still find it surprising that Toole shows the JBL Array 1400 and Revel Salon 2 spinoramas in figure 18.17 in the second edition of Sound Reproduction and writes "
When they are put against each other in double-blind tests, the audible differences are small, somewhat program dependent, and listener ratings tend to vary slightly and randomly around a high number. In the end there may be no absolute winner that is revealed with any statistical confidence; the differences in opinion are of the same size as those that could occur by chance." What does that suggest about the limitations imposed on this trained listening panel with respect to the cognitive task at hand, the listening setup and environment, and the listening material/program, also about the so-called Olive preference scores?
Wait, I guess this could be summarized: "Research from Harman shows that listeners can't actually determine a preference between the JBL 1400 and the Revel Salon 2," but that would be wrong.
Anyway, maybe it would still be reasonable to consider acknowledging ambiguity and nuance; avoid oversimplifying; "quoting" instead of inaccurately paraphrasing; and posting links and references whenever possible to contribute to a more meaningful discussion and exchange of ideas.
Also, going back to the original post, I was curious what hysteresis distortion sounds like. I found this link, which includes demonstrations at the bottom:
https://jatinchowdhury18.medium.com/complex-nonlinearities-episode-3-hysteresis-fdeb2cd3e3f6. "Crunchy" is actually a reasonable description, in my opinion