Are you suggesting that to determine whether something is scientific is merely to argue semantics,
@DMill ? If so, it wouldn’t be opinion for me to tell you such a stance is in error.
Measurements as collected by and shown on ASR are
data. How data are collected and used can constitute science, or not. Often on ASR, things skew to the “not science” camp.
Thanks,
@GXAlan . Yes. A common misconception is that “science proves…”. Science does not prove. Ever. By design it simply can’t. Science operates by testing hypotheses and either demonstrating
disproof for, or showing a lack of support to refute, a given hypothesis. A scientific test will have at least two competing hypotheses. Anyone who doesn’t understand the basics of hypothesis testing cannot understand science, but never fear - the internet is a big place and can be your friend, even if ASR omits standardized tutorials on the matter and doesn’t clearly state much of what’s needed for a process to be scientific, in many of (any of? ) the reviews.
An important weakness is how it’s not been rigorously tested, whether “transparent electronics” or speaker “preference scores” hold up at large, as those studies have been quite limited in scope. This is probably why some engineers roll their eyes over spins, or at least over non-engineer “common folk” eyeballing such graphics trusting they can predict more complex parameters of speakers based on the information at hand.
Scientific methodology employed in engineering can be straightforward; in behavioral studies on humans it’s… a different kind of complicated. So before anyone grabs pitchforks and torches with eyes on my user account, please realize that claiming studies are limited or weak is not the same as claiming they’re worthless, especially if they constitute the best available distillation(s) of a given topic for the time being. But it’s important that inferences made from those studies’ extrapolations are hedged on the
obvious relevant constraints; this is something I think is oft-lacking on ASR - the grain of salt disclaimer.
Yes again. Science is generally collaborative; rarely if ever a one man show. Lack of teamwork introduces more chance for error and eliminates peer input as part of the process. Also, sample sizes consistently of
n = 1 can be problematic if (e.g.) settings, tolerances or QC on kit are off (e.g. ASR’s Wavelet “review” ), and are inherently flawed if trying to extrapolate a broad generality from, say, one model. Sample size of 1 for listening impressions is scientifically inapplicable: if not part of a larger data set, it gets you uncontrolled bias, etc.
Now, if a speaker doesn’t measure well but gets a high preference score, then it does refute a hypothesis of good measurements = good sound. The converse (Ha: poor measurements = poor sound) would also be true. Outliers can be paradigm-killers, depending on the hypotheses being tested. What that’s (likely) telling us is that some aspect of speaker performance that we aren’t / cannot presently assess, perhaps complex interaction effect(s) of > 1 property, is / are responsible for a significant portion of the variance in preference scores.
This is why a study needs to be statistically robust: without testing the correct parameters on a large enough portion of the target “population,” it remains problematic if not impossible to meaningfully demonstrate anything more than correlation. That in turn beckons the phrase “correlation does not imply causation,” and then you’re back at the top of the scientific process flow chart. Try again.
Agreed. Sadly there’s no uniform property of the universe that cedes every electrical and / or mechanical aspect of human-engineered products to be tested with equal rigor and precision at the same point in time. What an inconvenience!
The elephant in this room is how little commercial hifi design vs. full-fledged behavioral (consumer preference) studies have been integrated; those fields are somewhat non-overlapping and I’m guessing funding for such collaborations from NIH, NSF and surely smaller sources must be in short supply.
To loop this gyre back into the thread, when those trial speakers arrive
@MKR , listen to them
without subwoofers first. Doing the opposite will not be an optimal approach, scientifically-speaking.
Alright fellas - I’m off - late for debate class again…