@pierre maybe this discussion about the SM_PIR is relevant here since
@amirm is cooking something...
Can you point me towards the modifications you made on the calculation?
Yes. And I need your opinions.
@Maiky76 @kimmosto
Kimmo pointed out to me, and rightly so, that the computation of SM (one of the components of the preference rating) was incorrect when the slope of the curve is close to 0.
I corrected the computation of SM and now it has changed all the scores and i was sure some people would react to it even if variations are small.
I see 2 options:
- keep the corrected formula for SM
- restore the old one since it is what everybody is using
Depending on which formula was used in the original paper, it may make more sense to use the original computation.
From my pov, people are over-indexing on the value of these preference scores. Since it is one of the very few acknowledged studies on what a good loudspeaker should look like, people go for a spinorama but don't spend the time to learn how to interpret each curve. They read the score and they think the higher the better.
ASR did a fantastic job at educating people and it is great if people now think that a CEA2034 is important to look at before buying a speaker.
Maybe a long windy way to say that it does not really matter.
Opinions?
Even recently, I was pointing towards the SM_PIR ambiguity and its issues with at least partially constant directivity speakers, including the 2020 link shared in the post...
Rehash of a discussion I had on DIYaudio.
https://www.audiosciencereview.com/...of-preference-score.72649/page-2#post-2652124
The SM_PIR discussion can only lead to higher level questioning of the model itself as the PIR is the culmination of the model.
We won't settle this
argument discussion here...
Speculatively one way to see things is that the further the DUT is from the “average” of the model the less valid the score will be.
I think the original study may have been somewhat limited by the lack of speakers with reasonably constant directivity (i.e. waveguide-based designs or large driver size or other more exotic arrangements) that are current today. This may have skewed the preference score. As more and more speakers now incorporate some degree of directivity tailoring, this could explain why the scores are not continuing to improve or may even be regressing despite the Spinorama measurements
looking increasingly better. These speakers may simply not fit the model the metric was designed around. They actually do not fit the model, by design.
However commendable
@kimmosto 's work on refining the score calculation may be, I'm not convinced it is supported by solid evidence just yet.
This doesn't make it incorrect, but the burden of proof is not met yet IMO.
Changing the SM would require at least redoing the extraction of the dimensions (ANOVA) with the new SM, but ideally one would perform a new study that includes CD speakers and other current designs with different philosophies, otherwise we might be just calculating "a" score for the sake of doing it but I might be too demanding/rigourous/idealist.
One step beyond
The model limits may or may not fully understood but one of the issues might be the preferred PIR vs the way the model works, the preferred PIR switched from a line with a slope in the model to something different as Harman moved on, see the 3rd edition of the Toole book check the PREFERRED IN-ROOM TARGET FOR LOUDSPEAKER or the Spinorama of e.g. the Revel F228 vs Kef R5
The Revel (Harman) scores lower SM_PIR than the Kef but the Revel exhibits a better compliance to the Preferred PIR.
The Revel also exhibits a more CD (flatish or very gently rising) DI from 300Hz and up.
Note that some auto room EQ have a target resembling the curve mentioned above, Pierre surely knows that. That is not new Tact RCS did it 20 years ago.
Another example are cardioid speakers. Their PIR regression does not fit a sloped linear PIR well although it
looks like it is a better approximation of the preferred PIR. Nonetheless they score lower than many designs that
look inferior.
Unless some modifications are made to the model (and validation) that’s the way it is : the preferred PIR differs from the favorized PIR from the model.
I am not even sure the preferred PIR is "one size fits all". The Room will have its impact, tastes will be factor not only on the response but the presentation itself (live symphonique concert ambiance vs razor sharp, dry, image) and so on… this is why it is a region rather than a single line (and so is the headphone "target").
So why not add a graph of the PIR compared to the Preferred in-room target region?
Would it be better than the score, a replacement, a complement, a counterpoint?
Why changing the SM_PIR is not straight forward?
Very few constant-directivity (CD) speakers that maintain pattern control down to 100 Hz have actually been built, spin-tested, and subjected to rigorous subjective listening trials or, if they have been, the results have not been published.
Intuitively, at the theoretical limit, the model ought to favor a flat PIR (particularly if the slope metric reached 1 for a horizontal line). However, this is by no means guaranteed.
Indeed Earl Geddes, who is as strong an advocate of constant directivity as anyone, noted on the related discussion on DIYaudio that even his "flat‑everything" loudspeaker required the high frequencies to be attenuated to avoid sounding overly bright, a practice he adopted for most of his designs. It is also not an uncommon practice for speakers with CD horns either.
This suggests that, in such cases, a flattish listening window may no longer represent the ideal. Consequently, the model would need to be adjusted – AGAIN - to reflect the preferred downwards LW, even if it goes against intuition and established data… where do one stop?
One may then think maybe “insert something, I choose smoothness” trumps everything else and a better “smoothness” metric would need to be developed... back to square one and speculations.
Remember that most of us are not actually listening to the DUTs, so IMO sticking to metrics whose relevance and limitations are reasonably well understood is probably the safest approach.
How the EQs relate to that
In my view, some speakers must be deliberately voiced (deviating from a flat LW) to compensate for directivity errors.
My optimizer is designed to find the flattest response that maximizes the preference score in the
general case.
Whether it fully achieves this is unclear some designs remain problematic, likely due to score limitations rather than the optimizer itself I suspect, but I am biased….
Still, I've often seen the algorithm converge toward stock tuning for well‑received speakers with directivity errors, suggesting that a PIR SM-defined as smooth does have its merits.
The Score EQ starts from the LW EQ and incorporates the score i.e. the PIR into the process, so it avoids correcting what shouldn't be corrected,
there is no fixed target.
Most speakers end up with a near‑flat ON/LW, though some don't, depending on directivity or EQ aggressiveness (set by LW complexity).
In all cases this not a target that I set but I'd say that speakers with reasonable directivity will tend to end up close to flat ON though.
The resulting LW/PIR/SP slopes resemble those in Olive's study so logical, if somewhat tautological since increasing the score will bring the system towards the model.
Sometimes the results are easy to understand but sometimes the deviation from flatness is so large (or so small) that the interpretation becomes difficult without listening trials.
As for other attributes, I am all for adding additional metrics but which metrics?
And have those metrics been calibrated against subjective preference data, ideally through peer-reviewed research?
A few opinions, for what they're worth:
The CEA2034 contains 2 orders of magnitude more information that the traditional on-axis only response,
when you know what to look for it is enough, but not everyone knows...
THD and IMD as measured, most (if not all) studies seem to indicate that these are not particularly important within reasonable limits. From my experience, THD is useful for telling you when a system is approaching or reaching clipping, but beyond that it doesn't tell you much, especially at LF. The industry's relatively limited interest, as in subjective correlation, in these measurements may also tell us something about their practical relevance. One may argue that 0 is the only acceptable goal that is what the EE guys are aiming with the latest "mine is smaller than yours" contest... Until someone comes with a subjectively correlated metric.
Distortion in general: The GedLee metric, despite its shortcomings, is currently the most advanced approach I am aware of but it tend to show that there is nothing to see.... Listen, Inc. has also recently published a paper on the subject. Perhaps the industry is finally becoming interested in establishing a stronger justification for chasing progressively lower "distortion" numbers and, ultimately, turning that effort into a meaningful marketing advantage. Wait (we've being a lot of that) and see.
Stereo: around 25 years ago, not knowing that other more illustre people also did, I noticed that mono tests were already sufficient for development work, particularly during A/B comparisons. Stereo introduces an additional variable: even small differences in speaker placement (e.g., 20 cm) between two pairs can potentially skew the results. Unless a relevant metric exists (around ITD + ILD? ), I do not see how stereo imaging can be captured praticaly. I have heard terrible speakers but with excellent sound stages.
I would be happy to be proven wrong.
Dynamics: this likely relates to maximum SPL before some threshold. The CEA2010 standard could be seen as an attempt to address this for subwoofers.
Could be used for the LF part of the DUTs?
However, since there is no agreed‑upon metric to define that threshold on the full range, I cannot see how it can be implemented effectively.
I would be happy to be proven wrong.
Remember Olive:
[…]we found that timbre-related attributes accounted for 94% of the variance in comments whereas nonlinear distortion and spatial-related attributes only contributed 3% each. It is our experience that timbre is the dominant factor related to loudspeaker preference, and speakers that accurately reproduce timbre generally have favorable spatial properties.[…]
One must therefore conclude that a reasonably well design source as per the CEA2034 + a DSP should take us a long way regardless of the score... The score might be then used to estimate the EQ "added value" or benefits.
