I fully expect that nobody will agree with my views on B&W, given the Harman focus of ASR. Of course, I currently or previously owned speakers from Revel, Genelec, and KEF, in addition to B&W, but never mind that.
I'm so glad that you mention the CDM1. I happen to own a pair and listened to them pretty regularly for about 10 years in 5 different rooms, so I'm pretty familiar with them. And I also have a pair of 802Ds, which I've listened to for about 8 years in 2 different rooms. And I recently did a very brief comparison between the two in the same room, just to refresh my memory.
If you were to believe these FR charts, you would easily conclude that the CDM1, which its ultra-flat response would easily be superior to the 802D, other than the bass response. But I can tell you that without a doubt, there is absolutely no way anyone could possibly prefer the sound of the CDM1 over the 802D, it's just not even in the same ballpark.
People can stare at FR graphs all day and pretend that they have "a gift" and can reliably predict how speakers will sound based on eyeballing them (and btw, these aren't even full spinoramas that you're posting). But the reality is that Harman's own research demonstrates that computerized analysis of a full spin measurements can only explain 74% of the variability in listener preference scores.
And spending time on ASR might help to reinforce that belief. People who believe similar tend to migrate here. And I started to believe it too, until I put my money where my mouth was and purchased the Revel m126be and Genelec 8351b, both of which were touted to have near perfect Harman qualities when measured and NEITHER outperformed my trusty B&W 802Ds downstairs. But to each his own.
I get your point that you shouldn't dismiss a speaker just because it doesn't measure 'harmanesque.' I don't necessarily disagree.
It's surely true that being on ASR can reinforce one way of thinking, though I personally have not shied away about mentioning times I've enjoyed speakers that don't measure perfect (the JBL L100 classic is one of my favorites, for instance). Sometimes that's led me to some insights about what kind of deviations from 'perfect' I like (as some here know, I'll take wide directivity, even if somewhat uneven, over 'perfect' narrow directivity any day).
In any case, I think we need to be pretty realistic about comparisons, and that's why I clarified things like 'timbral neutrality' vs preference. I don't expect small old monitors to best newer floorstanders, and I don't think most people here would either, for various reasons.
I'd like to highlight the distinction between figuring out "preference" and "what a speaker will sound like" from measurements.
I do think it can be hard to distinguish between which of two speakers someone will prefer just by eyeballing them, unless the differences are dramatic. However I disagree that one can't substantially determine what a speaker "will sound like" from looking at a few key measurements, especially when one is aware of how their own tastes correlate with the data.
Those are two different things. You reference the the famous olive preference paper (part 2), but for the most part all it tries to do is predict
preference from measurements -- it says very little about what listeners thought the speakers sounded like from these measurements.
It is from other research (including but certainly not limited to the oft-ignored part 1), that we know measurements can be largely associated with different aspects of sound quality, and that flattish on-axis and smooth directivity are generally perceived as neutral. (Maybe you've read it, but reiterating here for the sake of the wider discussion). And then Part 2 tells us that it just so happens most people prefer speakers that measure like that. But it doesn't imply "you can only predict what a speaker will sound like with 74% certainty."
By comparison, Part 1 shows both the measured spinorama and the perceived tonal balance, as well as top listening comments. So even though it's the smaller study, I find it much more interesting. The comments, and perceived tonal balances described, to me seem utterly predictable from the spinorama. Here was the top speaker:
Here was a the #9 ranked speaker:
And here's #10, one that's interesting because it's a bit more neutral but, still not perceived as great. The main negative comment was instead about imaging.
I do not think it's outlandish that someone can look at a spinorama and detailed off-axis data, and make a reasonable conclusion about what a speaker will sound like. It's just determining whether one speaker will be preferred over another for a specific listener in a specific home while those sound qualities that things gets tricky.
Anecdotally, I've almost never been surprised by how a speaker measured after extensive listening, nor surprised by the sound after seeing measurements. For the most part, it's pretty straightforward. There are exceptions, but in my experience that just proves the rule.
I suppose you might counter that the above examples are just broad trends and that it's hard to tell the subtleties from just eyeballing a measurement, not enough to make a decision among some high end speakers. It'd be an interesting experiment: get together a group of measurements-familiar audiophiles, ask them to predict sound based on measurements without telling them which speaker is which. Then make them listen to the speakers blind and match the sound with the measurements.