After analysing Fig. 7.14 I am convinced that, even though there are practical advantages to listening in mono (i.e. a stereo version of Harman speaker shuffler whould have been complicated to produce, as would one which positioned each single or pair of speakers optimally in the room) as well as benefits for critical assessment of particular aspects of performance, speakers should be evaluated for both preference and performance in stereo (and complemented with mono assessment sessions).
Agreed. Listening in mono is much more practical and in most ways much more useful.
But one of the questions I have, as a speaker designer, is WHY did the Quads move up so much in perceived sound quality in stereo, even if their rank remained third out of the three? Was is because of improvements in spatial quality, and in turn was that because it was a dipole?
Spatial quality matters. How much does it matter?
In his book Toole refers to a study done by Wolfgang Klippel (yes, the same Klippel) in which he examined the relative contributions of spatial quality and sound quality to "naturalness" (how realistic the speakers sound), and "pleasantness" (how enjoyable they are).
He found that “naturalness” (realism and accuracy) was 30% related to sound quality (coloration, or the lack thereof); 20% related to tonal balance; and
50% related to the “feeling of space”.
“Pleasantness” (general satisfaction or preference) was 30% related to sound quality and
70% related to the “feeling of space”.
In other words, according to Klippel, the "feeling of space" was
50% of what made speakers sound realistic, and
70% of what made speakers enjoyable! Given that stereo plays a major role in "the feeling of space", imo evaluations which leave it out are incomplete. But they are NOT useless!! They are still extremely valuable, just not the complete picture. (I am not arguing that these percentages are absolutes; only that spatial quality matters enough that it should be fully included in any subjective evaluation which claims to be complete.)
Also, Geddes and I design speakers that are supposed to be aggressively toed-in, like at about 45 degrees. This results in the first significant sidewall reflection being the long bounce off the far side wall. That reflection arrives at the opposite ear, which decreases the interaural cross-correlation, which in turn enhances the perception of spaciousness. Any perceptual benefit that might arise won't show up unless the speakers are evaluated in stereo with the recommended setup geometry.
And a few speakers - like the Polk Stereo Dimensional Arrays - genuinely require stereo operation in order to begin to be fairly evaluated.
Further, I find that it would be more fruitfull if we used Harman's research as a starting point not as the definitive answer to all speaker-related questions. Because it isn't.
Harman's research has generated an enormous amount of extremely useful information, but I draw on other sources as well. And kudos to Dr. Toole for including data at a high enough resolution in his book (such as Figure 7.14) so that alternative conclusions can be contemplated.
The one thing I'm a bit skeptical of is this: Calculating an ironclad metric of merit based entirely on data which has been processed a certain way. Even if there is a strong correlation between controlled listening data and the metric, from what I can tell the database used in the derivation of the metric does not include adequate representation of polydirectional loudspeakers (bipoles, dipoles, omnis, Shahinians, etc.); nor does it adapt to different rooms (a particular type of room acoustic environment is assumed). Notably (in my opinion) absent from Harman's data is the Mirage M1, a bipolar speaker which spent many years in Toole's living room. And the M1 was a highly informed choice on his part.
What was it about the M1 that made it stand out from its competitors? And how much of a role did its bipolar radiation pattern play? Harman isn't really going to tell us, so we have to look elsewhere, assuming this is a question which matters. Imo it does... was the Mirage M1 an outstanding execution of a fundamentally flawed concept, or did the concept have merit then, and if so, does it (or some evolution thereof) have merit now?