But come on now. Distortion in speakers can definitely be, or become audible. For example :
This is a review and detailed measurements of the GR Research "Little Giant Killer" (LGK) 2.0 bookshelf speaker. It comes in various forms from a kit ($313) to fully finished pair ($1,038). Our DIY expert @Rick Sykora built my sample using the flatpack ($100): Rick had to do the round over on...
audiosciencereview.com
Or take a clock radio, speakers on the TV. Turn up the volume really high and it sounds crap. Everyone can hear this, even those who do not into HiFi.
(It may be the small amplifier that in these cases that is driven into clipping.Which is the reason for the lousy sound, I should add)
Sensible HiFi speakers that have decent power, can handle decent power, then it will be trickier to hear, compared to the examples above.
By the way. Speakers can have straight FR with high, relatively high, distortion and vice versa. Uneven FR but low distortion.
I think that what matters, on a descending scale, begins with the most important thing:
FR
Directivity, dispersion
Distortion
Besides, aren't there slightly different opinions about this with first reflexes? How much you should dampen them. Though it might be a matter of taste?
Edit:
By the way, if you do not need to worry about distortion in speakers, why do so many people care about SINAD here at ASR when it comes to amplifiers and especially DACs? The little they add to the sound chain compared to the speakers is in this context then more or less negligible.Compare that to what the speakers add in the form of distortion.