So what is it we are hearing when we hear 'distortion' then? IMD or what....you can see how someone new to all this would be confused. I know I am, and I'll have you know I'm no longer new, I've been promoted to major contributor recently! Though, I may still try being addicted to fun, learning hmm....not so much.
First of all, HD measurements are not completely useless, you just have to be careful when interpreting them.
If we focus only on harmonic distortion, then we can give a theoretical lowest listening threshold given by masking.
The masking limits are mean values and can be individual different.
So we are not asking how harmonic distortion affecting the sound (which has not really been clarified), but below which threshold we can exclude that they affect the sound.
Any speaker whose HD is (well) below the masking curve is virtually perfect (in terms of HD) and produces no audible HD.
The following is a hypothetical assessment:
There are many ways in which tones or HD can be masked: The masker can be white noise, music, a wide or narrow band noise or just a single tone (ordered from best to worst masker).
The worst masker is a single tone (or very narrow-band noise). This could be a piano keystroke, for example (this still has harmonics, of course, but this would complicate our theoretical assessment considerably, so we ignore them***).
Here in the example spectrum of a piano keystroke at C6 1kHz (it's
Heather Pierson - When You Wish Upon A Star) you can see that it occurs almost as a pure tone (first and second overtones are already heavily damped, about -40dB and -45dB, no other overtones).
So next we consider how much a single tone (our masker) at 1kHz masks the nearby frequencies. It is important to note that masking is dependent on the sound pressure level (the higher the sound pressure level, the better the masking):
Source: Zwicker, Fastl - Psychoacoustics
In our case, the test tones to be masked from the 1kHz piano stroke is the generated HD2 (at 2kHz), HD3 (at 3kHz), ... HD5 (at 5kHz)... by the speaker.
The gray lines show to which limit HD2 is masked at a masker sound pressure level of 80dB (e.g. piano keystroke at 80dB):
up to -37dB attenuation, i.e. about 1.4% HD2
The blue lines show to which limit HD5 is masked at a masker sound pressure level of 80dB:
up to -60dB attenuation, i.e. about 0.1% HD5
This is the reason why it is said as a rule of thumb that the higher the order of the HD, the more problematic they are.
If one now carries out these "considerations" for different frequencies, one obtains the following for the masking limits for HD2 at different sound pressure levels:
This is the reason why it is said that HD is much less audible at low (and very high) frequencies than around 1kHz.
Any loudspeaker whose HD2 remains below these audibility limits is very unlikely to have any audible effect (but see ***). Now repeat this for HD3...HD7...
In case it's not entirely clear what I mean by that, I've babbled about it before:
In an interview with Earl Geddes, @hardisj uncovered this notable insight on how masking completely renders THD nearly pointless at higher SPLs as high listening volume significantly masks distortion to where it's inaudible (I've linked this specific segment below). This is significant because...
www.audiosciencereview.com
I'm not sure ... but masking thresholds are well-researched and documented in the field of psychoacoustics (which fwiw I am not in). There's not a lot of mystery left. So, the mystery to me is simply that us should rely on a hobbyists attempt to derive the said thresholds from somewhere else...
www.audiosciencereview.com
Might be why tweeters still seem to sound fine even with relatively high levels of 2nd-order distortion, f1+f2 is high enough that we don't care too much and f1-f2 is more or less outside their reproduction range. Agree wholeheartedly with your post, bar this point :p E.g: 2kHz + 3kHz = 5kHz...
www.audiosciencereview.com
*** Since HD2 corresponds to the first overtone, HD3 to the second overtone, and so on, it is clear why these greatly complicate our hypothetical assessment. One might ask at what HD portion "added" to the harmonics a tonal change is perceptible.