You are talking about single wide band systems such as amplifiers. That kind of simplified assumption is not whole truth with multiway loudspeakers having both mechanical and acoustical links between different ways/bands.
We listen to the whole speaker which acts as a system. It cannot get past the basic mathematics that says the distortion it applies to one tone, is the same as it does to two tones. Mathematically you can drive the IMD from THD (with phase) of individual tones.
The thing IMD does which was useful in the days we did not have FFT and such, is that it simultaneously tests a device at a low and high frequency. With a single tone test we don't have this. So this is why it became popular. It is a myth that it produces new data. It cannot.
Already mentioned few mechanisms such as cone of conventional hifi coaxial (Kef, Seas etc) which acts like wave guide vibrator causing IMD products at HF. That is completely different than result of non-linearities of individual radiators, and can be higher and more audible than harmonics - especially in 2-way applications. PA coaxials usually have some kind of wave guide for tweeter which isolates wavefront of tweeter from vibrating cone so sound clarity could be significantöy better than with hifi coaxial.
Said with a wet thumb in the air to detect air temperature. You want to tell me what is audible, present psychoacoustic research or controlled listening tests. Don't tell me a perceptually blind test like IMD tells you this and that. It does not at all by itself. You have to decompose the problem, apply masking and threshold of hearing to arrive at possibility of audible problems. Here is an example for you:
SubwooferPerformancefor Accurate Reproduction of Music*
LOUIS D. FIELDER
Dolby Laboratories, /nc., San Francisco, CA 94103, USA
AND
ERIC M. BENJAMIN
On Audio Eng. Soc., Vol.36,No.6, 1988 June (peer reviewed paper):
Now this research was about IMD distortion of subwoofers not whole speaker. The point is to show you that there is a proper way to make an argument in the forum, and then there is the hand waiving one that says, "I know more than you so shut up and listen." That would be what N. Korea dictators might do.
Another already mentioned mechanism is pressure leak to rear side of the radiator. Leak biases operating point of the motor causing intermodulation products. This happens if designer assumes that tweeter is sealed while it's actually not (enough). This is not all.
Measurement as harmonics is also limited to bandwidth of output port/stage so you cannot measure non-linearities as harmonics in acoustical domain above ca. cone break-up freq. / 2 without multi-tone.
You are talking about old measurement systems. We can measure harmonic distortion well past the audible limit with high sample rate ADC. And indeed I do this now. It is not needed in real life though as the energy at 20 kHz is so low in music that what intermodulation or harmonic distortion it generates is of no value. Test tones are run at full amplitude which is not at all representative of what happens in reality. We keep the volume down in these tests or we would instantly blow up the tweeter! Such of course doesn't happen with real music.
Let me finish by saying any measurement has value if you are designing something and are chasing a specific problem. Indeed often you create custom measurements for certain problem. In that regard, if you are troubleshooting some problem like you mention, maybe IMD output is more intuitive to you than THD graph. Indeed, the "ESS DAC IMD Hum" problem I found with DACs was far more easily visible in IMD graph. Later however, I managed to show the same with THD although it took some work. So I am not dismissive of these tests in general but rather, in the context of doing a review and measurements being compared across a range of speakers.
I actually tried to quantify the IMD "problem" with coaxial drivers but could not. Whether it was due to large variations between coaxial speaker and another, noise, etc., it simply did not turn out that the measurements showed a problem there. Even if it had, then the next challenge would have been to show audibility of such which would be super challenging.
As you see, the complexity of the topic is fully understood. Your view is simply too myopic to have value in the context of what we do here.