It is, of course, completely possible to damage loudspeaker drivers and/or crossover components with "too much power". The culprit is
heat.
Whether one
will damage a driver or loudspeaker system (or not) depends on the magnitude of the "over drive" (which could depend on the nature of program material, not just the short- or long-duration power applied to the speakers), the robustness of the loudspeakers, and the capabilities of the amplifier (in terms of "too much" or "too little" capability
).
The old truism is that most loudspeakers will complain audibly before irreversible damage is done -- but tweeters tend towards the fragile and may simply go silent (voice coil burnout) in an instant.
It is also possible for
damage to be done that leaves a driver (e.g., a tweeter) still functional but much decreased in output. Woofers can suffer deformation of the voice coil that results in voice coil dragging or even loss of excursion (and thus of LF response).
I've experienced
all of the above-mentioned phenomena over the decades. Oops.