All amps have constraints in their design, typically the power supply is undersized for the maximum power that the amplifier circuits can put out, hence we have a difference between rated continuous and peak power output. (Heat dissipation capability will also impact on the continuous to peak difference depending on design)
When looking at AVR's where there may be anywhere from 5 to 11 powered channels it gets worse (often a LOT worse) - and the full power output is typically only available when driving 2 channels, as soon as the driven channel count increases beyond that, you start to hit the constraints of the power supply in most cases. (yes there are edge case flagship models that are less constrained)
So in my case, using an Integra DRX3.4 ... rated at 100W@8ohm/channel - it was needed to drive my 5.1.4 setup, the front L, C, R speakers are very hard to drive Gallo speakers with an impedance that drops down to 1.6ohm - surrounds are B&O Penta's with a much higher SPL efficiency as well as a benign 8ohm impedance rating, heights are Gallo Nucles Micro's, not overl SPL efficient but again, with a benign 8ohm nominal impedance rating.
Calculating MLP requirements for my preferred listening level (72db continuous, 92db peaks) shows a requirement around 3.5W continuous and no more than 16W peak (@8ohm... double that at 4ohm, 4x at 2ohm etc...) .... which is to say, contrary to many people's beliefs, the power requirements are relatively modest.
However my experience was, that when setting up my system and trying to drive the setup using the AVR alone, the midrange frequencies, vocal area, sounded confused, imaging collapsed.... it wasn't right! (my previous flagship Integra DTR70.4 did not have this problem - but it also had a massive power supply!) - so I plugged in my external amp to handle the front L/R speakers - leaving the critical centre on the AVR as well as surrounds and heights.
The improvement was immediately noticeable.
Removing that load, left sufficient spare power supply capacity for it to handle the just as difficult center channel (also 1.6ohm) - while also handling surrounds and heights.... the clarity was back, vocals instantly became much more understandable, and the imaging and soundstage were back the way they should be.
In the end, I played safe, and provided myself with additional headroom, by running all 3 fronts L/C/R off external power amps appropriately capable of driving very difficult loads - and I have left all the other speakers on the AVR's internal amps.
My personal analysis is that the power supply is far too current constrained to handle those speakers... and once it starts to run out of current, you get a rise in distortion and immediate audible impact. (impact will differ depending on the specifics of the individual models design)
The problem with most AVR's is that they all have this issue. - But if your speakers present the AVR with a benign 8ohm load and reasonable efficiency - then the issue may never be encountered! (and if this is the case for a particular setup, then adding external power amps, will have no audible impact! - other than potential placebo effects)
Current AVR's that have "beefy" power supplies include the Onkyo/Integra RZ70/DRX8.4, Denon A1H... but all the models below those flagships in those brand ranges, do suffer from that type of constraint.
Hence in many cases (speaker dependent) there are benefits to be gained from moving the L/C/R onto external power amps, and value to be gained, by NOT doing the same for the surrounds and heights (assuming that they are an easier load, which is usually the case).
None of this relates to the reality that the surround and height channels are, on average, the most lightly used, on most material - that merely compounds the issue!