You are not going to see a 6th or 8th order crossover in a passive speaker any time soon. The crossover cost, complexity, size and losses would be outrageous, not to mention the matter of parts tolerances.
You also have far more freedom in shaping woofer frequency response. Bass extension or various highpass filter options for different level needs? No problem. (A godsend for inexpensive subs, too.) General response shaping to match different environment is a common feature as well.
You can also drive the speaker drivers with just about any source impedance, from pure voltage source unencumbered by crossover parts (~0 ohms) to virtually pure current driving. The latter may substantially benefit IMD performance in drivers that aren't exactly Purifi grade.
And that's not even getting into the kind of advanced trickery that goes into speakers like the Dutch & Dutch 8C or Grimm LS1.
Overall, there is no reason why passive speakers can't be good (they benefit from modern design tools just the same), but active speakers can be better. Would something like the Genelec Ones even be possible in passive form? Likewise, the world of Bluetooth speakers would be a whole lot sadder-looking, I bet. For goodness' sake, we are living in an age where tons of people are running around with wireless noise-cancelling in-ears.
Of course there are some potential drawbacks. Dynamic range management becomes a major concern, for example - one of the most common complaints in active speakers (and even more so their DSP-equipped cousins) is audible hiss. This can be overcome, however, as illustrated e.g. by the Kali v2 series. It also goes without saying that having a bunch of electronics crammed into the speaker doesn't exactly make things any more service-friendly (though on the flipside, doing everything externally tends to result in a level of clutter that only diehard enthusists with copious amounts of space are likely to tolerate).
