WHAT was the huge audible difference; what did you learn from this?
That decay and resonance issues in bass and lower midrange play a crucial role in perceived tonality, transparency and timbre, completely independent from frequency response and directivity.
Three loudspeakers could show absolutely identical frequency response with only their midrange enclosures differing, we will nevertheless perceive voices reproduced by the high-Q midrange as ´bloated´, ´overly warm´, ´lower midrange dominant´, ´resonant´, the low-Q one as ´warm, but balanced´ lower midrange, while the open baffle/cardioid version sounds comparably lean, more transparent and more like you would expect it from a recording of an acoustic event. Note, that the former does not necessarily mean a ´boxy´ sound, but something into that direction.
AFAIK this was not widely published back than, but discussed a lot, when several loudspeaker manufacturers (MEG, Amphion, Kii, D&D) decided to go for cardioids in the lower midrange, and some in the bass as well. Interestingly, in recent years we see a wave of cardioids hitting the market.
I think that the resonances (whether internal or panel) which result in "boxiness" begin to arrive AFTER the direct sound, so there is a little bit of a time gap.
That sounds like an explanation for lower frequency effects, but I cannot really imagine this to be the main effect for lower midrange ´boxiness´, or the effects described above in the 300-700Hz band. I mean, we are talking about midrange enclosure sizes of roughly 0.5-3l.
If group delay is not the main issue here, my personal take would be that decay is crucial, independent from frequency response. Particularly for typical fundamental frequencies of human voices, we seemingly have a sense for sonic events which are either decaying quickly, or decaying much longer, with the latter being recognized as ´sonorous´, ´warm´, ´full-bodied´.
Something like this happens in diffraction horns, wherein the diffracted energy arrives a little bit later in time and therefore is not masked and therefore is more audible and objectionable than we would suspect from the frequency response curve alone.
Again, absolutely can confirm your experience with diffraction horns, and it sounds like an explanation, but there might be other ones. I would argue that such delay between two events of the same frequency, does have an impact on frequency response as well, and our ears rather perceive the resulting comb-filtering effect and its peaks. With horns in general, it is pretty difficult to separate any of the effects taking place at the same time, may it be in listening tests or measurements.