The surprising thing is the false generalized conclusion you are drawing from one test. This is very bad science. More precise if you weren't ambiguous and said in this particular test for this particular speaker, you couldn't detect a difference between X and Y. But you cannot generalize from that the way you have intentionally or not to say there will not be a difference between the two (smoothing or not). Smoothing and spatial averaging aren't related except by coincidence. These kinds of conclusions (as most people don't have the time or too lazy or confirms their bias to look at the details carefully) creates bad understanding that can propagate.
There are two primary variables that affect the measurement - room characteristics and the speaker's dispersion pattern. Spatial averaging will average over the effects of both but its results will depend on the variance between locations. As a thought experiment, if the speaker has a very wide even dispersion, then that parameter will not contribute much to the variance between locations. Similarly if the room characteristics do not lead to localized variances, spatial measurements do not cause much variations and therefore will be fairly close to the average. So, in this case, a measurement at any one location might be close enough to the average of multiple location. Smoothing in this case is operating on the almost same data-set to contribute anything.
Or it could be a false indication. It is possible that the different locations have a wide variation but on the average they happen to coincide with a specific single location measurement (the drunk walking cartoon I posted earlier). In this case, it is just a coincidence as a measurement at a different location, could be very different because of the variance. Just smoothing in the latter is not going to help because the information content in the two is very different. Smoothing has no information about the location-dependent variance so if it happens to coincide with spatial average, then it is just that a coincidence. But asymptotically you would be correct, if you smooth enough that those location-dependent variances disappear (you lose location information entirely), then it would be indistinguishable from smoothing in any one location. However, smoothing and spatial averaging should not be conflated not seen as replacements for each other.
When you do a speaker measurement, you don't really know which of the above is the case with your speaker/room setup and whether you have large local variances. This is why you do spatial averaging. It is a trade-off not a perfect solution for room-eq. If you do few locations, your may do over-corrections from local variances. If you do many locations, then it may under-correct for any one location. So, you have to depend on some kind of engineering thumb rule rather than theory.
Not looking at the implication of variances when doing any kind of averaging is the most common problem in number-crunching. Investing strategies (e.g., using technical trading) also suffer from this exact issue to give bad indicators that can cost you money.