On top of these observations, I would expand to say that the human brain is constantly trying to "make sense of things," looking for recognizable patterns that it recognizes or "believes" it recognizes from prior events.
Also, our free will can interfere with attempts to listen in an objective way: we are "free" to sort of hear what we want to hear. And once we've "decided" that, for example, a centered mono source with no actual spatial cues associated with it based on time or phase actually sounds "distant" simply because it is somewhat reduced in level, has its highs rolled off a bit and has added reverb, then from then on we may "choose" to interpret that kind of treatment as an adequate illusion of depth.
For me personally, I can selectively "choose" to buy into that illusion or instead analyze what's is likely really going on and decide I 'm not in the mood to be fooled by it. Frame of mind is everything. (And if your speakers have a "BBC dip" at 2500Hz, things in the center are likely to be sound somewhat distant more often than not, exaggerating the illusion of depth.)