I'm not convinced of that. As I mentioned before, yes no doubt there is a super rich contingent, as there is for any product.
But I don't think $100K speakers only end up in the homes of the super rich audiophiles. If someone gets obsessed in a hobby like this, then you can end up spending more-in-relationship-to-what-you-earn than the average person. And for many audiophiles their gear is something of a financial stretch, they have to make a choice where to spend their disposable income. I have friends who are in to watches, and if I only judged them on their expensive watch I'd think "oh, you are very well off!" But, no, they earn no more than I do, that's just where their money goes. Likewise, I've even had someone in this forum infer that I was rich or well off due to some of my audio/home theater gear. But I'm very far from rich. I don't have any other expensive hobbies and what I have is a result built up over lots of time.
Similarly, you can see $100,000 speakers in the homes of audiophiles who aren't "super rich" but who are well off enough to afford them, but it doesn't mean it was easy or inconsequential financially or otherwise. They are just audiophiles playing in a somewhat higher price bracket than lots of us here.