The biggest problem is with the mastering.
No, the speakers/room is a bigger problem usually.
I agree that mastering is a much larger problem than HiFi electronics, and I "do something" by looking for the best (reputedly) masters or editions I can find.
We need to devote more attention to...