As far as I'm concerned, recording and mastering choices and quality are 90% of sound quality; equipment is just the rest, (assuming it's basically good quality and suitable to the listening room).I agree that "high fidelity" implies faithfulness to source. And generally I only care a lot about that with classical music and (sort of) Jazz, (ie where I have a solid live reference as a concert-goer).
Where things can get mixed up is how much of the original performance space are you supposed to hear in your listening room - what is accurate in that trade-off? There are close-mic'd, dead room recordings, and there are recordings in huge churches. I guess that's more of a recording issue than a reproduction issue, although ambience reproduction strikes me as an additional complication in your home setup.
I like mostly Classic and a big beef with me is the perspective in case of chamber music in particular. Some sound engineers seem to be aiming for an "seat among the performers" position whereas I'm much happier with a 4th row audience perspective.