I have not read the rest of thread but imagine everyone will say the same things, mainly:
- Mastering; and,
- Expectation bias.
Number 2 is easier: if you think something is going to sound better, then it probably will. That is why blind testing is so important for us to determine if the difference is really there.
Mastering is what many of us have found to be the difference. Mastering is one of the steps in the process that takes the sound at the recording microphone all the way to the CD or LP you play at home. Mastering is usually the last step and is when the sound engineer creates the "sonic balance" of the final recording. Effects like reverberation can be added, the equalization can be adjusted, and loudness adjusted (typically compressed). In short mastering greatly affects the sound you hear from the recording. A lot of CDs were remastered from the original recordings (tape or digital files) nd re-released in CD format; sometimes to preserve the songs, sometimes to "improve" the sound, and often both. The problem is that many CDs have dynamic range (difference between loudest and softest sounds) compressed and then the level boosted to make them louder. Louder generally sounds better to people, at least at first, and the record industry decided that making CDs louder would boost sales. Adjusting the equalization (EQ) to boost highs and/or lows was also sometimes done, and in some cases the mix (how the different voices and instruments are blended and leveled in the recording) were adjusted. Louder or not, the sound recorded to a CD is often quite different than that on an LP, and that is why they sound different to you (and to everyone else). Really nothing to do with the conversion to digital.
On the flip side, equalization (frequency response) is often adjusted on LPs to reduce the lows, and dynamic range may be compressed as well, to meet the limitations of the media. Another factor is that the higher crosstalk (left channel in the right and vice-versa) and distortion (often low-order) actually makes the sound "richer" and fuller" even though it is less accurate.
I have some CDs that are faithful to the original performance and they sound much better than the LP I have of the same recording. And many CDs that to my ears exhibit poorer performance after the remastering process (frankly, they suck). I
know it is not the digital conversion process, and there are lots of others with the same complaint, including a bunch of folk who have analyzed the dynamic range and frequency response of their CDs to prove how they have been degraded.
@mitchco has a link to a recording a friend of his did showing what havoc loudness compression causes. A shame to take the huge dynamic range made possible by digital sound and squash it to much worse than the LP!
HTH - Don