I have heard about this decay potential.Welcome to ASR.
I have overcome this problem by putting my music server on the other side of the room from my listening seat. I set an album to play, walk across the room, sit down, and get lost in the music.
I could never go back to spinning optical discs, the biggest problem with which is that they decay and can too easily be destroyed. I keep two backups of my music server, one made monthly that I carry to the office and lock in a desk drawer and another that I keep persistently up to date in a remote location via FTP.
A bit obsessive, yes, but better safe than sorry.
I have had discs which were physically damaged and couldn't be read at all by some transports, skipped on others but seemed fine on yet others but despite owning about 7000 CDs and having being buying them since the day the first CD player was available here (literally) I have never come across a disc with decay.
How are other people's experiences? Have I been inordinately lucky?
I have largely given up computer audio after over 15 years personally.