You are spot on. However, almost all expensive DACs are rather pre-Amps with filters and DSPs. Specifically, RME ADI-2 has an EQ even.
For my CCA, I was able to reduce the noise floor by switching to a cleaner USB power supply from iFi, it was pretty significant, noticeable improvement.
I...