All forms of noise and distortion become audible beyond certain thresholds.
In the case of most DACs, levels of noise, nonlinear distortion, and phase distortion etc. are likely to be low enough as to not cross these thresholds so, broadly speaking, yes, I think your view is valid.
However, there are clearly a number of DACs on the market which exhibit levels of (in particular) nonlinear distortion such as to suggest that they would likely sound different from a better-performing DAC with an identical frequency response.