Yes the noise is equal with or without input source. So the noise floor of the interface is sufficiently low at the output level I use...
Keeping the noise floor down, and not wasting headroom, is generally a challenge with mixed interfacing. The Adam solution is good, a late, remote-controlled gain setting. Dynaudio Air had this, Genelecs with GLM, and probably others, too.
Reducing the risk of accidental, traumatic sound exposure is another important benefit. Late gain should not be set higher than needed at any point in time.