I suspect it is the other signal processing in the Yamaha streaming amplifier and not (just) the class D audio amplifier that is increasing idle consumption. It is true that efficiency drops for class D amplifiers at low output, just as for any other class of amplifier, but is still generally better than class AB at comparable power. You'd have to know more about where the power is being used to make a true comparison.
Here is a random picture pulled from the Internet for a class D amplifier module (by ICEpower):
This indicates very low efficiency at low power, but consider the actual idle power for this amplifier:
A class-AB amplifier with ~100 W output may have idle power around 10 W or so depending upon the design (some use more, some less, but I do not know of any with only 1 W idle consumption).
HTH - Don