The cold-shorted-to-ground at the unbalanced device definitely helps to a degreer, but it's not removing the noise entirely like powering down the PC does.
Makes sense. This way you are eliminating the effect of ground currents on your return connection, but they are still circling through the unbalanced connection ground coming into the FX device and the FX device itself. You have to actually break the loop by leaving the cable shield open instead of connecting it to cold. The only direct ground connection of the FX device will be via the input side.
If that still doesn't cut it, check whether the FX device possibly is also grounded via the rackmount or something, things will never work properly like that unless you go transformers (btw, I tested the HD400 at one point and found no substantial issues in terms of frequency response or distortion at levels around 1 Vrms). Also, be sure to test with one that has low output impedance, so that you get a decent amount of CMRR going on the balanced input. At this point the entire signal you are trying to get rid of should effectively be appearing as a common-mode voltage on the FX device output, so if you manage to balance the connection well it should be suppressed quite effectively.