Ground loops will cause a very specific 50Hz/100Hz (in Spain) humming, or buzzing. Was that what you were hearing when it sounded "quite bad"?
If so, then yes, it probably was a grounding issue. I'm assuming that "sound ok by RCA again" means the RCA outputs of the wifi Zen?
If not, then you...