The term "ground loop" gets over used. It originates in the days of large, high voltage valve equipment when the grounds where often offset by some stupid DC voltage and other bizarre things. Spend time with the valve aficionados and you'll see some things. It will also ruin your day setting up a large high power rig for a gig etc. With high AC currents you get real magnetic fields building up in loops of wire and loops between devices, it can start to resonate, usually the 50Hz mains hum or the 100Hz stage drone that ends up sounding like feedback.
What people in consumer audio are usually taking about is just common mode noise and power rail noise. Digital gear is noisy. All those sudden transitions require current to overcome the capacitance of the circuit. 24 million times a second or 1 billion times a second, or maybe it's just clicks and buzzes. If it gets audible on your gear it's a real pain in the backside to locate it and remove it. The usual culprits are digital hardware devices, the higher the performance and the lower the price the noisier they will be. PCs are bad, laptops are hideous usually.
When you get one device with a noisy ground connected to others, they all have noisy grounds. If the noise on the grounds is exactly the same and your ground paths are all low impedence ... you are extremely lucky. If however you end up with slightly different noise on one devices ground compared with the others, you get common mode noise between them, which may, or may not get onto your audio. "Mixed signal" design is a whole thing in itself on keeping the two noises domains apart.
Optical provides total electrical isolation, but if you then power the two devices from the same power supply it defeats a large part of the purpose.