1: There is an RCA to XLR cable (or converter plug) where on the XLR side one of the input pins is shorted to ground (otherwise it won't work).
This cable usually has either a female (socket) RCA connector, in case of a converter plug, or a male RCA plug in case it is a cable and a male XLR connector. Sometimes you can find cables or adapters that don't adhere to this convention for specific use cases and could have a female XLR connector (this is the potentially dangerous cable when used the wrong way around).
When you are using this the source must be RCA out and the input of the following device must be XLR.
The potential problem with this type of converter cable is that when it is used the other way around this will SHORT one of the balanced outputs when the source is XLR and the following input is RCA. This cable can be physically the same as the cable (or plug) described below.
2: Then there is another cable (converter) specifically for XLR to RCA where on the source side only 1 of the 'hot' pins is connected to the center-pin of the RCA and the other pin is left 'open'.
This cable usually has either a female (or male in case of a cable) RCA connector and a female XLR connector.
There are some cables converters) that look exactly the same and can be used for connecting a balanced (XLR) output of a source to an RCA input without shorting the XLR output. When this cable is used the other way around there may or there may not be proper sound depending on how the XLR input is designed (transformer input or differential input using op-amps) but won't damage anything.
3: Then there is the 'proper' and safe way and that is to use a converter (box) using a 'balun' transformer inside. This truly converts a balanced signal from XLR to an RCA signal AND works perfectly the other way around as well.
Another advantage is there is galvanic separation (isolation) between the devices which prevents groundloops (should they exist).
A disadvantage is the low impedance these boxes can have which may be on the low side for some sources when going from RCA to XLR.
All XLR outputs should be able to drive these transformer boxes fine though.
This box, however, technically reduces bandwidth and introduces (very low) amounts of distortion and is limited in max. voltages (transformer can saturate) but this is at unusual high voltage levels only. Certainly not at any normal RCA levels.
So it depends on what cable you bought (1 or 2).
Both cables can sound just fine but using cable 1 could (in some cases) damage the source (when it is an XLR output) and this may or may not be audible.
In all cases (1, 2 and 3) there will be music playing and may well be not distorted but option 1 may (potentially) not be a good way to connect stuff this way.