You can read and understand more about the phantom/or ghost voltages I referred to:
Phantom Voltages This Bulletin is intended to address the occurrence of so-called “phantom”* voltages, a phenomenon detected during the testing of electrical conductors in the field. Due to the high impedance of measuring instruments, a voltage reading may be detected on open conductors where there is no hard electrical connection to a voltage source. Conductors that are installed in close proximity to one another, and are capacitively coupled to each other, can cause this a.c. voltage reading. Such a reading could be 2 or 3 volts, or it may be as high as the voltage on the adjacent conductors. This is what is referred to as a “phantom” voltage. According to Underwriters Laboratories Inc., this can be a harmless reading and can be caused by the high input impedance of the measuring instrument, which places very little loading on the circuit under test. The capacitance is increased as the length of the run is increased. A 50-foot run may produce a pronounced capacitance effect whereas a one-foot sample may not produce any. Since the “phantom” voltage is a physical phenomenon involving very small values of capacitance, it cannot energize a load or cause physiological damage to a person. Care must be taken to be sure that the voltage reading is a phantom voltage, which is caused by improper use of high impedance multimeters, and not as a result of a cable defect or improper installation, which may result in a shock hazard. In order to help minimize the likelihood of reaching a wrong conclusion from this phenomenon, NEMA recommends the use of a Listed low impedance multimeter in place of a high impedance multimeter or other high impedance measuring device for testing on open conductors where there is no hard electrical connection. Without a low impedance measuring device, a high voltage reading is an inconclusive indication of possible faults in the cable. 2 *Phantom voltage is an industry term used to describe an unexpected voltage of a nonelectrically connected conductor.
Phantom voltage should not be confused with stray voltage which is the occurrence of an electrical potential between two conductive surfaces that should ideally not have any voltage potential between them and can result in a hazard to people and animals. This bulletin only addresses phantom voltages.
My concern is, without having the power supply in my hand for more testing, and without any information of how the stock PS is constructed and wired, I cannot say it is "normal", or "safe" to use. So as others mentioned before, return it, and may be try another one if you really like the amp a lot. I know it is a nuisance to return something, but for the sake of safety, you should do it. If you can find someone who can help you do some tests to confirm whether the voltage is of phantom nature, or a real issue that could be hazardous then you should do that first.