What rules? Curious... My system shaped the noise to move it out of band and control the amplitude. What sort of psychoacoustic rules are used to optimize dither in audio systems? Do they correlate to the math? Noise decorrelation is pretty well defined and has been for decades; I don't know anything (relatively) about psychoacoustics.
from https://en.wikipedia.org/wiki/Noise_shaping
Noise shaping is a technique typically used in digital audio, image, and video processing, usually in combination with dithering, as part of the process of quantization or bit-depth reduction of a digital signal. Its purpose is to increase the apparent signal-to-noise ratio of the resultant signal. It does this by altering the spectral shape of the error that is introduced by dithering and quantization; such that the noise power is at a lower level in frequency bands at which noise is considered to be less desirable and at a correspondingly higher level in bands where it is considered to be more desirable. A popular noise shaping algorithm used in image processing is known as ‘Floyd Steinberg dithering’; and many noise shaping algorithms used in audio processing are based on an ‘Absolute threshold of hearing’ model.