Looks like AKM ADC chip. Have you checked the data sheet how it's digital filter looks like at the sampling rate you are running at? The analog filter looks maybe 3rd order. And if you want fairly flat phase response in 100 kHz bandwidth it means fc around 400 kHz or so. Or if just for 20 kHz then fc around 100 kHz.
Then again, many ADC and DAC chips also exhibit noise modulation anyway... So could be that too.