Why do you think they put that op-amp there, to waste money?Why not use RCA to XLR adaptors instead of RCA input for better performance?
RCA inputs have additional and innecessary gain as well as op-amp 'converter'.
XLR has enough input impedance to drive single-ended voltage.
You will lose quite a bit of output if you just feed se into the xlr. It will work, but it won't be better. You would need an active rca-xlr converter instead of an adapter. The 'innessary' gain and op-amp ARE your active converter, before they feed it into the balanced circuit of the amp.
So no, don't do that. It's a workaround for an amp that only had xlr in, it's not an improvement over Fosi's design.