You're right, this calculator doesn't help you much here.
What it does tell you is (assuming the walls are perfect reflectors at all frequencies, which in reality they are not) where your first reflections will create peaks and nulls in the frequency response; however, even in its ability to do this is there significant limitation, because (1) the drivers are all at different heights and (2) the nulls will really also depend on the location of your ears in relation to the acoustical centre(s) of the driver(s).
One thing I can tell you, without knowing much about your room, is that moving the subs so that they are in more optimal locations in the room is likely to better eliminate (reduce) room modes.
Harman did extensive modelling to try to solve this problem, and then tested the models in real rooms, and concluded that the optimal locations for
two subs were halfway along any two parallel walls. So basically like this:
View attachment 26892
Depending on the quirks of your room, however, YMMV.
Also, getting the crossover to work correctly will be more work than in the relatively straightforward configuration you have atm with them direcly under your mains.
Do you have a measurement microphone?