@ecxopraxia
As you are OP I will try to answer your question, although we stepped off-topic.
"What does that mean when your in-room response is so uneven that you have huge nulls and dips? Should you boost the sub until the minimum response matches the mains, or decrease them until the maximum response matches the mains SPL, or something in between? "
Say you are aiming for typical XO point at 80Hz. Most quality subs are quite linear in the 50-110Hz range so you should be able to draw a straight line in that range. Adjusting gain on the sub (or on the sub channel if dealing with passive sub) should be done so that it meets same that line from the mains response. That should give you a good starting point to start matching the phase. As @Duke pointed out, it is an iterative process, so patience and lots of measurements are required to make it right.
As you are OP I will try to answer your question, although we stepped off-topic.
"What does that mean when your in-room response is so uneven that you have huge nulls and dips? Should you boost the sub until the minimum response matches the mains, or decrease them until the maximum response matches the mains SPL, or something in between? "
Say you are aiming for typical XO point at 80Hz. Most quality subs are quite linear in the 50-110Hz range so you should be able to draw a straight line in that range. Adjusting gain on the sub (or on the sub channel if dealing with passive sub) should be done so that it meets same that line from the mains response. That should give you a good starting point to start matching the phase. As @Duke pointed out, it is an iterative process, so patience and lots of measurements are required to make it right.