Thanks to VilhoValittu's nice and improving software, I've done several runs of DecayCore and here's my current workflow:
The starting point is a set of .mdat files. Each of the 8 speakers (includes one subwoofer) was measured in REW with UMIK-1 at 5 positions, the Main Listening Position (MLP) plus 7.5cm away from MLP in 4 directions: directly in front of MLP, directly in back of MLP, directly to the left of MLP, and directly to the right of MLP, producing a total of 40 .mdat files. Each of the 7 (non-sub) speakers is measured with the sub already integrated (sub delay, sub gain, sub polarity, crossover set) and with the sub on. The subwoofer is also measured separately by itself.
Workflow:
1 Open in REW the 5 .mdat files for one speaker, such that the measurements at the five different positions are now seen in REW.
2 Click "All SPL" button in the horizontal button bar near the top of the REW window.
3 In Actions -> press "Cross Correlate Align" followed by "Vector avg"
4 Under Export, Export impulse response as WAV, which opens a menu of options
5 In this menu of options, choose Mono, 32-bit float, Export measured IR, 88.2khz sample rate, 64k samples.
6 Save as a .wav file.
7 Repeat 1-6 for the other 7 speakers to result in 8 impulse response .wav files, one for each speaker. Use these as 8 files as input for DecayCore.
8 In DecayCore, I don't run bass integration (sub already integrated) and generate linear phase FIR. Under the XO tab, include the built-in crossovers for each speaker, and also an XO at 80hz (except when doing sub only) because the sub and crossover are active for each speaker measurement. When running the sub, set the leveling and gain setting under menu 4 target to 20hz to 200hz. Because DC expects a speaker pair, I just run Center, Center and sub, sub.
Still not 100% sure about step 3 in the workflow above. Choosing 'vector avg' rather than "RMS + phase avg" seemed to give better results in DC so I went with that, but if there is a math-based or physics-based reason to choose another averaging method, please explain.
Here's an overlay graph from REW measurements L with sub (red), R with sub (blue), sub only (green), with before DC in light color, and after DC in darker color. I've also included the summaries for L and R and sub.
The reason I want to do sub only is for 5.1 audio. In that specific case I need sub only filters for the 0.1 channel. In contrast, 2.0 and 5.0 audio are fully covered by the filters produced for each speaker measured with sub.
The dips around 110hz for sub and R speaker (they are next to each other) are likely due to null in room. The dip for the sub had been at 80hz, but I re-positioned the sub and moved the dip away from 80hz crossover and it ended up the way you see it around 110hz. Fine for 2.0 and 5.0 which is crossed over at 80hz, but too bad for 0.1 channel at 110hz.
A result I get with the DC auto mode is that often different speaker pairs get a different target curve. I guess that's ok, and I suppose it's possible to fix it to one (perhaps the target curve that best fits L and R) but I haven't tried yet.
Any advice or comments on how to improve results or understanding is welcome.