I finally took a measurement of the SPL difference I'm getting from left to right ear. The chart below shows the level reduction in the right ear compared to the left ear for a left panned signal in my 3 speaker array. It's not natively quite this good, but tonight I managed to successfully apply recursive crosstalk reduction to the side channels.
!!!
This is a total trip. I contacted Rogue Aeomeba asking if they knew a way to do sample level delays in Audio Hijack.
They gave me the link to this free AU component that does the trick:
https://www.voxengo.com/product/sounddelay/
This lets me apply a 3 sample delay, which is about right for recursive crosstalk elimination at 48kHz sample rate. I played with the normal recommended sound reduciton of 2 to 3 decibles but it just sounded confused. Then I realized that I already had a head start because of my 3 speaker array inherently creating some crosstalk reduction. I measured that to be around 7.5 to 15 decibles, so I initially reduced it with roughly appropriate EQ curve and then applied successive recursions with -2.5dB.
Eureka!
I think this is a big step up for this array, and it has an advantage over recursive crosstalk with a 2 speaker system for two reasons:
1. The center signal is completely untouched by the recursion alogrithm.
2. The alogrithm starts off much quieter so it has much less audible effect when listening off axis and less troublesome effects on tonality overall. To my ears it sounds cleaner and more natural than without it.
I'm getting excellent width now. A left panned sweep sounds like it's completely off to my side, 180 degrees! I can probably get it better with a little more measurements and experimentations.
View attachment 322614