Sorry, I neglected to look at your signature.

I see, the SVS subs don't have high-pass crossovers.
Also, makes sense that with and without subs results in the same noises coming from the Polks.
You have great speakers, amps, and preamp. The subs are very limited in features for integration.
An external crossover might be cheaper than starting over with an AVR.
Here is the likely problem. The Polks have good bass to below 80Hz, shelved down a bit.
View attachment 494294
View attachment 494287
The port maximum output is at ~40 Hz. It does a good job of making a nice shelved bass response that allow the speaker to be placed near room boundaries and sound good. In fact, the woofer barely does any work at ~40 Hz, the port is providing the majority of the bass output at the is frequency. Here is an example, a simulation of a single 6.5" Audax driver with my attempt at
similar tuning:
View attachment 494293
Below 40Hz, the port and woofer no longer work together so nicely, and the woofer excursion becomes uncontrolled. Here is the excursion of that Audax woofer at just 10 Watts:
View attachment 494295
This Audax woofer is very unhappy with more than 10 Watts of power at 30 Hz. It's unlikely your Polk woofers have more than 5 mm of excursion. Most music doesn't have much content below 30 Hz. Movies do go well below 30 Hz.
I am hesitant to encourage you to spend more money. But here I go

: A proper crossover is the best way to go with the SVS crossover set to flat. Using the mains without a high-pass crossover is just not going to provide the best bass. Alternately speakers with more SPL capability, but that won't fix your mains / subwoofer integration problem, and may actually sound worse.
In the meantime, you need to keep the Polks from hitting their limits. Plugging the port will help a bit. Here is the excursion of that Audax with the port completely sealed (solid blue line) compared to the port open (dashed turquois line):
View attachment 494319