If you keep the cables from the preamp to the power amp reasonably short (0.5M to 0.75M) the impedance won't be an issue.
In simple terms what happens is this; Depending on the position of the volume control, the output impedance can be high enough to interact with the cable capacitance which causes a very slight reduction i nthe to pend frequencies.
With any decent cable (even Amazon basics), so long as you keep the length short this won't be an issue.
However - ideally you want an attenuator that's no higher than 10k ohms in value. 22k ohms would be just about ok.
Something like this would work well.
https://www.amazon.co.uk/TENEALAY-Passive-Preamplifier-selector-FV2-Black/dp/B0BWN5XR3P
The other thing to be mindful of is the quality of the actual potentiometer used - some can result in uneven channel balance at various positions.
This is why people tend to recommend a stepped attenuator, however you need to ensure that you get one that operates in 1dB steps, otherwsie it can be tricky to get the volume level right (can be too loud or too soft depending on which step is selected).
Good quality stepped attenuators also tend to be expensive.