...he likes to stab the woofer's surround with his claw.
I can give you lots of advice for cats in general. However, your cat likes this? Likes the feel and the sound probably, it feels like a "kill" is my guess. Which means you cannot change it while you have that cat. That cat will always hunt drivers, and the fact that they are "hiding" behind the grills just makes it all the better when they can be attacked.
Best thing you can do is only take the grills off when listening, and put them back on when you leave the room.
I do a have a floor to ceiling cat tree, and training cats (ideally as kittens) where it is ok to scratch works well. Just move them to the right spot. The key is putting tree/posts in places they want to mark. Near but not right next to doors to the outside is a good spot. And being immediate with your redirecting.
Assuming you have a good relationship with your cat, you can try this. Get the most dirt cheap speakers you can find, broken is fine. Let the cat kill those, praise him, give him a treat if he is food oriented. Then when he looks at and approaches your real speakers, gently tell him no and redirect him to the sacrificial speakers. My experience has been that if you give a cat what they want, they will let you have what you want. They will cooperate, they just have to understand and feel like you understand them.
It might work, but I would still try to keep the grills on for a long time when I was not in the room.