I'm using MBP 13" so vanilla M2 (just an Air with a fan basically, I bought it to keep the TouchBar for as long as possible).
But I don't associate the issue with cpu load
per se. As mentioned, I usually notice the issue when Safari has upped the memory pressure without necessarily taxing the cpu. That machine has 24 GB RAM (less than I'd prefer but more than average) and I tend to accrete a lot of tabs when working or looking up catalogs (alongside general use) so it's easy to push that.
The easiest fix is to quit Safari and start again without tab detritus, so while it may not be a memory issue at all it relates to Safari's use of resources somehow. I haven't tried loading up memory with other apps specifically, and I'm not suggesting Safari use is the only way to invoke the issue, just how it usually arises in my use of that machine (the laptop isn't my main work machine).
I note
@Ulfgard's mention of the 66% memory pressure metric and I'll have to check that next time.
I'm just using the Music app to stream, as I dropped Tidal a while back (and rarely slum it with Spotify, YouTube et al). But I think the OP is correct that the stutters affect all audio, it's just more irritating with music. First thing I tried was dropping the Core Audio sample rate (I usually set it to 24/192 so as not to downsample) but looks like that isn't a factor.
My guess is it's an issue with macOS and Core Audio scheduling of application-level services that arises in my particular configuration (and some others) and one or more of the audio services isn't prioritised as I'd prefer. Also note I'm using macOS 12.6 (latest is 13.2.1) so the issue may or may not persist after upgrading. I think Safari 16.0 is current though?