One often overlooked reason 96k once offered real improvements and doesn't any longer is the DAWs in use. For some years quite a few did compression or reverb or other processing at 96 khz as that is what the plug ins were written for. So any other sample rate had to be converted. Some conversion wasn't good and some orders of processing would lead to significant aliasing and other artefacts. Sometimes the math was truncated or had other issues. With lots of processing this could be audible. Record in 96k and only 96 k (88k was no better) and you avoided these problems. So a 96k recording of heavily processed material would indeed sound different, and cleaner than other rates.
This for the most part has become a complete non-issue in the last few years. Now most plugins when needed will upsample to extremely high rates and use 64 bit precision. So there are no issues from such activity related to sample rate. Many compression plug ins let you choose how much and whether to upsample to either use aliasing as a choice or prevent it from happening. So this is a situation where maybe you convinced yourself back when with earlier recordings that hirez was a wee bit of an improvement. The same recording done currently likely offers zero audible benefit versus 44 or 48 sample rates.
Yes, you are exactly right. This was a problem in some cases, but is not any longer with competently-designed processors.
DAW plugins that do nonlinear processing are often working at sample rates far above 192kHz nowadays. I use a hard clipping plugin for nipping off the peaks of certain program material, and the user can select the oversampling rate. Aliasing is reduced substantially at 32x oversampling, but the software I use goes up to 256x for maximum transparency. That's 1.4 MHz and 11.3 MHz, respectively. Even if we were to run DAW projects at 192 kHz, it's merely a drop in the bucket compared to what nonlinear processes
really need.
I'm not saying the difference between 32x and 256x is always audible, but it's easy to see the difference in measurements.