Friend, as another engineer I hear your frustration and empathise. I agree a Capacitor is very simple; two conducting plates, separated by insulating dielectric material to store space charge; AC passes according to the capacitive impedance law, DC is blocked. ESR and other simple parameters are fully understood and characterised to be considered in designs. Shouldn't matter how its made if viewed as an ideal component.......except that in the real physical world there have to be other differences which will be and are measurable between capacitor classes, or there would not be so many types available and recommended for different applications. That is not to say there are audible differences in many or most audio applications between these various types - probably if designed correctly into an application there will be none, at least when new.
Electrolytic caps are electrochemical components however. They can suffer reverse polarisation effects causing damage which may or may not be self healing. They also suffer from chemical electrolyte changes due to time, temperature and humidity (if case sealing is compromised particularly), leading to variance in tolerance and leakage of DC possibly while in operation. Probably not audible of course unless the device actually fails....but that is more likely to be a risk in tube equipment than SS. Film caps are closer to the theoretical ideal cap - or at least less likely to exhibit parameter variances due to environmental and age related changes.
Am I saying electrolytic caps are bad for audio path use.....no I am not and they are of course used everywhere.....but old, hot or tired ones may misperform and especially in any application where reverse polarisation of the cap is possible then its not going to function as designed over time. Smaller value electrolytics don't seem to last well in particular - I've measured brand name electrolytic caps with ESR value >5 X the already high "acceptable" value in 10 year old amps, hardly vintage....the 10uF - 47uF ones, and bipolars, are often worst for some reason. Their main advantage of course is much much higher capacitance per unit volume vs film caps, and they are cheap too. As ever there are compromises to be made when selecting caps for any design application.
Film caps are almost always more stable and highly reliable over long periods of time. For those reasons I'd design them into any audio gear where cost is a somewhat lower consideration than performance, or physical size is not a limitation - otherwise, as Bottlehead obviously did, then go for it and use electrolytics - but I'd keep an open mind about swapping them out as there just might be reasons for other types to work -or sound- better.
Of course YMMV, you are quite free to be "reverse polarized" to my viewpoint !
Cheers, SC