To the OP, my solution was to use an AV pre/pro, a quality amp and speakers front left and right with powered speakers for center, surrounds and subs. So I've pretty good stuff for stereo, and good surround. I do just switch to stereo mode via the Marantz pre/pro. The Marantz is the bottle neck on performance, but if you want it to be usable by someone else it is among the more convenient solutions.
I'd also suggest buying one 3 or so years old where you can get a great discount of 50 to 70% vs original pricing. You lose out on the very latest version or two of advanced surround formats, but the availability of those in media lags their introduction by that much anyway.
If you were to do this, you could have a quality line level selector which switches line inputs. One uses what comes from the pre/pro for front left and right for movies, and the other uses line level from another source like a good DAC. That way your pre/pro isn't a bottle neck for stereo. Much simpler than speaker switching.
This way the only part you might periodically change out is the pre/pro, speakers, amps etc aren't something you need to change very often.