How would it work if one had a setup where every instrument had its own route to a speaker placed where you would want the musician to be in your listening space?
so no stereo mix..
Great. I would love to hear the Mahler Symphony of a Thousand recorded and reproduced that way. Yeah, right.
Unfortunately, you still need a lot more speakers than instruments and singers in the frontal soundstage (which is much less than one thousand for the Mahler, by the way) to also capture the enveloping reflections from the hall as they surround you in 360 degrees. There is science to verify this. And, your listening room is hopelessly inadequate for the purpose of recreating or resynthesizing that from just two frontal channels.
What we hear in most seats at a live musical event usually is really much more diffuse reflected energy, and much less the direct sound of the instruments and voices. So, as is often the case, you are misfocused, trying to reproduce the wrong thing on the assumption that we just need to reproduce the direct sound of performers as our eyes see them, ignoring the invisible and complex reflected sound field they also create, which is just as important to our perception.
Concerns about exactly reproducing the direct sound dispersion patterns of individual instruments are also relatively unimportant and misplaced compared to reproducing the resulting direct plus reflected sound field in the hall that we hear in our seats.
Bottom line, it is not just about direct sound. So, one speaker = one performer does not cut it. You have to adequately reproduce the reflected sound field, as well. Reflections are nature's own imperfect amplifiers. We have grown used to them since cave man days. And, we prefer music in the reverberant, reflective sound field of the hall. Stereo, unfortunately, restricts that in significant ways, especially directionally, though it sometimes does that for the frontal direct sound component plus some frontal reflections adequately via phantom imaging. The rest is largely just omitted in stereo. And, yes, some halls are better than others, subjectively preferred by more people who have heard and compared. Measurements in-hall have identified what acoustic factors lead to putting the Concertgebow, Bayreuth, Symphony Hall Boston, etc. on the pantheon of "best" halls for large scale symphonic music. Attempts to reproduce those acoustic parameters have been iffy in modern halls.
Stereo recordings in acoustically dead studios from multitrack masters can be quite enjoyable as music. But, what is the faithfulness to a live event they are artfully, though artificially, trying to reproduce?