Accuracy matters. Absolute. If you have some non linear speakers and use them at "normal" low levels, you adapt to the sound. Your brain fills in holes and flatens out peaks. So you are fine and like your "good" speaker system. Sorry to say, this has not much to do with the retail price of your cabinets.
Now you start to turn up the volume to higher, maybe concert levels. Your trained ear/brain complex will not like what you present to it. The peaks will get anoying and the holes will make voices and instruments unrealistic. Worst are increasing 3rd order distortions, that will finish up the experience. If you are not drunk, you will lower the volume.
Now take a linear speaker, with a good dispersion under different angles and a well suited subwoofer. Not much work for your brain, because no sound equalizing needed. Which makes it very relaxed listening. Now you increase SPL, the result: You can turn up the volume to insane levels, the subwoofer will reduce 3rd order distortion quite significantly. Nothing will disturb listening, you will enjoy it loud!
That's the difference between accurate and not.
The minimum cone size for such a system, IMO is a 6 1/2" two-way, or, even better, two 5" in MTM configuration, with a steep 80-100Hz crossover. Don't start with less than two 10" active subs, DSP'd to the room.
Multi channel audio suffers from serious "too many speakers, build too cheap" problems. If you use the same high quality speaker systems all around, with a good and powerfull AVR, it get's hard for good ol' stereo to keep up. DTS neo:x and Dolby (Atmos) do a very good job even on older recordings. There are only very few titles that do not like being processed.
If you insist on recycled plastic cups pressed in the shape of a 5.1 BOSE speaker set to sound "soo fantastic", get an ear lobotomy.