To get back to the original questions, the problem is that the relation between sound level and the power required to produce it is not linear. A little bit louder requires a lot more power. Now you may think you do not need much power because you are not the extravert kind. However, depending on type of music, peaks in the sound level can be a lot louder than average levels, and particularly so with highly dynamic music like big orchestral music. So to get those clean peaks, you will need a lot of power, and if you don't have that, it will be audible. Add to that two other factors: room size and speaker efficiency. Reproducing large peaks in a large room using inefficient speakers suddenly demands almost unimaginable amounts of power, where not very dynamic music in a small room using very efficient speakers requires almost nothing. In short, there is no answer that applies in all cases. However, my rule of thumb is that you will need more power than you thought. The good news is that watts have become cheap. Also, active subwoofers can provide a large chunk of where power is needed most: in the bass region.