How loud we listen to music should probably be given in context if the situation. Back in 1979 testing average western listeners liked to hear native language spoken at 52dB(A) when the room noise floor was itself 40dB(A). While for every 10dB(A) increase in the room noise floor above 40dB(A) they preferred another 3dB(A) listening level above 52dB(A).
While there was a confounding factor if the language spoken was one of the listeners second languages. Then they preferred listening at dB55-57dB(A) when the noise floor was 40dB(A).
I wonder if that last kind of preference variability might be related to how some people adjust listening levels depending on which kind of music they hear playing, even with the same room's noise floor. Suppose nature (melodic) sounds are our default evolutionary sound recognition pattern in any noise floor and we turn up (say) rock music to try and pick up it's nuances in better detail from the noise field.