This depends a lot on the particular situation.
Some buildings, especially modern buildings, are better at preventing noise transmission between apartments. However, the laws of physics being what they are, subwoofer frequencies are notably hard to stop. Sadly, here in the UK building regulations don't take subwoofers into account: they do specify noise attenuation minimums, but they stop at 125 Hz.
Conventional "soundproofing" solutions for existing rooms don't work at subwoofer frequencies (
this company said so themselves when I asked them). At these frequencies, absorbers are impractical; the only thing that will work is mass. A
lot of mass. I suspect the best situation is to share something like a massive concrete load bearing wall (or perhaps a void) with any neighbours, and in fact the details of the walls was one of my selection criteria when I bought my current flat (though it did get me some confused looks from estate agents!).
It's very difficult to predict how well and how far noise will spread in a building. For this reason my personal policy is to assume I'm not bothering anyone until proven otherwise. I only change my behaviour if a neighbour knocks on my door and complains. In my current building (in London) in which I've been living since 2019 (same year it was built), so far that has never happened. This is despite me sometimes listening at loud-ish levels (e.g. 85+ dBA).
However, in my previous building (London, built in 2012, lived there from 2012 to 2019), I did have problems with one of my neighbours due to the separating wall being too lightweight to handle the subwoofer frequencies, and their bedroom being directly adjacent to the room where my system was. In the end I managed to reach an understanding with them and they let me use my subwoofer at certain hours of the day. At some point that neighbour moved out, and was sadly replaced by someone who was much less amenable and wanted me to stop using it at all times, period. At this point I had no choice but to use headphones. That was one of the main reasons I moved.
How do you know if you are you being too loud or if they are complaining without good reason?
My two cents: everyone has a right to enjoy their own accommodation in piece and quiet, without interference from neighbors. If you do something that prevents that, then in my opinion you are morally at fault, no matter how tolerant you think they are or should be. The burden is on you to not bother your neighbours, not on them to tolerate any sort of noise pollution.
I think that's how most people would see it; I don't think many people would argue that you have an inalienable right to use a loud sound system in an apartment building, especially since reasonable alternatives exist, namely headphones. That, or moving to a better isolated apartment or better yet, a house.