Well, I have already admitted that it will be slightly less energy, but I doubt it will be so huge of a difference that the neighbours suddenly hear nothing.
Since I design cardioid loudspeakers I actually have thought about it quite a bit. And from a more practical perspective, I've been around cardioid speakers quite a bit. In one of my rooms one of the speakers is situated towards an area that opens up to the side of the speaker, so it's possible to actually walk past the speaker. And there is certainly less bass on the side and on the rear, again that is the whole point. But if I go to a different room in the house, there's not suddenly very quiet compared to normal speakers. I have not A-B tested it, but I think "your neighbours will not be disturbed" is pretty far down on the feature list to consider when purchasing cardioid speakers.