The irritation for me comes from the fawning public and the fawning media, who want to invent a fantasy about the DoE's lifetime of hard work and service. My dad was the same age, likewise fought in WW2 (army not navy) and then grafted hard for 50 years, responsible for himself and his family, often broke, often struggling, never giving up, always soldiering along.
Whereas the DoE had it pretty easy - zero money worries, everything done for him, mollycoddled, chauffeured everywhere, etc, etc. No knock against him personally, but it's crazy that ordinary folk (who all had dads like mine) can't see the difference.