What you will also find sometimes are inrush current limiting resistors or NTCs on power supplies that are bypassed by a relay once the DC supply rails have ramped up.
The most common use of a delay relay seen here is (from my observation) what PaulD mentioned, headphone/Speaker protection purposes. Depending on the design, it might be a simple delay or delay combined with a fault detection (most times DC-Offset) circuit.