I think this come down to the question, what is HiFi and what is it for? Yes, today's portable devices are objectively HiFi in terms of technical performance, the dBs and kHz are all good, but I question the use to which this HiFi is put to, and whether that then constitutes HiFi listening.
Wearing headphones on the bus or in the office implies that the wearer is doing something else at the time, and consequently listening is not the primary activity on which the wearer is concentrating on. It's the personal equivalent of in-store background music, an accompaniment to other activities. Is it therefore correct to call it HiFi? The equipment may be, but the circumstances aren't.
That's why I cannot find that listening to headphones, however good, is HiFi in the way that at home, sitting quietly concentrating on the music spread out in front of one, in the way it is at a concert or gig can be.
S.
Headphones are to HiFi as condoms are to sex.