It's certainly durable but it's quite firm. When I first put on my XS's it's OK, but after 5-10 minutes I can feel it on the top of my head. I don't know why HiFiMan could not have made a real effort to put a super soft material in there but this seems to be an afterthought for many manufacturers. I've got a few other headphones modded with the OEM Sennheiser 650 headband pads (Audio Technica ATH-M40x, Drop HE-X4, SIVGA SV021) for just this reason--the hard headband pad-- and they really do the trick for me. The OEM is critical as you can depend on the padding material being soft, which is the whole point.
However, for my XS's I've modded mine with a Sennheiser 4-bump pad (like on the old HD-600's) attached with some thin velcro ties and it's really comfortable now and gives a bit more clamp too. This pad was one of those aftermarkets from Amazon that I bought a couple of years ago and I got lucky because it had the OEM-like very soft pads. I since got another 4-bumper that has much firmer pads and it, for me, is not usable. An OEM 650 pad would also work, as I tried one of those also, but the bulk of the 4-bumper made more sense for Edition XS. Plus it's a little easier to attach unobtrusively.
There's lots of solutions for modding the headband pad (search headband mod on this thread and there's an XS mod thread on head-fi) so, for me, this was not going to be a deal killer, I just knew I'd have to deal with it at some point.