I also don't think the architecture is the issue, but what is an issue is the bad build quality of many, and the biggest issue is that IC's go fast out of production so repairing class D amps is a lot harder than repairing Class A or AB amps as parts are harder to find. And repairing an SMD board is also harder than through-hole components. But those are not limited to class D, those are general issues in modern electronics. It's less durable due to the way the systems are build physically, not due to the architecture of the amp.
And yes, i'm also aware of the advantages of IC's and SMD, they are cheaper to build, smaller, and take less natural resources. But they are not a free lunch...