True, some SET amp effect, but not very much in reality. The SET amp behavior is much more complex, and dynamic. These things make them fun to listen too. It is true that if you listen to he SET then say a decent solid state amp, you will notice difference, but personal preference will decide which you prefer, often folks find the SET amplifier more dynamic.
SET amps and output transformers have come a long way from your grand dads tube radio receiver (which many of them actually sound quite nice) while retaining the essential elements of the SET such as ultra low distortion at low power levels (where your music spends most of its time) and dynamics due to the tube/transformer/speaker interaction and what I call the chorus effect sound, kind of like reverb but not reverb.
Good SETs produce very few harmonics higher than the fourth, and that reduce IM quite a bit, but they make up for that by producing a whole bunch of more close in tones so in some ways its a wash.
Also the group delays contribute in some SETs to a "fat" sound which can be quite pleasing. And these units can be made with no feedback, where only some solid state amps can do that, and they have to separate the audio signal in to positive and negative halves in order to do that, which is never really ultimate high fi, more hi fi than a SET, yes, but not ultimate.
I don't think output transformer design has progressed much since the tube era.. More to the point, affluence has allowed many users to afford higher quality past designs. The hysteresis in OPs is always going to add distortion without feedback.
As you say, it is preference.