For several years, I used the Audeze MM-500 powered straight from the headphone output of an Apogee Symphony (a $5,000 pro DAC, higher z) for my work. I tried switching to the Topping A50 III (very low z) recently, and the reduction in low-end distortion was obvious. Output impedance definitely matters a lot here.
Correlation is not causation . If the Apogee is black and the Topping a silver box, then should it be concluded that the colour is important in how well it drives the headphone?
Of course not, and neither should we conclude the same about output impedance .
However, the output impedance can imply something about the intention of the headphone output design and what they think the intended customer wants from it.
High impedance outputs on studio equipment is intended for deliberately chosen, high impedance, dynamic headphones. If you look at studio headphones, the models often come in different impedence versions for this reason. High impedance helps with very long cable runs, daisy-chaining headphones outputs to more than one musician/engineer and also means the headphone is driven by voltage and not current, meaning the current draw does not effect the working of the equipment's other features, as a converter in the Apogee's case. Headphones outputs on this kind of gear is often for simply checking what is coming out the outputs - not for critical listening
High impedance is not designed for low impedance planar headphones as planars require current delivery and high output mpedance makes this much much harder to do .
The Topping on the other hand is designed to provide high current to drive planars, as well as a high enough voltage capability for dynamics too.
Meanwhile the Apogee is only designed to deliver a specific type of dynamic headphone (a high impedance one).
The power and type of power needed for a headphone depends on the headphones. The Apogee simply is not designed to be powerful enough to work well with everything, whereas the Topping basically is.