Besides the physical aspects, which have already been mentioned (build quality, materials, fit, etc), I do believe there's a big part of the sound presentation on headphones and speakers that cannot be measured. For example, I own to open back headphones: Audio Technica ATH-AD900 and Grado SR60e. Both are equalized. The presentation of the music is completely different: the AD900 is a lot more spacious, with a way bigger soundstage, compared to the Grado.
This may have something to do with the size and positioning of the drivers relative to my head, the shape of my ears, etc. but the differences are there, and they're very noticeable. It's probably akin to the effects of a room's shape, size and furnishing in a speaker system.
Measurements are great, as the establish a basis for comparison and, in the case of amps/DACs, they may be able to tell 99% of the story (maybe, maybe not? Maybe someday we'll discover new testing methods, gotta keep an open mind), but in the case of speakers and headphones, there are many other factors.