I once tested with an iPhone 5c (3 generations older than the SE) and Objective 2 using my HD650, which most people considered difficult to drive.
With the same output volume I can barely tell the difference in an abx test.
I would mostly agree with you on that one but, contrary to the received audiophile opinion, I don't think the HD650 is that hard to "drive"
Due to it's low-ish sensitivity plenty of people might want more volume than most onboard headphone outs can provide but it's 300 ohm nominal impedance makes it an easy load for such current starved outputs keeping what voltage it can provide clean. With the limited volume in mind, my HD650s sound pretty similar no matter what I plug them into.
Keep in mind I'm not talking about any audiophile magic here, just some just some basic physics you can mostly work out with ohm's law.
Balanced armature IEMs usually need a ~1 ohm output impedance to prevent their FR from changing. Even pleanty of dedicated amps don't met that spec. Others are super sensitive and are very susceptible to hiss, requiring an external amp to be used as an attenuator/buffer in order to kill the hiss by maximizing the dynamic range of the source.
Plenty of full size headphones may sound just fine from onboard sound but most will be limited in volume. Some planars in particular are very inefficient and won't even reach moderate levels without a dedicated amp.
If you're recommending something to someone on a tight budget or who's just looking for something nicer than what they have and who won't be upgrading to a new headphone they like slightly better a few times a year then it won't be to find a set of headphones they'll like and which won't have any of those problems with an iphone's headout that I mentioned above. If one wants to keep their options open and can afford to splurge a little then a external DAC/amp is a better choice.
Versatility is the key point. If a new IEM comes out and reviewers you trust have said good things about it do you want to have to worry about its crazy high sensitivity or impedance curve which looks like a boutique speaker and has enormous peaks in all the wrong places? Are you going to scratch the hot new full size off your list because the sensitivity is too low?
Not worrying about a device's onboard sound also gives you a lot more options when choosing new devices. It frees up a variable and let you focus on other features and specs. It's also a variable that's rarely covered in most tech reviews anyway which makes the ability to ignore it all the more valuable. From the measurements I've seen Apple's products are reliably decent, but that also assumes you like playing in their walled garden or aren't already heavily invested in someone else's. For a lot of people the Topping NX4 or JDS Element would be a bargain compared to re-buying copies or equivilents of all their apps on the other app store.
As for hardware cost, Apple's phones are competitive, but for desktops and laptops a Windows or Linux build plus an separate DAC/amp will be cheaper. A seperate DAC and amp will most likely last longer than the phone, tablet, or laptop one pairs it with and maintain higher resale value to boot.
Dedicated DACs or amps don't always provide audibly better performance in every situation and certainly aren't the most value driven proposition but they win hands down in versatility.
