Of course I do not have golden ears - they are platinum certified! Seriously though, this was not a case of straining to hear a difference - it was obvious.
I believe what you heard is true. Maybe the soundcard output was a bad match for the HD650 for some specific reason. I think there are more variables involved than just impedance matching, that's why some combinations sometimes don't work.
Do you think the AKG K240 Monitor (Austrian made, 600 ohm) is hard to drive? It is much harder to drive than the HD650. You can look at the specs. We are talking 600 ohm, 88 dB/mW vs 300 ohm, 100 dB/mW. Technically you would "need" an amplifier to drive such a beast to its absolutely full potential, shouldn't you? The common perception is that you need a heavy duty amp to drive them. For example, take a look at the video below:
As I am typing this I am listening to the Blade Runner Vangelis OST issue from 1994 on my K240M 600 ohm using a portable mp3 player that is powered by an AAA battery, and I can't even listen to it at 100% volume because it is a little too loud (I listen to it at either 90% or 95% depending on the source). There is no distortion anywhere or flaws I can notice by A/B comparison with my O2/ODAC. I have been testing this morning, trying to convince myself that the bass is slightly better or more controlled from the O2, but I simply wouldn't be able to pass a blind test.
If an AAA battery can power for 7 or 8 hours a beast of a headphone that many people claim you need a nuclear reactor to drive properly, what does that tell you? It's probable it's not as much about having a powerful dedicated amp, but about having a well designed source whose integrated amp matches well with whatever you want to run.
The mp3 player I use has a Sigmatel DAC, which is the same thing that drove the iPod Shuffle 1st Generation. Back then it was praised for its musicality and full spectrum reproduction (including a deep and controlled bass). But it runs off an AAA battery, and I am listening to it right now. I am not trying to convince anyone of anything, but hearing is believing. I just hope folks could be a little more open minded.