Anecdotal evidence: went headphone shopping with my Step-Dad, brought along my iPhone with Mojo with some high bitrate MP3s for reference audio. Spent some time, he ended up liking the HD650s the best for the $$.
Got home, he's listening to some other stuff via iPad or direct from computer. $2 DAC, ya know. And he says "Hey there's something missing, it just doesn't sound as good." Hook the Mojo into the signal path and "Ah, that's better. What's that thing cost?"
So I got him a Mojo for his B-day.
Anyhow, I've heard difference from my Burr-Brown on the integrated amp vs the Cirrus on the CD player and just decided to get a RME ADI-2. Very happy with the RME. I heard differences between each but they were subtle and if the speakers weren't very good or the amp or CD transport... no difference would be found between the DACs.
So, sure, if you are playing through amp & speakers that aren't very good then it doesn't matter how good the signal you give it.