I would suggest you try it and see if you can notice a difference. I use my stack (same as you but A50 instead of A50s) and have the amp at about 11 o'clock and then use the remote for the DAC volume. It can be anywhere from -20 to about -5 and always sounds great. This is with Sennheiser HD660s...
Well I can tell you that with my HD660s I have the D50s between -10 and -20 and the A50 (note not A50s) at about 11 o'clock. This is with a balanced cable.
Not sure if that helps due to the older A50, but regardless I love the sound
Thanks
-Mat