Correct, in this case it was connected to an amp that has a bypass mode that simply passes signal through unaltered to avoid any potential influence the amp's signature might have to the comparison.
Yea it was obviously volume matched or else would not be very useful comparison, I even went to go as far as lowering the DAC's volume one step whenever I found something having an advantage and listen again. The differences found are no way huge and mostly noticeable because of the quick switching possibility. Having mastered music for 14 years as a hobby paying close attention to various differences when making small adjustments to EQ, stereo width/soundstage, phase adjustments, reverb, dynamics control/compression/limiting, obviously helps in these cases too (trained ears).