It probably won't work well when you quickly turn the dial...
I've recently
built a preamp "box" with an arm board (similar to a raspberry pi 3) and amongst other things a rotary encoder that controls the volume of CamillaDSP. I initially ran into issues where if I sent all the "events" when quickly rotating the encoder's dial I'd then have to wait for 2-3 seconds until the volume would "settle". And that's without even the latency of wireless communication (wi-fi or infrared) as the rotary encoder is wired to the same board CamillaDSP is running on.
The solution I came with was to send the first "volume event" to CamillaDSP, then send any "intermediate" event every X milliseconds, then send the final value. That ended up working quite well, closely tracking the encoder dial's movement. However that means you have to store and "aggregate" relative increments - ie. if you quickly rotated the dial reading absolute values like -3, -4, -5, -6, -7 (= 5 "clicks" counterclockwise - starting from -3 because that's what was stored the last time someone rotated the dial), you'd then only send the relative increments to CamillaDSP, eg. -1, -3, -1 (where in that case -1 is the first event/click, then the dial was rotated too quickly so the algorithm aggregated 3 consecutive "clicks" that happened within X milliseconds and eventually sent -3, then the person rotating the dial slowed and -1 was sent.).
With infrared where you only have vol up and vol down you wouldn't be able to send a volume instruction like "volume down by 5 increments"; instead you'd have to send a long series of -1s. Add to this the latency of infrared (no idea what that'd amount to though - both for signal processing and waiting before sending a consecutive event), potential range problems (= missed/skipped events) and you'd probably swear at your remote more often than not.
Solutions: rotate slowly
; or put another preamp/interface before your preamps to manage the volume with something better than infrared (eg. BT or Wi-Fi), leaving your preamp(s) at max vol and ditch your infrared remotes. Wiring would be "transparent" if your interface has enough in/out channels (or enough "out" channels if your sources are digital).
Side note: I used to be an advocate of "the less gear in the signal path the better" but with DACs nowadays exhibiting such SINAD there's really no issue (well, except maybe space consideration, power consumption, and making the setup a bit more complex).
Hope this helps...