Roon Core/connected DAC: which Device should I choose?

I set up my home office in my old music room, so I’ve connected Roon Core via USB —> Schiit Modi 3—>my Citation Receiver. I’ve selected System Output (i.e, the USB connection to my Modi) and controlling the system volume. Should I be using the WASAPI device that Roon set up for the Modi 3? Does it matter?

Depends if they provide ASIO drivers for it. It’s your choice wether to use ASIO or Wasapi. Make sure you have it set to exclusive.