You could try connecting the Denon via HDMI to give support for 5.1 multichannel - that works for a Nucleus or a NUC running ROCK, but I’m not sure whether it would work with a QNAP.
There’s a long thread about connecting via HDMI here.
Does the qnap have usb out playback? If so, you could get a usb-> s/pdif converter and run it to the optical or coax digital input on your denon.
I use a Topping D10 (less than $100) for exactly this purpose (denon x3300w). It can also be used as a dac for playing dsd files. (I don’t use it for this, but tested it to see if it works and it does.) With the D10 connected via toslink I can play any files up to 96k via wasapi (don’t know if qnap has something similar) without any resampling/downsampling like you get with airplay. For files > 96k you have to install a topping asio driver. Don’t know if you can do that on qnap.
Also don’t know if any of this works for multi-channel. Probably not.
I started out using hdmi, which sounded great and supports any resolution (but not dsd). I switched to toslink via d10 because denon handshaking problems (related to source switching or zone 2, not sure) kept resetting the hdmi port to 16/48 only.