I have the exact same problem, with a very similar setup, although my Roon core (server) Mac Mini is connected to the LS50W via USB input.
Interestingly, if I connect via LAN (ethernet) the issue does not arise. But via USB, the symptoms are identical to the OP
I have come to this thread via Google, so clearly there is something going on! Any updates or fixes much appreciated, thanks