Just a thought, but I would look at the network. Can you change the wiring on a temporary basis to isolate some components and see what the effect is? E.g. take out the switch or the WAPs.
It’s just that I found a supposedly modern switch with management capabilities introduced dropouts and skipping with Roon. It sounds similar to your issue.