This is not a typical issue that everyone is having when using Roon, so it sounds like there is something else going on here.
So we can better assist you, please provide a brief description of your current setup using this link as a guide.
Make sure to describe your network configuration/topology, including any networking hardware currently in use, so we can have a clear understanding of how your devices are communicating.
Is this happening with all Zones?