As Allan mentioned in the post above, playing to Chromecast groups can definitely be pretty hard on the network, so it may be networking difficulties that are causing this behavior.
I’d like to enable diagnostics on your account so the team can take a closer look to verify. Can you reproduce the error once more and make a note of the time the error occurs? Respond here with the timestamp and your timezone and I’ll enable diagnostics.