That’s actually a whole other issue. Did you connect it with toslink perhaps ?
See here: Chromecast Audio dropouts
It’s the main reason why I purchased the Teufel streamer as a drop in replacement for the Chromecast Audio I had there before. But with the Teufel I have the issue as described in this topic.