Tracks Slow to Load & Playback Randomly Stops on Synology 3615XS+ Roon Core

Hi @c2c2c2,

All I need here are the exact local times in your country of when this issue occurs, and after receiving this I can enable diagnostics mode and what that will do is automatically upload a set of logs to our servers for analysis.

If you wish, you can also note the timestamp and manually send me the logs using these instructions, a shared Dropbox/Google Drive or Firefox Send link would be preferable.

– Noris

@Noris,

Okay, thank you. Here are three time stamps to investigate:

4.30.2019 @17:10 to 17:30 Very slow to start playing
4.30.2019 @20:30 Very slow to start playing
5.1.2019 @05:30 This is an example of playing radio and the system just stopping.

Control was always from an IOS device. iPhone (mostly) or iPad. Both current and top-end models.

Hi @c2c2c2,

Thank you for letting me know those timestamps, now that I have this information I have enabled diagnostics mode for your Core and what this will do is automatically upload a set of logs to our servers for analysis. I’ll be sure to reach out to you once more when this report has been reviewed.

– Noris

Okay @noris, thank you. I will wait to hear from you.

Regards.

@noris,

As one further data point. I can play to this Bluesound node, the very same High Res files, across the very same network connection, but using the Bluesound app instead of Roon. When I do that, it never stops playing, there is never a delay in starting the music, it works perfectly. The only difference being I’m not using Roon. I mention this as it would be another point of elimination of a hardware or network problem - which I fear is the diagnosis when it can’t be figured out.

Regards.

Hi @c2c2c2,

Thanks for letting me know that additional timestamp. I am taking a look through your diagnostics and here is what I am seeing at the timestamps you mentioned:

04/30 17:12:09 Trace: [Master Bedroom] [Enhanced, 24/44 FLAC => 24/96] [100% buf] [STOPPED @ 0:00] Rock It Till the Cows Come Home - Doug MacLeod
04/30 17:12:09 Trace: [dbperf] flush 0 bytes, 0 ops in 3 ms (cumulative 32356340 bytes, 15836 ops in 72510 ms)
04/30 17:12:09 Warn: [raat_ll/client] [Bluesound POWERNODE 2i @ 10.0.0.46:43563] error writing to connection: System.NullReferenceException: Object reference not set to an instance of an object
  at Sooloos.Audio.Raat.Client._BeginWrite () [0x00053] in <20e726da17014e3fbb7592d73803b940>:0 
04/30 17:12:09 Trace: [raat] [Bluesound POWERNODE 2i @ 10.0.0.46:43563] => Disconnected
04/30 17:12:09 Trace: [raat] [Bluesound POWERNODE 2i @ 10.0.0.46:43563] lost client connection. Retrying
04/30 17:12:09 Trace: [push] restarting connection (Unable to read data from the transport connection: interrupted.)

It looks like your Powernode lost connection to your core and around the same time your Core also lost connection to our servers. More traces:

04/30 17:13:31 Warn: [raat/tcpaudiosource] send failed: The socket has been shut down
04/30 17:13:31 Warn: [raat/tcpaudiosource] disconnecting + retrying
04/30 17:13:31 Debug: [raat/tcpaudiosource] disconnecting
04/30 17:13:32 Info: [stats] 5030mb Virtual, 1825mb Physical, 1011mb Managed, 0 Handles, 69 Threads
04/30 17:13:35 Trace: [Master Bedroom] [HighQuality 13.6x, 24/176 FLAC => 24/96] [100% buf] [PLAYING @ 0:00] Rock It Till the Cows Come Home - Doug MacLeod
04/30 17:13:39 Warn: [Master Bedroom] [zoneplayer/raat] Endpoint failed to become ready in 10s. Proceeding in state Buffering
04/30 17:13:39 Trace: [Master Bedroom] [zoneplayer/raat] Endpoint Bluesound POWERNODE 2i State Changed: Buffering => Ready
04/30 17:13:39 Trace: [Master Bedroom] [zoneplayer/raat] wait for ready in 10000ms
04/30 17:13:39 Trace: [Master Bedroom] [zoneplayer/raat] Adjusting playback start offset from 50ms to 220ms
04/30 17:13:39 Trace: [Master Bedroom] [zoneplayer/raat] Doing 'ASAP' Start since we are just playing to one device
04/30 17:13:39 Trace: [Bluesound POWERNODE 2i @ 10.0.0.46:43563] [raatclient] SENT [18]{"request":"start","min_offset":220000000,"stream_sample":0}
04/30 17:13:39 Trace: [Master Bedroom] [zoneplayer/raat] Endpoint Bluesound POWERNODE 2i State Changed: Ready => Playing
04/30 17:13:39 Trace: [Bluesound POWERNODE 2i @ 10.0.0.46:43563] [raatclient] GOT [17] {"status":"Playing"}
04/30 17:13:39 Trace: [Bluesound POWERNODE 2i @ 10.0.0.46:43563] [raatclient] GOT [18] {"time":173968549846,"status":"Success"}
04/30 17:13:39 Debug: [raat/tcpaudiosource] connecting to 10.0.0.46:40799
04/30 17:13:39 Debug: [raat/tcpaudiosource] connected
04/30 17:13:39 Warn: [raat/tcpaudiosource] send failed: Connection reset by peer
04/30 17:13:39 Warn: [raat/tcpaudiosource] disconnecting + retrying
04/30 17:13:39 Debug: [raat/tcpaudiosource] disconnecting
04/30 17:13:39 Debug: [raat/tcpaudiosource] connecting to 10.0.0.46:40799
04/30 17:13:39 Debug: [raat/tcpaudiosource] connected
04/30 17:13:40 Trace: [Master Bedroom] [HighQuality 32.8x, 24/176 FLAC => 24/96] [100% buf] [PLAYING @ 0:00/4:10] Rock It Till the Cows Come Home - Doug MacLeod
04/30 17:13:41 Warn: [raat/tcpaudiosource] send failed: The socket has been shut down
04/30 17:13:41 Warn: [raat/tcpaudiosource] disconnecting + retrying
04/30 17:13:41 Debug: [raat/tcpaudiosource] disconnecting
04/30 17:13:41 Debug: [raat/tcpaudiosource] connecting to 10.0.0.46:40799
04/30 17:13:41 Debug: [raat/tcpaudiosource] connected

Looks like your Powernode disconnected yet again and is having issues maintaining a stable connection here. Here is what I saw regarding your iPhone:

04/30 17:14:18 Trace: [raatserver] [RaatServer c2s-iPhone @ 10.0.0.5:50519] lost client connection. Retrying
04/30 17:14:18 Trace: [raatserver] [RaatServer c2s-iPhone @ 10.0.0.5:50519] connecting (attempt 1)
04/30 17:14:18 Trace: [push] restarting connection (Unable to read data from the transport connection: interrupted.)
04/30 17:14:18 Trace: [push] retrying connection in 34207ms
04/30 17:14:18 Warn: [rnet/RnetJsonClient] failed to connect Connection refused
04/30 17:14:18 Trace: [raatserver] [RaatServer c2s-iPhone @ 10.0.0.5:50519] client connection failed. Retrying in 500ms
04/30 17:14:19 Trace: [raatserver] [RaatServer c2s-iPhone @ 10.0.0.5:50519] connecting (attempt 2)
04/30 17:14:19 Warn: [rnet/RnetJsonClient] failed to connect Connection refused
04/30 17:14:19 Trace: [raatserver] [RaatServer c2s-iPhone @ 10.0.0.5:50519] client connection failed. Retrying in 750ms
04/30 17:14:20 Trace: [raatserver] [RaatServer c2s-iPhone @ 10.0.0.5:50519] connecting (attempt 3)
04/30 17:14:20 Warn: [rnet/RnetJsonClient] failed to connect Connection refused
04/30 17:14:20 Trace: [raatserver] [RaatServer c2s-iPhone @ 10.0.0.5:50519] client connection failed. Retrying in 1125ms
04/30 17:14:20 Trace: [Master Bedroom] [HighQuality 34.1x, 24/176 FLAC => 24/96] [100% buf] [PLAYING @ 0:41/4:10] Rock It Till the Cows Come Home - Doug MacLeod
04/30 17:14:21 Trace: [raatserver] [RaatServer c2s-iPhone @ 10.0.0.5:50519] connecting (attempt 4)
04/30 17:14:21 Warn: [rnet/RnetJsonClient] failed to connect Connection refused
04/30 17:14:21 Trace: [raatserver] [RaatServer c2s-iPhone @ 10.0.0.5:50519] client connection failed. Retrying in 1687ms
04/30 17:14:23 Trace: [raatserver] [RaatServer c2s-iPhone @ 10.0.0.5:50519] connecting (attempt 5)
04/30 17:14:23 Warn: [rnet/RnetJsonClient] failed to connect Connection refused
04/30 17:14:23 Trace: [raatserver] [RaatServer c2s-iPhone @ 10.0.0.5:50519] client connection failed. Giving up

At the 20:30 Timestamp:

04/30 20:32:40 Trace: [raat] [Bluesound POWERNODE 2i @ 10.0.0.46:43563] => Disconnected
04/30 20:32:40 Trace: [raat] [Bluesound POWERNODE 2i @ 10.0.0.46:43563] lost client connection. Retrying
04/30 20:32:40 Trace: [raat] [Bluesound POWERNODE 2i @ 10.0.0.46:43563] connecting (attempt 1)
04/30 20:32:40 Trace: [raat] [Bluesound POWERNODE 2i @ 10.0.0.46:43563] => Connecting

At the 5:30 Timestamp:

05/01 05:31:47 Trace: [Master Bedroom] [Enhanced, 24/96 FLAC => 24/96] [100% buf] [PLAYING @ 1:50/3:53] Something You Get Through - Willie Nelson
05/01 05:31:48 Warn: [raat/tcpaudiosource] send failed: Connection reset by peer
05/01 05:31:48 Warn: [raat/tcpaudiosource] disconnecting + retrying
05/01 05:31:48 Debug: [raat/tcpaudiosource] disconnecting
05/01 05:31:48 Debug: [raat/tcpaudiosource] connecting to 10.0.0.46:40703
05/01 05:31:48 Debug: [raat/tcpaudiosource] connected
05/01 05:31:50 Info: [stats] 5226mb Virtual, 1360mb Physical, 1015mb Managed, 0 Handles, 74 Threads
05/01 05:31:50 Warn: [raat/tcpaudiosource] send failed: The socket has been shut down
05/01 05:31:50 Warn: [raat/tcpaudiosource] disconnecting + retrying
05/01 05:31:50 Debug: [raat/tcpaudiosource] disconnecting
05/01 05:31:50 Debug: [raat/tcpaudiosource] connecting to 10.0.0.46:40703
05/01 05:31:50 Debug: [raat/tcpaudiosource] connected
05/01 05:31:50 Warn: [raat/tcpaudiosource] send failed: Connection reset by peer
05/01 05:31:50 Warn: [raat/tcpaudiosource] disconnecting + retrying
05/01 05:31:50 Debug: [raat/tcpaudiosource] disconnecting
05/01 05:31:50 Debug: [raat/tcpaudiosource] connecting to 10.0.0.46:40703
05/01 05:31:50 Debug: [raat/tcpaudiosource] connected
05/01 05:31:50 Warn: [raat/tcpaudiosource] send failed: Connection reset by peer
05/01 05:31:50 Warn: [raat/tcpaudiosource] disconnecting + retrying
05/01 05:31:50 Debug: [raat/tcpaudiosource] disconnecting
05/01 05:31:51 Debug: [raat/tcpaudiosource] connecting to 10.0.0.46:40703
05/01 05:31:51 Debug: [raat/tcpaudiosource] connected

I know you don’t want to hear this, but it clearly looks like your Core is disconnecting from both your iPhone and Bluesound around the same time, so whatever the issue is it is most certainly networking related. I have a few follow-up suggestions for you here:

  1. Can you let me know if your Bluesound zone has the newest firmware installed?
  2. Do you by any chance have “Open VSwitch” enabled on that Synology? (See this thread for more info)
  3. Have you tried a different Ethernet cable into the Synology Core yet? Might be worth a try in case the Ethernet cable itself has issues (I have seen this too often before).

Please let me know if any of my suggestions are helpful.

– Noris

@noris,

Thank you for the reply. I saw that in the logs myself, I was hoping you might have more insight into the cause.

Regarding your questions:

If you’re seeing a loss of connection to your servers as well, I’m wondering if there is some interaction issue with the Roon core and DSM (Synology O/S).

Have you run into any issues with your network traffic traversing a SFP+ connection? I have a Synology card in my NAS which gives me the 10GB fiber connect to my router. In theory the physical connection is lower in the network stack and should have no impact. Further, I’ve run with this config for a couple of years and have lots of different network traffic across it with no problems… perhaps Roon has an issue? As a test, I will try both a different fiber cable and also connecting with just a regular 1GB Ethernet cable.

@noris

I have confirmed that the problem is not resolved by replacing the SPF+ link from the NAS to the router with a standard ethernet (copper) connection. This would rule out the cable theory and also eliminate anything related to transiting on fiber.

I understanding you suspect a network problem, however, in addition to the various previously described tests, it is suspicious that no other application or use has any performance or reliability issues. These include:

  • Streaming Hi-resolution music to Bluesound using Bluesound native application.
  • Streaming Video, Apple TV, Netflix, Hulu, HBO GO.
  • Video and photo editing of files on NAS (CIFS)
  • Analytical work with large data sets on the NAS (CIFS)
  • Home automation device connectivity: WEMO, Hue, etc.
  • Wireless Cameras: Reolink
  • Other file protocols: FTP, NFS

And of course, I’ve tested Roon with all of these other network applications disconnected. Perhaps there is some issue related to the stability of the RAAT protocol? Although this would not explain a loss of connectivity to your servers as I don’t think you use RAAT for that purpose?

Hi @c2c2c2,

Thanks for running the test with the different Ethernet cable. I spoke to the team regarding this issue and we have a few other suggestions:

  1. Can you please send us the logs from your Bluesound zone? You can get these logs by navigating to Powernode IP Address in a web browser -> Diagnostics -> More -> Copy and paste the text into a text editor and save the info and send it as a shared Dropbox/Google Drive/Firefox Send link. If this issue occurs on both Node as well, please send those too.

  2. Do you by any chance have another Core around that you can use to temporarily host Roon on? It would be interesting to see if another Core is able to reproduce this behavior for the same zone.

  3. Is there any change if you temporarily plug the Powernode 2i via Ethernet to eliminate the WiFi from the equation?

– Noris

@noris,

I can get you the BS log, I monitor that myself (it’s annoying that their time stamps aren’t in local time as you no doubt know). To be certain that you can align all the logs, let me get you a clean set of contemporaneous logs from Roon and Bluesound. It will take me a day or so as I’m not home now.

I have a dual-CPU 56 core windows workstation on which I can put a core that’s network wired. I’ll setup that test as well, it’s a good suggestion.

On question #3, I’ve not tried that as getting the speaker cables on and off the BS is a real pain… I can certainly do it however as it’s the next obvious test.

I’ll get back to you on the above.

Cheers.

1 Like

@noris,

I thought I would get back with my preliminary result. I tried your suggested of moving my Roon core to another system (a Windows workstation). When running on the Windows system and using the same library files, it appears that everything works fine. There is no hesitation in starting tracks, mid playback stops, etc.

This Workstation is physically wired to the same router as the Synology, which points to a problem with Roon running on Synology. I noted you’ve handled a similar issue before:

Synology Roon Problems

Collectively, this indicates it’s not a network issue per se. I’m doing nothing fancy with the network interface on the Synology, MTU of 1500, etc.

There are no aggresive power off settings on the Synology, and as you can see it’s a top-of-the-line model with capacity to spare. Furthermore, I’ve confirmed the problem occurs when the Synology is otherwise idle so I’m confident it has nothing to do with capacity there.

I will continue to run in this config for awhile to see if anything goes bad, but the problem was always quickly repeatable so I suspect it will continue to be stable.

What’s next?

Hi @c2c2c2,

Thanks for sharing those preliminary findings with me, it does indeed to appear to be something related to the NAS as a Core that is triggering this issue. The next steps would be:

  1. Confirm if there is any change in behavior if you plug in the Bluesound zone via Ethernet instead of WiFi.

  2. Make sure that no other apps are interfering with Roon on the Synology NAS. I know you mentioned you have quite a few things on there: analytic work programs, video editing tools, home automation stuff, ect. What happens if you leave just Roon running on it, does the same issue occur?

– Noris

@noris,

As mentioned previously, this happens with wired as well as wireless nodes. The problem can be more pronounced on wireless nodes, it is however there nonetheless on wired nodes as well showing all the same symptoms.

Yes, of course. All these results were confirmed when the NAS was doing nothing else - this includes any housekeeping chores that can be scheduled for background execution.

Note that these apps are all really just file access as the computing runs on a workstation. However, I can confirm that no file access or other processes were running when the testing of Roon was conducted - is was just Roon.

Hi John,

Thanks for giving that a try. I’m going to reach out to our QA team and ask if they have seen any similar reports for your NAS model. If you make any new findings over the weekend do let me know. Maybe it’s worth to check if you have then newest Roon package on the NAS or reinstall the app?

– Noris

Thanks @noris,

I do have the latest Roon Core installed. I may try for a complete de-install and re-install on the NAS. I presume the backup will get everything? Playlists, ratings, Meta, etc.?

@noris,

I conducted some additional tests this weekend as follows:

  1. I deinstalled Roon entirely from the NAS, then reinstalled and restored from backup. RESULT: no improvement.
  2. I desintalled Roon entirely from the NAS, then reinstalled on a different volume – this time rather than on the SSD I installed on a HDD volume. I then restored from backup. RESULT: no improvement.

Hi John,

Thanks for running those tests. Just to confirm something here, does this same behavior occur for both Local Library tracks or do TIDAL/Qobuz tracks show the same symptoms?

I would next like to take a closer look at the SFP+ connection, can you reproduce the same behavior when you are connected via a regular Ethernet cable and if so can you let me know some timestamps?

– Noris

@noris,

Yes, the same behavior occurs with either local library tracks or TIDAL, I get the same symptoms. I do not use Qobuz.

As you may recall, I’ve already taken SFP+ out of the equation, here is my previous message on that.

Hi @c2c2c2,

Yes I do recall that you took the SPF+ out of the equation before but I have not had a chance to review the diagnostics with just the Ethernet cable connected and the SPF+ out of the way. Can you try connecting it via Ethernet once more and let me know the timestamp if the behavior is the same so I can review and see if the diagnostics traces are identical?

– Noris