Trouble connecting to Roon on Debian Linux container (ref#EXUXMH)

[Jeremy_Andrews]

Full form submission

What’s happening?

I'm having trouble connecting to Roon

This morning Roon was non-responsive on my mobile devices and laptop. Roon is running on Debian in a Linux container (Proxmox managed). I tried restarting the roonserver service, as well as restarting the container. Later I powered down all of my network speakers and tried again, but it still doesn't work.

On iOS I just get the half-circle icon with lines dancing next to it, forever. On macOS I just see Waiting for your Roon Server.

I do see that the Roon process is busy:

top - 11:56:29 up 8 min, 1 user, load average: 48.13, 50.16, 51.79
Tasks: 20 total, 1 running, 19 sleeping, 0 stopped, 0 zombie
%Cpu(s): 23.8 us, 28.4 sy, 0.0 ni, 45.4 id, 0.0 wa, 0.0 hi, 2.4 si, 0.0 st
MiB Mem : 40560.0 total, 38587.2 free, 1357.4 used, 615.5 buff/cache
MiB Swap: 8192.0 total, 8192.0 free, 0.0 used. 39202.6 avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
300 root 20 0 8715812 1.5g 372320 S 100.3 3.8 10:51.00 RoonApp+

The roonserver service status is as follows:

● roonserver.service - RoonServer
Loaded: loaded (/etc/systemd/system/roonserver.service; enabled; preset: e>
Active: active (running) since Sun 2024-02-11 11:48:12 UTC; 11min ago
Main PID: 115 (start.sh)
Tasks: 86 (limit: 308936)
Memory: 2.3G
CPU: 14min 19.951s
CGroup: /system.slice/roonserver.service
├─115 /bin/bash /opt/RoonServer/start.sh
├─125 /opt/RoonServer/RoonDotnet/RoonServer RoonServer.dll
├─300 /opt/RoonServer/RoonDotnet/RoonAppliance RoonAppliance.dll ->
├─304 /opt/RoonServer/Server/processreaper 300
└─350 /opt/RoonServer/RoonDotnet/RAATServer RAATServer.dll

Feb 11 11:48:17 roon start.sh[125]: 00:00:00.006 Info: get lock file path: /tm>
Feb 11 11:48:17 roon start.sh[125]: 00:00:00.040 Info: GetLockFile, fd: 34
Feb 11 11:48:17 roon start.sh[125]: 00:00:00.041 Info: GetLockFile, res: 0
Feb 11 11:48:17 roon start.sh[125]: 00:00:00.042 Trace: Nope, we are the only o>
Feb 11 11:48:18 roon start.sh[125]: Initializing
Feb 11 11:48:18 roon start.sh[125]: Started
Feb 11 11:48:23 roon start.sh[125]: Not responding
Feb 11 11:48:27 roon start.sh[300]: aac_fixed decoder found, checking libavcode>
Feb 11 11:48:28 roon start.sh[300]: has mp3float: 1, aac_fixed: 1
Feb 11 11:48:38 roon start.sh[125]: Running

When I restart the iOS client, for example, I get the following in the logs:

02/11 12:00:48 Info: [stats] 8832mb Virtual, 2064mb Physical, 1461mb Managed, 262 Handles, 49 Threads
02/11 12:00:48 Trace: [remoting/brokerserver] [initconn 10.20.235.158:54145=>10.20.100.202:9332] Connected
02/11 12:00:48 Trace: [remoting/brokerserver] [initconn 10.20.235.158:54145=>10.20.100.202:9332] Initialized Fresh Session
02/11 12:00:48 Trace: [remoting/brokerserver] killing old client from 620a61bc-60e3-49f7-8a53-251a167eeb5f because we have a new one
02/11 12:00:48 Info: [remoting/serverconnectionv2] Client connected: 10.20.235.158:54145
02/11 12:00:48 Trace: [remoting/remotingserverv2] SENT NONFINAL DistributedBroker.ConnectResponse={ BrokerId=514bc3f5-c3ee-4a86-aa3e-5fc5bb464766 BrokerName='epyc' }
02/11 12:00:48 Trace: [remoting/remotingserverv2] SENT NONFINAL DistributedBroker.UpdatesChangedResponse={ IsSupported=True WasJustUpdated=False Status='UpToDate' HasChangeLog=False CurrentVersion={ MachineValue=200001365 DisplayValue='2.0 (build 1365) production' Branch='production' } }
02/11 12:00:49 Debug: [easyhttp] [30] GET to https://api.roonlabs.net/messaging/1/api/messages/6a6cbd50-86eb-4be4-8187-2c668a148b43 returned after 594 ms, status code: 200, request body size: 0 B
02/11 12:01:03 Info: [stats] 8960mb Virtual, 2085mb Physical, 1490mb Managed, 262 Handles, 70 Threads
02/11 12:01:18 Info: [stats] 8800mb Virtual, 2103mb Physical, 1508mb Managed, 262 Handles, 46 Threads

I'd appreciate help debugging why the server has stopped working, and getting it working again!

[Menzies]

What happens when you click on here?

[Geoff_Coupe]

Hallo @Jeremy_Andrews

Running Roon in a VM or Docker is not an officially supported environment. It is considered Tinkering. Roon Labs doesn’t prevent customers from experimenting with their software in unsupported environments; however, when you do so, you also take on the ongoing support and troubleshooting for that environment.

I wanted to provide a brief explanation before moving this [thread/post] to the Tinkering section. Hopefully, other customers who may also be using your environment can help you with diagnosing and resolving your issue.

If this does not work for you, then I would recommend running Roon in an officially supported environment for the stability and support you are looking for.

Having said all that, the suggestion from @Menzies is definitely worth trying.

[Jeremy_Andrews]

During my debugging I did click that button at one point, and it told me I would lose access to my music and playlists. I didn’t pursue that path, instead I went to power cycling everything. Having done that, Roon on macOS is now showing the same screen as on iOS:

Can someone point me at which logs could tell me why Roon is consistently using 100% of the CPU, but not responding? Are there any ways to debug into what is going wrong?

I currently have 16T of music. I’ve not added any for a while, and it was playing fine and regularly up until this morning. The only change I’ve made is I used the BluOS app to turn off a speaker light at which time the speaker became non-responsive: I’m still debugging that, and have no idea if it’s related.

Looking through your supported devices, it seems the Nucleus isn’t supported outside of the US? So I need to assemble a standalone Linux Ubuntu server, load the music from the NAS (what is Roon’s recommendation there? With Proxmox I can use a loopback, but with pyhsical hardware it will be a network mount - do you recommend/support NFS? Samba?). I’ll install this on supported hardware once I properly understand a supported configuration that will work in my network environment.

[Geoff_Coupe]

@Jeremy_Andrews

Looking through your supported devices, it seems the Nucleus isn’t supported outside of the US?

Sorry - where did you get that impression? The Nucleus is supported worldwide by the Support team

[Jeremy_Andrews]

By not reading carefully! I got as far as the “buy now” button, and didn’t read underneath. Sorry.

My Roon server has finally started and is working normally again: it seems like power cycling everything helped, it just took a long time to come up again.

[Geoff_Coupe]

@Jeremy_Andrews

it just took a long time to come up again

It will do. My Library has 70,000 tracks and it takes several minutes for Roon Server to be fully up and running (on a ROCK/NUC i5 system)

[Jeremy_Andrews]

It was up briefly, but then when back to displaying the Roon logo. Any ideas on where I could look in the logs or elsewhere to understand where it’s spending all it’s time and why it’s non-responsive?

[Jeremy_Andrews]

In the end, without any ideas on how to debug, I built a new container, installed a fresh copy of Roon, then imported my latest backup. I pointed my clients to the new container and it’s working fine again. I do not know what was wrong, but I’m happy to have my music back.

[Jeremy_Andrews]

A final update: I’ve fixed all the problems I’ve had by making the following changes on my Proxmox server, after reviewing the best practices in this forum:

• I’ve moved the Roon container (and therefor its database) to a standalone and very fast SSD - it is a single point of failure, so I’ve scheduled daily backups to my highly redundant ZFS pool

• I added a mirrored ZIL with two additional fast SSD’s (a ZFS write-cache) to the ZFS pool where the music is stored

• I added a final fast SSD and enabled a L2ARC (this had the smallest impact on overall performance) to the ZFS pool where the music is stored

With these changes, I’m using my original Roon installation, and everything is extremely snappy. All performance issues that I was seeing previously have gone away.