Core Machine (Operating system/System info/Roon build number)
I run Roon inside a docker image, hosted on my Kubernetes homelab cluster. So I understand that this isn’t necessarily a well supported configuration, but I’m hoping someone can help me understand whats going on here, because the logs and errors are not terribly descriptive.
Network Details (Including networking gear model/manufacturer and if on WiFi/Ethernet)
Unifi Gear. Devices connected via Ethernet.
Audio Devices (Specify what device you’re using and its connection type - USB/HDMI/etc.)
HifiBerry’s in different places.
Description Of Issue
Roon works fine most of the time, but frequently will have hiccups where it will crash, and then the stdout logs just print
Initializing
Started
Error
Over and over. There are usually about 20 log files in the /var/roon/RoonServer/Logs/ directory, which all complain about this NullReferenceException, not sure if this is related.
03/01 00:08:05 Info: Starting RoonServer v1.8 (build 764) stable on linuxx64
03/01 00:08:05 Trace: Checking if we are already running
03/01 00:08:05 Warn: get lock file path: /tmp/.rnsems0-roon
03/01 00:08:05 Trace: Nope, we are the only one running
03/01 00:08:05 Info: Is 64 bit? True
03/01 00:08:05 Info: Command Line Argument: -watchdogport=44641
03/01 00:08:05 Trace: [childprocess] using .NET child process
03/01 00:08:05 Trace: [realtime] fetching time from NTP server
03/01 00:08:05 Info: [broker] starting d1e776f8-1d40-4b7a-974f-951b58e9558a
03/01 00:08:05 Trace: [httpcache] loaded 864 cache entries from /var/roon/RoonServer/Cache/httpcache_2.db, current: 92mb / 128mb
03/01 00:08:05 Trace: [realtime] Got time from NTP: 03/01/2021 00:08:05 (3823546085610ms)
03/01 00:08:05 Info:
Local Time: 03/01/2021 00:08:05 +00:00
Device Serial Number: 3EB3278A-33E3-4AD7-A2ED-35E692D67569
Roon Version: 1.8 (build 764) stable
OS Version: Linux 5.4.0-65-generic
Mono Version: 6.10.0.104 (tarball Wed Feb 24 12:17:16 UTC 2021)
Application Domain: RoonAppliance.exe
Assembly Codebase: file:///opt/RoonServer/Appliance/RoonAppliance.exe
Assembly Full Name: RoonAppliance, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
Exception Source: Roon.Broker.Core
Exception Type: System.NullReferenceException
Exception Target Site: State..ctor
Exception Message: Object reference not set to an instance of an object
Exception Data: none
--[ Stack Trace ]------------
Sooloos.Broker.State..ctor(BrokerConfig config, StorageManager storage)
Roon.Broker.Core.dll, IL 265, N 1387
Sooloos.Broker.Modules.Core.Create(BrokerConfig config, String platform)
Roon.Broker.Core.dll, IL 130, N 415
Sooloos.Application.Main(String[] argv)
RoonAppliance.exe, IL 1284, N 5763
03/01 00:08:05 Error:
Local Time: 03/01/2021 00:08:05 +00:00
Device Serial Number: 3EB3278A-33E3-4AD7-A2ED-35E692D67569
Roon Version: 1.8 (build 764) stable
OS Version: Linux 5.4.0-65-generic
Mono Version: 6.10.0.104 (tarball Wed Feb 24 12:17:16 UTC 2021)
Application Domain: RoonAppliance.exe
Assembly Codebase: file:///opt/RoonServer/Appliance/RoonAppliance.exe
Assembly Full Name: RoonAppliance, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
Exception Source: Roon.Broker.Core
Exception Type: System.NullReferenceException
Exception Target Site: State..ctor
Exception Message: Object reference not set to an instance of an object
Exception Data: none
--[ Stack Trace ]------------
Sooloos.Broker.State..ctor(BrokerConfig config, StorageManager storage)
Roon.Broker.Core.dll, IL 265, N 1387
Sooloos.Broker.Modules.Core.Create(BrokerConfig config, String platform)
Roon.Broker.Core.dll, IL 130, N 415
Sooloos.Application.Main(String[] argv)
RoonAppliance.exe, IL 1284, N 5763
In my current configuration I use Ceph (Ceph Ceph storage - Ceph) to back the persistent storage used by Roon. When I switch it to use the container filesystem (backed by the solid state drive on the node running Roon, but it is lost every time the container restarts, so not a long term solution) these issues don’t seem to happen. However, from inside the container the ceph filesystem looks like a regular filesystem mount at /var/roon/. Here is the output of df:
Filesystem 1K-blocks Used Available Use% Mounted on
none 114075828 73546648 34691356 68% /
tmpfs 65536 0 65536 0% /dev
tmpfs 4004872 0 4004872 0% /sys/fs/cgroup
10.43.178.177:6789,10.43.159.155:6789,10.43.123.155:6789:/volumes/csi/csi-vol-f3c40104-4ee1-11eb-99a8-c63f737644e4/2468b5da-29e3-4d87-a1e8-0df28ffe29de 367001600 148250624 218750976 41% /music
10.43.178.177:6789,10.43.159.155:6789,10.43.123.155:6789:/volumes/csi/csi-vol-6f9e3aa8-6277-11eb-85c1-3e8815950603/4c881565-6877-48d7-b680-f3e584e08a15 20971520 13381632 7589888 64% /backups
/dev/mapper/ubuntu--vg-ubuntu--lv 114075828 73546648 34691356 68% /etc/hosts
/dev/rbd0 20511312 2507424 17987504 13% /var/roon
shm 65536 8 65528 1% /dev/shm
tmpfs 4004872 12 4004860 1% /run/secrets/kubernetes.io/serviceaccount
tmpfs 4004872 0 4004872 0% /proc/acpi
tmpfs 4004872 0 4004872 0% /proc/scsi
tmpfs 4004872 0 4004872 0% /sys/firmware
However, the one difference ceph almost certiantly has is that disk reads/writes are a little more latent than the SSD on the node, but this seems like something which shouldn’t cause too many issues, disks are all latent to some degree, I can’t imagine there’s some timeout this is pushing up against.
Thanks in advance,
Jack