I’ve been experiencing persistent crashes on my Roon core machine for about the last couple of weeks after months of running smoothly. Sometimes it’s up for a couple days, sometimes a few hours, sometimes it immediately crashes on boot. If I try to ssh into the machine, the host is not reachable. So it’s not just roon application crashing but rather the OS.
I was tailing /var/log/syslog during a crash and this is the last thing:
Jun 1 17:54:24 roon-nuc systemd[1]: Starting Time & Date Service…
Jun 1 17:54:24 roon-nuc dbus-daemon[901]: [system] Successfully activated service ‘org.freedesktop.timedate1’
Jun 1 17:54:24 roon-nuc systemd[1]: Started Time & Date Service.
Jun 1 17:54:54 roon-nuc systemd[1]: systemd-timedated.service: Succeeded.
Jun 1 18:02:07 roon-nuc kernel: [ 778.516401] show_signal_msg: 21 callbacks suppressed
Jun 1 18:02:07 roon-nuc kernel: [ 778.516411] .NET Server GC[1080]: segfault at 8 ip 00007f6310c7963c sp 00007f630c8f8470 error 6 in libcoreclr.so[7f6310b37000+254000]
Jun 1 18:02:07 roon-nuc kernel: [ 778.516438] Code: 6a 7a 24 00 c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 40 84 ff 74 18 66 48 8d 3d 6f 4a 21 00 66 66 48 e8 37 fb ba ff 48 8b 00 83 48 08 40 48 8d 05 80 8b 23 00 48 8b 38 48 85 ff 74 09 48 8b
Jun 1 18:02:13 roon-nuc start.sh[969]: Not responding
Jun 1 18:02:41 roon-nuc kernel: [ 812.964336] traps: apport[1833] genepal protection fault ip:4ef180 sp:7ffc1a49a120 error:0 in python3.8[423000+295000]
Jun 1 18:02:41 roon-nuc kernel: [ 812.964352] Process 1833(apport) has RLIMIT_CORE set to 1
Jun 1 18:02:41 roon-nuc kernel: [ 812.964353] Aborting core
tail: error writing ‘standard output’: Illegal seek
tail: write error: Illegal seek
I rebooted the machine and found this in /var/crash:
ls -lah /var/crash
total 261M
drwxrwxrwt 2 root root 4.0K Jun 1 18:02 .
drwxr-xr-x 14 root root 4.0K Feb 11 16:11 …
-rw-r----- 1 root root 261M Jun 1 18:02 _opt_RoonServer_RoonDotnet_dotnet.0.crash
So, I guess the system crash is caused by roon crash?
In /var/roon/RoonServer/Logs the log for the time this crash happened is 0 bytes, which is weird.
The NUC is solely used for Roon so nothing else of note running. I install any updates quickly. I use Samba share for adding new music to the library from a MacBook Pro. The library is on an SSD internal to the NUC (separate drive only used for roon library).
I see that you have customized your Ubuntu installation, running a low-latency kernel. Have you tried running Roon on the stock kernel? I have been running Roon on Ubuntu 20.04 and recently on 22.04, and since it was ported from Mono to .NET it has been running very stable.
A lot of us are having Roon crashes on the latest builds anyway due to a memory leak.
Are you sure it is not related to that.
My Rock server is restarting Roon about every 12 hours since I got back from holiday and did the update
In this case it seems as if Ubuntu OS was crashing, not just Roon server. Two years ago I have seen Roon crashing on Linux with a problem that was easy to work around, but I have never had Ubuntu Linux crashing on me…
This is interesting. After changing to generic kernel and rebooting, the OS would not boot as errors were detected in root file system requiring manual fsck. I ran that and it reported lots of deleted/unused inodes (including a log file in /var/roon/RoonServer/Database/Core//broker_2.db) and lots of unattached 0 length inodes. After fsck repaired those, I rebooted. Roon is running and music library seems intact. Roon seemed to be re-analyzing as RoonAppliance process is >500% CPU.
While on the machine running top, after a few minutes it suddenly crashes and outputs dumps including error:
watchdog: BUG: soft lockup - CPU#5 stock for 23s! [Worker (13):3848]
Fan is going strong and it seems to keep looping output crash logs.
That doesn’t look good… Seems likely a hardware problem. If you get Roon to run again, throttle the analysis… better it takes longer than causing overheating…
We have recently had a bug fix release for a few memory-related issues in Roon, are you still seeing this behavior on the latest Roon release? Also, how is your Core cooling configured, does the unit have enough airflow?
So far so good. After installing latest update, uptime on my Ubuntu core is almost 4 days.
For cooling, I’m using the Intel NUC’s default settings. It’s balanced between cool and quiet. I don’t recall the exact settings. The machine is on an open shelf, about 6" from wall in a room that stays cool. So I think airflow is ok. I’s far away from any listening room so I could change to favor cool settings since more fan noise wouldn’t be an issue.
