I’m assuming a reboot and running
sudo systemctl restart roonserver.service
Did not work?
Correct, that did not work.
it is, so if you ls -l /opt/RoonServer/.update does the file exist?
find /opt/RoonServer | grep -i update
I think the install is broken.
You may wish to rip it out and try again.
Also last 3 lines
Apr 02 21:40:57 nuc systemd[1]: roonserver.service: A process of this unit has been killed by the OOM killer.
Apr 02 21:40:58 nuc systemd[1]: roonserver.service: Failed with result 'oom-kill'.
Apr 02 21:40:58 nuc systemd[1]: roonserver.service: Consumed 17h 41min 2.796s CPU time.
Suggesting critically low memory
Yeah, that doesn’t exist. And it can’t find any update folder too.
Well, yeah, that’s what I did the first time. I can easily do it again, sure, but the chance is high that the problem will just be back after a few days.
/opt/RoonServer/Server/../.update is not a properly normalised path to a Linux hidden filename. It should work fine but isn’t the optimal expression of the path.
1 Like
I let the easyinstaller do its thing when it installed roon. I didn’t do any custom paths or anything.
It may happen again. @AA_Parky points to an OOM kill.
The Link helps figuring out why/what it may be.
I abuse my setup, I have not experienced your issue. My sample size of 1 being just that though.
How to Find Which Process Was Killed by Linux OOM Killer | Baeldung on Linux
Same as @MamaTried , I’m pretty rough on my homebuilt frankenmonster of a NAS and have not incurred a repeat of what you are experiencing after an upgrade.
I don’t run Plex on the same install but I do run Emby on a separate vm.
Could it possibly come down to running Plex, decoding, and roon dsp consuming all memory?
Yeah, so the only thing I can think of that could’ve caused the OOM issue is because of Plex transcoding folder pointed to RAM. However, the folder that is used is /dev/shm which only uses 50% of RAM max. Could Roon be using a lot of RAM even if the NUC has 32GB?
Also, I’ve had Plex pointed to that folder for quite a while now and Roon acted like this just recently.
Checked historical data for RAM usage and what do you know:
Blue is memory usage (left is actual usage and right is swap usage)
Not sure how this occurred though.
@Kevin_Mychal_Ong not in my experience. Roon behaves really well for me memory and network wise. However, I don’t use any DSP unless on my headphones and just a convolution filter but even then not noticeable.
I think those times match your journal right?
Yeah. There are like a handful of clients playing through Roon and they’re mostly pi’s and one Windows machine. Everything is streaming off of the network.
It looks like it. Is there a way to check which service consumed all?
Using the guide @MamaTried linked above, I was able to get the memory footprint before the OOM killing occurred:
Apr 02 13:40:57 nuc kernel: Tasks state (memory values in pages):
Apr 02 13:40:57 nuc kernel: [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
Apr 02 13:40:57 nuc kernel: [ 354] 0 354 14143 64 118784 231 -250 systemd-journal
Apr 02 13:40:57 nuc kernel: [ 372] 0 372 5565 15 65536 430 -1000 systemd-udevd
Apr 02 13:40:57 nuc kernel: [ 373] 0 373 1292 0 45056 42 0 blkmapd
Apr 02 13:40:57 nuc kernel: [ 568] 0 568 749 4 45056 49 0 rpc.idmapd
Apr 02 13:40:57 nuc kernel: [ 604] 107 604 1960 0 53248 122 0 rpcbind
Apr 02 13:40:57 nuc kernel: [ 621] 0 621 24956 2 73728 337 0 dhclient
Apr 02 13:40:57 nuc kernel: [ 647] 104 647 2219 54 57344 273 -900 dbus-daemon
Apr 02 13:40:57 nuc kernel: [ 648] 0 648 63627 86 126976 567 0 NetworkManager
Apr 02 13:40:57 nuc kernel: [ 651] 0 651 58362 0 81920 233 0 polkitd
Apr 02 13:40:57 nuc kernel: [ 652] 0 652 55185 0 81920 866 0 rsyslogd
Apr 02 13:40:57 nuc kernel: [ 653] 0 653 2869 47 61440 307 0 smartd
Apr 02 13:40:57 nuc kernel: [ 654] 0 654 44454 38 94208 198 0 systemd-logind
Apr 02 13:40:57 nuc kernel: [ 655] 0 655 3353 34 61440 164 0 systemd-machine
Apr 02 13:40:57 nuc kernel: [ 656] 0 656 171012 293 163840 996 0 udisksd
Apr 02 13:40:57 nuc kernel: [ 657] 0 657 3640 14 69632 151 0 wpa_supplicant
Apr 02 13:40:57 nuc kernel: [ 668] 0 668 78695 0 118784 433 0 ModemManager
Apr 02 13:40:57 nuc kernel: [ 690] 0 690 1268 4 45056 129 0 rpc.mountd
Apr 02 13:40:57 nuc kernel: [ 700] 0 700 502619 2702 376832 5450 -999 containerd
Apr 02 13:40:57 nuc kernel: [ 745] 0 745 3323 0 61440 234 -1000 sshd
Apr 02 13:40:57 nuc kernel: [ 782] 0 782 567491 5626 520192 6416 -500 dockerd
Apr 02 13:40:57 nuc kernel: [ 786] 0 786 17101 78 118784 527 0 nmbd
Apr 02 13:40:57 nuc kernel: [ 921] 996 921 2866 104 65536 158 0 pmcd
Apr 02 13:40:57 nuc kernel: [ 926] 0 926 3633 0 65536 189 0 pmdaroot
Apr 02 13:40:57 nuc kernel: [ 940] 0 940 2896 145 65536 155 0 pmdaproc
Apr 02 13:40:57 nuc kernel: [ 943] 0 943 2577 28 57344 87 0 pmdaxfs
Apr 02 13:40:57 nuc kernel: [ 944] 0 944 2812 169 57344 99 0 pmdalinux
Apr 02 13:40:57 nuc kernel: [ 946] 0 946 2569 0 57344 115 0 pmdakvm
Apr 02 13:40:57 nuc kernel: [ 1052] 996 1052 3744 20 73728 179 0 pmproxy
Apr 02 13:40:57 nuc kernel: [ 1571] 996 1571 2699 57 57344 117 0 pmie
Apr 02 13:40:57 nuc kernel: [ 1727] 996 1727 3544 429 61440 603 0 pmlogger
Apr 02 13:40:57 nuc kernel: [ 1743] 0 1743 1671 35 49152 30 0 cron
Apr 02 13:40:57 nuc kernel: [ 1747] 0 1747 387039 0 299008 2993 0 libvirtd
Apr 02 13:40:57 nuc kernel: [ 1749] 106 1749 60619 75 233472 1231 0 minidlnad
Apr 02 13:40:57 nuc kernel: [ 1754] 0 1754 1446 0 49152 30 0 agetty
Apr 02 13:40:57 nuc kernel: [ 1888] 0 1888 268794 141 131072 3615 -500 docker-proxy
Apr 02 13:40:57 nuc kernel: [ 1894] 0 1894 269082 24 135168 2711 -500 docker-proxy
Apr 02 13:40:57 nuc kernel: [ 1915] 0 1915 177694 498 106496 2112 -998 containerd-shim
Apr 02 13:40:57 nuc kernel: [ 1958] 0 1958 177758 513 110592 2375 -998 containerd-shim
Apr 02 13:40:57 nuc kernel: [ 1991] 0 1991 186708 2507 188416 1761 0 portainer
Apr 02 13:40:57 nuc kernel: [ 2002] 0 2002 400 1 40960 15 0 sh
Apr 02 13:40:57 nuc kernel: [ 2033] 0 2033 177758 478 98304 1837 -998 containerd-shim
Apr 02 13:40:57 nuc kernel: [ 2056] 0 2056 913 36 49152 116 0 smartd
Apr 02 13:40:57 nuc kernel: [ 2319] 0 2319 406 15 40960 7 0 crond
Apr 02 13:40:57 nuc kernel: [ 2792] 0 2792 20647 184 143360 583 0 smbd
Apr 02 13:40:57 nuc kernel: [ 2794] 0 2794 20107 150 135168 582 0 smbd-notifyd
Apr 02 13:40:57 nuc kernel: [ 2795] 0 2795 20109 155 131072 577 0 cleanupd
Apr 02 13:40:57 nuc kernel: [ 2796] 0 2796 20643 152 139264 615 0 lpqd
Apr 02 13:40:57 nuc kernel: [ 2838] 101 2838 22041 39 73728 138 0 systemd-timesyn
Apr 02 13:40:57 nuc kernel: [ 5003] 0 5003 90880 36 159744 870 0 packagekitd
Apr 02 13:40:57 nuc kernel: [ 6993] 0 6993 20749 274 151552 525 0 smbd
Apr 02 13:40:57 nuc kernel: [ 7862] 0 7862 1689 1 57344 68 0 start.sh
Apr 02 13:40:57 nuc kernel: [ 7866] 0 7866 5666867 2639 692224 7952 0 RoonServer
Apr 02 13:40:57 nuc kernel: [ 7896] 0 7896 14162306 7966525 69505024 131108 0 RoonAppliance
Apr 02 13:40:57 nuc kernel: [ 7900] 0 7900 867 0 49152 20 0 processreaper
Apr 02 13:40:57 nuc kernel: [ 7954] 0 7954 5872607 235 917504 26742 0 RAATServer
Apr 02 13:40:57 nuc kernel: [ 47077] 0 47077 268794 0 147456 3728 -500 docker-proxy
Apr 02 13:40:57 nuc kernel: [ 47091] 0 47091 287163 25 143360 3215 -500 docker-proxy
Apr 02 13:40:57 nuc kernel: [ 47106] 0 47106 177822 484 106496 2261 -998 containerd-shim
Apr 02 13:40:57 nuc kernel: [ 47126] 0 47126 51 0 32768 4 0 s6-svscan
Apr 02 13:40:57 nuc kernel: [ 47221] 0 47221 51 0 32768 5 0 s6-supervise
Apr 02 13:40:57 nuc kernel: [ 47538] 0 47538 51 0 32768 3 0 s6-supervise
Apr 02 13:40:57 nuc kernel: [ 47541] 1002 47541 26301 580 266240 17015 0 python3
Apr 02 13:40:57 nuc kernel: [ 48710] 997 48710 33141 3465 299008 7434 0 Plex Media Serv
Apr 02 13:40:57 nuc kernel: [ 48829] 997 48829 15723 67 147456 8102 0 Plex Script Hos
Apr 02 13:40:57 nuc kernel: [ 48880] 997 48880 10313 41 98304 363 0 Plex Tuner Serv
Apr 02 13:40:57 nuc kernel: [ 49002] 997 49002 641 0 45056 29 0 EasyAudioEncode
Apr 02 13:40:57 nuc kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=docker-f41386b5f0583b9f2f00e38b3fb1bf8b9874165a6906ccecfeda7310f2d684fe.scope,mems_allowed=0,global_>
Apr 02 13:40:57 nuc kernel: Out of memory: Killed process 7896 (RoonAppliance) total-vm:56649224kB, anon-rss:31866100kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:67876kB oom_score_adj:0
Plex was using a mere 33MB of RAM when this happened. RoonServer and RAATServer, on the other hand, were using an enormous amount of RAM when this happened. So I’d say we can isolate Plex as the cause of the problem here. The roon process was killed because it was itself the one consuming an unusual amount of resources because it flapping on and off trying to run but ultimately failing because it is not responding.
What other steps can I take in troubleshooting this?
Does anyone have any other ideas here? It happened again just now.
Apr 05 05:33:58 nuc kernel: [ 2795] 0 2795 20109 105 131072 631 0 cleanupd
Apr 05 05:33:58 nuc kernel: [ 2796] 0 2796 20643 79 139264 688 0 lpqd
Apr 05 05:33:58 nuc kernel: [ 2838] 101 2838 22041 37 73728 140 0 systemd-timesyn
Apr 05 05:33:58 nuc kernel: [ 5003] 0 5003 90913 27 176128 3056 0 packagekitd
Apr 05 05:33:58 nuc kernel: [ 47077] 0 47077 268794 0 147456 3727 -500 docker-proxy
Apr 05 05:33:58 nuc kernel: [ 47091] 0 47091 287163 26 143360 3214 -500 docker-proxy
Apr 05 05:33:58 nuc kernel: [ 47106] 0 47106 177822 502 106496 2254 -998 containerd-shim
Apr 05 05:33:58 nuc kernel: [ 47126] 0 47126 51 0 32768 3 0 s6-svscan
Apr 05 05:33:58 nuc kernel: [ 47221] 0 47221 51 0 32768 4 0 s6-supervise
Apr 05 05:33:58 nuc kernel: [ 47538] 0 47538 51 0 32768 2 0 s6-supervise
Apr 05 05:33:58 nuc kernel: [ 47541] 1002 47541 28392 4834 282624 14660 0 python3
Apr 05 05:33:58 nuc kernel: [ 151101] 998 151101 142010 43 163840 1075 -900 libvirt-dbus
Apr 05 05:33:58 nuc kernel: [ 167644] 0 167644 1689 2 49152 67 0 start.sh
Apr 05 05:33:58 nuc kernel: [ 167649] 0 167649 5666871 1702 675840 8301 0 RoonServer
Apr 05 05:33:58 nuc kernel: [ 168038] 0 168038 14129615 7941819 68734976 129705 0 RoonAppliance
Apr 05 05:33:58 nuc kernel: [ 168039] 0 168039 867 0 45056 19 0 processreaper
Apr 05 05:33:58 nuc kernel: [ 168096] 0 168096 5872607 729 913408 25729 0 RAATServer
Apr 05 05:33:58 nuc kernel: [ 249104] 997 249104 39874 3363 339968 5183 0 Plex Media Serv
Apr 05 05:33:58 nuc kernel: [ 249152] 997 249152 15723 705 151552 7481 0 Plex Script Hos
Apr 05 05:33:58 nuc kernel: [ 249202] 997 249202 10315 44 98304 361 0 Plex Tuner Serv
Apr 05 05:33:58 nuc kernel: [ 265494] 997 265494 649 2 45056 36 0 EasyAudioEncode
Apr 05 05:33:58 nuc kernel: [ 267027] 0 267027 20754 366 151552 455 0 smbd
Apr 05 05:33:58 nuc kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=minidlna.service,mems_allowed=0,global_oom,task_memcg=/system.slice/roonserver.service,task=RoonAppliance,pid=168038,uid=0
Apr 05 05:33:58 nuc kernel: Out of memory: Killed process 168038 (RoonAppliance) total-vm:56518460kB, anon-rss:31767276kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:67124kB oom_score_adj:0
Apr 05 05:33:59 nuc kernel: oom_reaper: reaped process 168038 (RoonAppliance), now anon-rss:28kB, file-rss:0kB, shmem-rss:0kB
To be honest, no.
I would be inclined wipe my install and start over with Debian’s minimal net install manually apt the dependencies and then reinstall Roon. Monitor its stability for a while before adding anything else to the install and go from there.
1 Like
If you’re running Plex on the same machined then you might have a port conflict on 1900
1 Like
I understand that reinstalling the OS would be a thing to try but being in IT that is usually my ultimate workaround. I’m still not ready to go that route because I feel I haven’t done enough troubleshooting yet but then again I’m not an expert with Roon.
1 Like