Is Roon Server running?
· Yes, Roon Server is turned on and running.
What do you see on your screen?
· "Waiting for your Roon Server"
What happens if you press the "Select a different Roon Server" button?
· I don't see Roon Server.
Please try to restart your network setup by unplugging, waiting 30 seconds and then replugging in your networking gear.
· No, the issue remains the same
Please select how you've connected your Roon Server to the internet
· Roon Server is connected by *Ethernet*
Have you checked your firewall settings to ensure that Roon is allowed through?
· Roon still won't connect even after checking this aspect
Have you verified that Roon Server is on the same subnet as your Remotes?
· My Remotes and Server are on the same subnet and I still can't connect
Sometimes the issues can be resolved with a reinstall of your Roon Remote app. Let's try to perform a reinstall and see if it helps.
· I've reinstalled the Roon Remote but it did not help
What is the operating system of your Roon Server host machine?
· *Linux Server* (Ubuntu, Fedora, ArcLinux...)
Select any of the following components that are present in your local network setup
· *VPN* installed on RoonServer or Roon Remotes
You mentioned a VPN. Have you tried disabling it?
· I tried disabling my VPN/proxy connection and Roon still won't connect
# What I see
·
My RoonServer hangs roughly every two days. It doesn't crash. The process
stays alive, so systemd never restarts it, audio and remote control just
stop, and I have to `systemctl restart roonserver` to get it back. There's
nothing in the log that looks like an error at the moment it dies, the
`[stats]` heartbeat line just stops appearing.
This is a NUC15CR running Debian 13 (trixie), kernel 6.12, RoonServer v2.67
(build 1661), stock tarball under systemd. The box also runs Plex and serves
ARC over Tailscale.
# What I found
·
I got tired of the random outages and started sampling the open file
descriptors of the RoonAppliance process every 15 minutes, bucketed by type.
One bucket grows and nothing else does:
```
03:36 total=427 socket=89
15:25 total=2089 socket=1751
```
That's +1662 sockets in about 12 hours, ~140/hr, climbing in a straight
line. FILE handles, pipes, eventfds, all flat. So it's sockets, and only
sockets.
Looking at them in /proc/net/udp, almost all of the growth is ephemeral-port
UDP sockets with a zero remote address, i.e. unconnected one-shot `sendto`
sockets that never got closed. RAAT and the other long-lived sockets are
steady.
Then I lined the socket count up against the log, sampling both every 10s.
The socket count is flat, then it steps up exactly when "Kicking off
discovery cycle" fires, and only then:
```
time leaked-udp discovery-cycles
08:53 80 21
08:55 80 21 (flat)
08:56 84 23 (cycle fired -> +sockets)
08:57 88 25
08:58 92 27
09:01 100 31
09:02 104 32
```
Over that window the discovery-cycle count went up by 12 and the leaked
sockets by 24, so it's about 2 sockets leaked per cycle. It tracks the cycle
itself, not the M-SEARCH send (the M-SEARCH count barely moved over the same
window). The cycle looks healthy otherwise, my gateway answers the M-SEARCH
with a 200 OK every time, so discovery is succeeding. It just leaks a socket
each time it runs.
# Why it turns into a hang
·
Roon's check.sh pins the descriptor limit at 8192:
```
check.sh:68 display_check "Testing ulimit -n 8192"
check.sh:69 ulimit -n 8192
```
`ulimit -n` sets both the soft and hard limit, so the process runs at
8192/8192 even though systemd set it to 524288. Once the leaked sockets fill
that up, the runtime can't get the descriptors it needs to spin up a new
threadpool thread, and .NET reports that as a misleading OutOfMemoryException:
```
Exception Type: System.OutOfMemoryException
Exception Target Site: Thread.StartInternal
System.Threading.Thread.StartInternal(...)
System.Threading.PortableThreadPool/WorkerThread.CreateWorkerThread()
```
It's not actually out of memory. The box has 16GB of RAM and normally runs
with ~11GB free; when it threw the OutOfMemoryException, RoonServer's own
[stats] line showed it using only ~2.7GB physical. It's out of file
descriptors, not memory. (This is the same .NET
misreport described in dotnet/runtime#71761:
https://github.com/dotnet/runtime/issues/71761 ) The last `[stats]` line before
it hung showed 8171 handles, pegged there for several samples, and the
heartbeat stopped a couple of minutes later. After that the process is alive
but doing nothing, which is why nothing restarts it.
# Things I ruled out
·
It's not Tailscale and it's not my NAS.
For Tailscale I measured the discovery-cycle rate over two matched 20-minute
windows, once with tailscaled running and once stopped: 66/hr running vs
72/hr stopped. No drop, and the sockets kept leaking the whole time it was
off. For the NAS, it's been powered off this entire time, the CIFS automount
is failed and there's no SMB traffic at all, and it still leaks at full rate.
So neither of those is involved.
What I haven't nailed down is what keeps raising "network reachability
changed" so often on my network in the first place. The gaps between events
are irregular, anywhere from a couple of seconds to a few minutes. My network
is all UniFi with UPnP on the gateway and three VLANs, and my best guess is
IPv6 router-advertisement churn (short address lifetimes plus a privacy
address regenerating), since that's what would actually raise a .NET
NetworkChange. I haven't proven it. I can correlate `ip monitor address
route` against the log lines if that's useful to you, but it's beside the
point for the leak itself.
# Why this is probably rare in the field
·
The leak is constant, ~2 sockets per discovery cycle. What varies host to
host is how often that cycle fires. On a quiet network it fires rarely enough
that you'd never reach 8192 before a routine restart. Mine fires about once a
minute, so I hit the wall in roughly two days.
If anyone wants to confirm it on their own box: sample your RoonAppliance
unconnected-UDP socket count and your "Kicking off discovery cycle" count
together over time. You'll see the same staircase regardless of how fast your
cycles fire. Happy to share the script I used.
# The fix
·
The descriptor probably just needs to be closed after the discovery probe
finishes, or a single per-interface socket reused across cycles instead of a
fresh one each time. You know the code.
Separately, it would help a lot if RoonAppliance exited non-zero when thread
creation fails instead of hanging, so a supervisor can actually restart it.
Right now the failure is completely silent.
This is the same leak Steven Moore reported on build 1392 back in 2022,
where staff acknowledged the recurring reachability changes but it was never
fixed. It's still here on 1661.
https://community.roonlabs.com/t/too-many-open-files-on-ubuntu-with-build-1392/270546
I have all of it if you want it (sanitized, the logs carry an auth token):
the fd-by-type series, the 10s socket-vs-log correlation, a pcap of the
M-SEARCH probes and the gateway replies, and a RoonServer log around a hang
with the frozen heartbeat and the OutOfMemoryException stack.
Describe the issue
# RoonServer hangs every couple of days on Linux: it leaks a UDP socket
# per network-reachability cycle and dies at Roon's own 8192 fd limit
Posting this because I think I've tracked down a real leak in RoonAppliance,
and it looks like the same one from the 2022 [Too many open files on
Ubuntu](https://community.roonlabs.com/t/too-many-open-files-on-ubuntu-with-build-1392/270546)
thread that never got fixed.
Describe your network setup
Everything on my side is Ubiquiti UniFi. My router/gateway is a UniFi Cloud Gateway Fiber (UCG-Fiber) at 192.168.0.1. Its WAN connects to the AT&T-provided fiber ONT, which is in passthrough/bridge mode, so the UCG-Fiber holds the public IP and there's no double-NAT. Wired clients run through a UniFi USW-Flex-2.5G-8-PoE switch, and Wi-Fi is served by UniFi U7 Pro Wall access points. The network has three VLANs, each with its own SSID: one for IoT, one for the main/wired network, and a third that's defined but currently unused. UPnP/IGD is enabled on the gateway. The RoonServer machine (an Intel NUC15CR running Debian 13) sits on the main/wired VLAN, wired through the USW-Flex switch. I also reach Roon ARC remotely via Tailscale on that host.