· None of the above quite fits
· None of these quite match
· Roon 2.7.1 runs out of file descriptors and stops working
· Gigabit ethernet wired
Can I downgrade to 2.70?
After 2.71 “upgrade” everything works fine for a few minutes or hours then stops playing from library or streaming radio. It says to check my internet connection. I think the real problem is it’s running out of file descriptors because they’re being allocated but not freed/garbage collected.
At 2.71 startup this message repeats many times per second continuously:
08/05 14:37:43 Critical: scx: System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'LevelDb'.
at Roon.Analytics.EventManager.SetLocalState(String name, PropertyValue value)
at Roon.Broker.Transport.AnalyticsHelper.SetLastState(IAudioDevice device, String state)
at Roon.Broker.Transport.AnalyticsHelper.NotifyDiscovered(IAudioDevice device)
at Roon.Broker.Transport.AirPlay.<>c__DisplayClass22_0.<ev_device_seen>b__0()
at Sooloos.SynchronizationContextThread._Dispatch(SendOrPostWrapper& ret)
08/05 14:37:43 Critical: scx: System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'LevelDb'.
at Roon.Analytics.EventManager.SetLocalState(String name, PropertyValue value)
at Roon.Broker.Transport.AnalyticsHelper.SetLastState(IAudioDevice device, String state)
at Roon.Broker.Transport.AnalyticsHelper.NotifyDiscovered(IAudioDevice device)
at Roon.Broker.Transport.AirPlay.<>c__DisplayClass22_0.<ev_device_seen>b__0()
at Sooloos.SynchronizationContextThread._Dispatch(SendOrPostWrapper& ret)
Nevertheless Roon works fine, then after some time it no longer plays music. The log shows this message repeating continuously:
08/06 06:12:14 Critical: scx: System.Net.Sockets.SocketException (10055): An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full.
at System.Net.Sockets.Socket.UpdateStatusAfterSocketErrorAndThrowException(SocketError error, Boolean disconnectOnFailure, String callerName)
at System.Net.Sockets.Socket.DoBind(EndPoint endPointSnapshot, SocketAddress socketAddress)
at Roon.Http.HttpServer._Start()
at Roon.Audio.AirPlay.DACPServer.Start()
at Roon.Broker.Transport.AirPlayDevice..ctor(AirPlay airplay, MdnsDiscovery discovery, DACPManager dacp_manager, AirPlayDeviceData data)
at Roon.Broker.Transport.AirPlay.<>c__DisplayClass22_0.<ev_device_seen>b__0()
at Sooloos.SynchronizationContextThread._Dispatch(SendOrPostWrapper& ret)
08/06 06:12:15 Critical: scx: System.Net.Sockets.SocketException (10055): An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full.
at System.Net.Sockets.Socket.UpdateStatusAfterSocketErrorAndThrowException(SocketError error, Boolean disconnectOnFailure, String callerName)
at System.Net.Sockets.Socket.DoBind(EndPoint endPointSnapshot, SocketAddress socketAddress)
at Roon.Http.HttpServer._Start()
at Roon.Audio.AirPlay.DACPServer.Start()
at Roon.Broker.Transport.AirPlayDevice..ctor(AirPlay airplay, MdnsDiscovery discovery, DACPManager dacp_manager, AirPlayDeviceData data)
at Roon.Broker.Transport.AirPlay.<>c__DisplayClass22_0.<ev_device_seen>b__0()
at Sooloos.SynchronizationContextThread._Dispatch(SendOrPostWrapper& ret)
Rebooting the Windows 10 box makes it start working again for a short time.
Static code analysis in your QA process should have caught this. Allocating but not freeing resources similar to a memory leak is a pretty basic but in software dev, are you doing static analysis of the code? Maybe your tool missed this.
Please fix asap or let me roll back to 2.70. The 2.70 version worked fine for weeks at a time without anything needing to be restarted.
FWIW: Things are fine on my Windows 11 server machine. Windows 10 is end of life, so specific testing on it won’t necessarily happen.
I don’t think this is a Windows 10 issue because Roon 2.70 worked fine for weeks at a time with no reboots.
Roon does not provide the option to downgrade is not due to potential database issues. I just posted this in another thread..
Hi @James_Ockers,
Thanks for the details. Since Roon 2.7.0 was stable for you for weeks at a time, this does not look like a Windows 10 issue. The next step we would like you to try is the latest Roon release, the hotfix, and then let us know whether you see the same behavior there.
Please report back once you have tested it. If the issue is still present on the latest hotfix, we can take it from there.
Hi @noris ,
Thanks for that. I installed 2.71 build 1683
08/06 19:35:11 Debug: [appupdater] Update installed: 2.71 (build 1683) production
and I bet it’s not better, I still see this repeating in the RoonServer_log.txt logfile:
08/06 19:39:12 Critical: scx: System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'LevelDb'.
at LevelDb.Database.CheckDisposedClosed()
at LevelDb.Database.Put(Byte[] key, ByteBuffer val)
at LevelDb.Database.Put(Byte[] key, Byte[] val)
at Roon.Analytics.EventManager.SetLocalState(String name, PropertyValue value)
at Roon.Broker.Transport.AnalyticsHelper.SetLastState(IAudioDevice device, String state)
at Roon.Broker.Transport.AnalyticsHelper.NotifyDiscovered(IAudioDevice device)
at Roon.Broker.Transport.AirPlay.<>c__DisplayClass22_0.<ev_device_seen>b__0()
at Sooloos.CallingThread.<>c__DisplayClass12_0.<Post>b__1(Object <p0>)
at Sooloos.SynchronizationContextThread._Dispatch(SendOrPostWrapper& ret)
08/06 19:39:13 Debug: [music/storage] queueing 2 mods for 9969f8d7-2dfc-462a-a6b2-217725b94e30 (\)
08/06 19:39:13 Trace: [devicedb] [autodetect] No Match for DeviceAutodetectData[Type=AirPlay Model=ShairportSync DescriptiveName=office shairplay]
08/06 19:39:13 Critical: scx: System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'LevelDb'.
at LevelDb.Database.CheckDisposedClosed()
at LevelDb.Database.Put(Byte[] key, ByteBuffer val)
at LevelDb.Database.Put(Byte[] key, Byte[] val)
at Roon.Analytics.EventManager.SetLocalState(String name, PropertyValue value)
at Roon.Broker.Transport.AnalyticsHelper.SetLastState(IAudioDevice device, String state)
at Roon.Broker.Transport.AnalyticsHelper.NotifyDiscovered(IAudioDevice device)
at Roon.Broker.Transport.AirPlay.<>c__DisplayClass22_0.<ev_device_seen>b__0()
at Sooloos.CallingThread.<>c__DisplayClass12_0.<Post>b__1(Object <p0>)
at Sooloos.SynchronizationContextThread._Dispatch(SendOrPostWrapper& ret)
This is the problematic message that I speculate leads to the exhaustion of file descriptors as I explained in my initial trouble report.
But I will let you know if it is stable now.
Thanks,
James
Hi @noris
I didn’t reboot Windows after the hotfix was installed so as expected it quit working a few hours later, with the “Critical: scx: System.Net.Sockets.SocketException (10055)” error repeated continuously in the logfile.
My usual workaround for these 2.71 “Critical: scx:” issues is to restart Windows (reboot the MediaPC) which restarts RoonServer and everything else. I did that this morning:
08/07 09:32:09 Info: Starting RoonServer v2.71 (build 1683) production on windows
I’m pleased to report I don’t see any further logfile lines containing the string “scx” after the restart.
I will let this topic auto-close if I don’t see any further file descriptor exhaustion related issues.
Thanks!
James
Hi @noris
May someone please tell me what this message means? It repeats in the RoonServer_log.txt about 6 times per second:
08/09 20:58:57 Critical: scx: System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'LevelDb'.
at Roon.Analytics.EventManager.SetLocalState(String name, PropertyValue value)
at Roon.Broker.Transport.AnalyticsHelper.SetLastState(IAudioDevice device, String state)
at Roon.Broker.Transport.AnalyticsHelper.NotifyDiscovered(IAudioDevice device)
at Roon.Broker.Transport.AirPlay.<>c__DisplayClass22_0.<ev_device_seen>b__0()
at Sooloos.SynchronizationContextThread._Dispatch(SendOrPostWrapper& ret)
08/09 20:58:57 Critical: scx: System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'LevelDb'.
at Roon.Analytics.EventManager.SetLocalState(String name, PropertyValue value)
at Roon.Broker.Transport.AnalyticsHelper.SetLastState(IAudioDevice device, String state)
at Roon.Broker.Transport.AnalyticsHelper.NotifyDiscovered(IAudioDevice device)
at Roon.Broker.Transport.AirPlay.<>c__DisplayClass22_0.<ev_device_seen>b__0()
at Sooloos.SynchronizationContextThread._Dispatch(SendOrPostWrapper& ret)
08/09 20:58:57 Critical: scx: System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'LevelDb'.
at Roon.Analytics.EventManager.SetLocalState(String name, PropertyValue value)
at Roon.Broker.Transport.AnalyticsHelper.SetLastState(IAudioDevice device, String state)
at Roon.Broker.Transport.AnalyticsHelper.NotifyDiscovered(IAudioDevice device)
at Roon.Broker.Transport.AirPlay.<>c__DisplayClass22_0.<ev_device_seen>b__0()
at Sooloos.SynchronizationContextThread._Dispatch(SendOrPostWrapper& ret)
08/09 20:58:57 Critical: scx: System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'LevelDb'.
at Roon.Analytics.EventManager.SetLocalState(String name, PropertyValue value)
at Roon.Broker.Transport.AnalyticsHelper.SetLastState(IAudioDevice device, String state)
at Roon.Broker.Transport.AnalyticsHelper.NotifyDiscovered(IAudioDevice device)
at Roon.Broker.Transport.AirPlay.<>c__DisplayClass22_0.<ev_device_seen>b__0()
at Sooloos.SynchronizationContextThread._Dispatch(SendOrPostWrapper& ret)
08/09 20:58:57 Critical: scx: System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'LevelDb'.
at Roon.Analytics.EventManager.SetLocalState(String name, PropertyValue value)
at Roon.Broker.Transport.AnalyticsHelper.SetLastState(IAudioDevice device, String state)
at Roon.Broker.Transport.AnalyticsHelper.NotifyDiscovered(IAudioDevice device)
at Roon.Broker.Transport.AirPlay.<>c__DisplayClass22_0.<ev_device_seen>b__0()
at Sooloos.SynchronizationContextThread._Dispatch(SendOrPostWrapper& ret)
08/09 20:58:57 Critical: scx: System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'LevelDb'.
at Roon.Analytics.EventManager.SetLocalState(String name, PropertyValue value)
at Roon.Broker.Transport.AnalyticsHelper.SetLastState(IAudioDevice device, String state)
at Roon.Broker.Transport.AnalyticsHelper.NotifyDiscovered(IAudioDevice device)
at Roon.Broker.Transport.AirPlay.<>c__DisplayClass22_0.<ev_device_seen>b__0()
at Sooloos.SynchronizationContextThread._Dispatch(SendOrPostWrapper& ret)
I think this is the issue that causes RoonAppliance/RoonServer to run out of file descriptors and stop working. There are several AirPlay devices on the network including one raspberry pi running “shairplay.” Any way I can get Roon to ignore AirPlay devices? Here are airplay log lines:
# grep -i airplay ~ockers/roon/ockers/AppData/Local/RoonServer/Logs/RoonServer_log.txt
08/09 21:20:52 Trace: [devicedb] [autodetect] No Match for DeviceAutodetectData[Type=AirPlay Model=Mac15,11 DescriptiveName=MacBook Pro]
08/09 21:20:55 Trace: [devicedb] [autodetect] No Match for DeviceAutodetectData[Type=AirPlay Model=ShairportSync DescriptiveName=office shairplay]
08/09 21:20:59 Trace: [devicedb] [autodetect] No Match for DeviceAutodetectData[Type=AirPlay Model=MacBookPro16,2 DescriptiveName=MacBook Pro]
08/09 21:25:48 Trace: [airplay] expired device AirPlayDevice[DeviceId=C2F40E7D94AA@office shairplay._raop._tcp.local, Name=office.local, Model=ShairportSync, IPEndPoint=192.168.0.21:5000] because we got a TTL=0 mDNS packet
08/09 21:25:48 Trace: [airplay] disconnected
The shairplay device disconnected when I rebooted it to see if Roon would notice and what it would log.
I restarted Windows today for an update which restarted RoonServer/RoonAppliance, it’s the latest version
08/09 20:52:25 Info: Starting RoonServer v2.71 (build 1683) production on windows
but unlike the last time I restarted in which there were no log messages referencing “scx”, this time I see the above messages repeating several times per second.
Based on this I expect Roon to stop working in a few minutes or hours, and I’ll have to restart windows to get it working again. After my previous message a couple days ago Roon worked fine and there were no scx messages in the log for the last 2 days.
How can I get rid of these scx LevelDb log messages and whatever’s causing them?
Thanks
James
Hello @James_Ockers
Your reading of this is close, and thank you for the log excerpts. Two corrections and one piece of good news.
First the good news: your server has been running cleanly for the last 11 hours with none of those messages at all. So it is behaving right now.
On the LevelDb message itself: it means Roon is writing to an internal store that has already been closed. It is not a sign that your library or database is damaged and there is nothing for you to repair. That store holds internal state, not your music, playlists or edits. Please do not restore from a backup or start a new library on account of it. Neither would help.
What your logs show is more specific than a steady leak. The flood only happens in some sessions, not all of them. Across your logs there are exactly two episodes, one lasting 25 minutes and one lasting 8 minutes, and between them your server ran for 29 hours and then 13 hours with none of these messages at all. Both episodes began immediately after a cluster of quick restarts where Roon Server only stayed up for ten or fifteen seconds each time.
That points at a race condition during startup: on some starts the internal store is closed while the part of Roon that watches for AirPlay devices is still running, and it then writes into it repeatedly. On other starts the ordering works out and the session is clean. Which is exactly what you noticed, one restart curing it and the next not. That is not something you are doing differently, it is our initialization order.
We are writing this up for our developers with your logs attached. Your original diagnosis about resources being allocated and not released was the right instinct, and the startup detail sharpens it considerably.
If it starts up dirty again, the quickest thing you can do is simply restart Roon Server rather than the whole machine, and check whether the messages stop. If they do, that confirms the picture and saves you a reboot each time.
Hi @vadim ,
Thank you for this. I agree it’s intermittent and also that it seems like a race condition in the multithreaded process. These are notoriously difficult to diagnose and debug (even more so in some languages than others), so thank you for looking into it. I’m glad my trouble report and logs were helpful.
Yes I’ve tried both rebooting and stop/restart RoonServer and it does seem kind of random if it starts up with the scx errors or not. For posterity, this is how I stop the Roon process on Windows, IIRC I found this command long ago on a helpful support thread somewhere else on the Roon community forums:
C:\Users\ockers\Desktop>type stoproon.bat
@echo off
taskkill /im RoonServer.exe /f
Roon 2.71 is indeed stable provided we don’t encounter this race condition. I suppose I’m not entirely convinced it happens only at startup: yesterday I had what I thought was a perfectly stable instance, it had been running for 1-2 days then suddenly stopped in the middle of playing a track yesterday afternoon. The Android app did the circle thing and then changed to the uh-oh something’s not right screen. I checked the logs and sure enough, scx errors. RoonServer/RoonAppliance had started up apparently normally but then this happened much later, I think.
I’ve disabled the shairport-sync (shairplay) process on my super old raspberry pi that was running it, in case it’s causing some Bonjour/mDNS bug or timing issue. It’s some ancient unsupported raspbian 5+ years old. The hardware and RAATServer continue to work fine.
Thanks
James
Thanks for the reply @James_Ockers, our development team has a ticket in for a fix for the above, and it’ll be included in one of our upcoming Roon updates.
Thanks again for your time, and for your patience while we work on shipping a fix!
Hi @James_Ockers,
This ticket is proceeding through the pipeline. While we don’t have a scheduled timeline for release at this time, we do expect it will eventually ship with a public Early Access release for testing.
https://community.roonlabs.com/c/early-access/software-release-notes
If you’d like to join the Early Access branch for testing, you can find instructions here: