Terminate called after throwing an instance of 'OpenHome::NetworkError'

[Torpi]

Hello,

Since you discouraged me from using Roon Core in a docker swarm cluster, I set up Ubuntu 18 LTS on a separate computer and installed Roon with the latest version (followed steps from your site).

Got this NIC: Broadcom Limited NetXtreme II BCM5709 Gigabit Ethernet (rev 20)

Description Of Issue

The service start but crashes right after (see the log below).
I initially had a service on port 9001 and understood it is one used by roon (still waiting for the list…) so killed this service but this does not help.

Again, the reason I wanter Roon Core in a container, aside from the HA, is because I do not really like service with root privileges on my servers. Anyway, the log did not help me to understand what more to do to make it run.

Something related to the network, OpenHome and Upnp (I found this strange since you explain that it is crap here: https://kb.roonlabs.com/Why_Core%3F and here: Roon / RAAT Architecture and protocol VS DLNA / UPnP AV)

Yes, I checked with using your script if the machine is “compatible”:

=================================================================                                                                                                                                                                                        
Got a SIGABRT while executing native code. This usually indicates                                                                                                                                                                                        
a fatal error in the mono runtime or one of the native libraries                                                                                                                                                                                         
used by your application.                                                                                                                                                                                                                                
=================================================================                                                                                                                                                                                        
                                                                                                                                                                                                                                                         
Error                                                                                                                                                                                                                                                    
Initializing                                                                                                                                                                                                                                             
Started                                                                                                                                                                                                                                                  
aac_fixed decoder found, checking libavcodec version...                                                                                                                                                                                                  
has mp3float: 1, aac_fixed: 0                                                                                                                                                                                                                            
Not responding                                                                                                                                                                                                                                           
Running                                                                                                                                                                                                                                                  
terminate called after throwing an instance of 'OpenHome::NetworkError'                                                                                                                                                                                  
Stacktrace:                                                                                                                                                                                                                                              
                                                                                                                                                                                                                                                         
  at <unknown> <0xffffffff>                                                                                                                                                                                                                              
  at (wrapper managed-to-native) OpenHome.Net.ControlPoint.CpDeviceListUpnpServiceType.CpDeviceListCreateUpnpServiceType (intptr,intptr,uint,OpenHome.Net.ControlPoint.CpDeviceList/CallbackDevice,intptr,OpenHome.Net.ControlPoint.CpDeviceList/Callback
Device,intptr) [0x00030] in <21295a8acb1a4b59920209b723ef0864>:0                                                                                                                                                                                         
  at OpenHome.Net.ControlPoint.CpDeviceListUpnpServiceType..ctor (string,string,uint,OpenHome.Net.ControlPoint.CpDeviceList/ChangeHandler,OpenHome.Net.ControlPoint.CpDeviceList/ChangeHandler) [0x00042] in <21295a8acb1a4b59920209b723ef0864>:0        
  at OpenHome.Net.ControlPoint.CpUpnpDeviceListFactory.CreateListServiceType (string,string,uint,OpenHome.Net.ControlPoint.CpDeviceList/ChangeHandler,OpenHome.Net.ControlPoint.CpDeviceList/ChangeHandler) [0x00007] in <21295a8acb1a4b59920209b723ef086
4>:0                                                                                                                                                                                                                                                     
  at Sooloos.Broker.Transport.SongcastDirect.ReInit () [0x00235] in <c3f83e578441427188bf8c55d2a55639>:0                                                                                                                                                 
  at Sooloos.Broker.Transport.SongcastDirect..ctor (Sooloos.Broker.State,Sooloos.Broker.Transport.Module) [0x0007e] in <c3f83e578441427188bf8c55d2a55639>:0                                                                                              
  at Sooloos.Broker.Transport.Module.Enable () [0x001e9] in <c3f83e578441427188bf8c55d2a55639>:0                                                                                                                                                         
  at Sooloos.Broker.Transport.Module.UpdateEnabled () [0x00035] in <c3f83e578441427188bf8c55d2a55639>:0                                                                                                                                                  
  at Sooloos.Broker.Transport.Module.Init (Sooloos.Broker.Database) [0x000bf] in <c3f83e578441427188bf8c55d2a55639>:0                                                                                                                                    
  at Sooloos.Broker.State._InitTransportThreadAsync () [0x0000c] in <c3f83e578441427188bf8c55d2a55639>:0                                                                                                                                                 
  at Sooloos.SynchronizationContextThread/<>c__DisplayClass45_0.<Post>b__0 (object) [0x00006] in <25d73f18cc0f460fa0e05f69a9433340>:0                                                                                                                    
  at Sooloos.SynchronizationContextThread._Dispatch (Sooloos.SynchronizationContextThread/SendOrPostWrapper&) [0x0001e] in <25d73f18cc0f460fa0e05f69a9433340>:0                                                                                          
  at Sooloos.SynchronizationContextThread._Go () [0x00025] in <25d73f18cc0f460fa0e05f69a9433340>:0                                                                                                                                                       
  at System.Threading.ThreadHelper.ThreadStart_Context (object) [0x0001f] in <370a0c27f4b74d1a81431037df6d75bf>:0                                                                                                                                        
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00073] in <370a0c27f4b74d1a81431037df6d75bf>:0                                                                    
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00004] in <370a0c27f4b74d1a81431037df6d75bf>:0                                                                            
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object) [0x0002f] in <370a0c27f4b74d1a81431037df6d75bf>:0                                                                                 
  at System.Threading.ThreadHelper.ThreadStart () [0x00014] in <370a0c27f4b74d1a81431037df6d75bf>:0                                                                                                                                                      
  at (wrapper runtime-invoke) object.runtime_invoke_void__this__ (object,intptr,intptr,intptr) [0x0004f] in <370a0c27f4b74d1a81431037df6d75bf>:0                                                                                                         
/proc/self/maps:                                                                                                                                                                                                                                         
00400000-0096e000 r-xp 00000000 08:02 3546672                            /app/RoonServer/RoonMono/bin/mono-sgen                                                                                                                                          
00b6d000-00b71000 rw-p 0056d000 08:02 3546672                            /app/RoonServer/RoonMono/bin/mono-sgen                                                                                                                                          
00b71000-00bc8000 rw-p 00000000 00:00 0                                                                                                                                                                                                                  
01c56000-024cc000 rw-p 00000000 00:00 0                                  [heap]
414a4000-4184d000 rwxp 00000000 00:00 0                                                                                                                                                                                                                  
41dd8000-41de8000 rwxp 00000000 00:00 0                                                                                                                                                                                                                  
                                                                                                                                                                                          
7ffafe794000-7ffafe9bb000 r--s 00000000 00:42 12936735456681816830       /data/RoonServer/Cache/httpcache_2.db/000924.ldb                                                                                                                                
7ffafe9bb000-7ffafebd7000 r-xp 00000000 08:02 3285069                    /app/RoonServer/Appliance/libohNet.so                                                                                                                                           
7ffafebd7000-7ffafedd6000 ---p 0021c000 08:02 3285069                    /app/RoonServer/Appliance/libohNet.so                                                                                                                                           
7ffafedd6000-7ffafede9000 rw-p 0021b000 08:02 3285069                    /app/RoonServer/Appliance/libohNet.so                                                                                                                                           
7ffafede9000-7ffafee78000 rw-p 00000000 00:00 0                                                                                                                                                                                                          
7ffafee7c000-7ffafeefc000 rw-p 00000000 00:00 0                                                                                                                                                                                                          
7ffafeefe000-7ffaff100000 r--s 00000000 00:42 10030070389189320421       /data/RoonServer/Database/Core/aa363c64d5754a999e6c0ab266009d81/broker_2.db/004734.ldb                                                                                          
7                                                                                                                                                                                                 
7ffaffdde000-7ffafffe3000 r--s 00000000 00:42 11845224959325877581       /data/RoonServer/Database/Core/aa363c64d5754a999e6c0ab266009d81/broker_2.db/004829.ldb
7ffafffe3000-7ffb001e6000 r--s 00000000 00:42 12785873974644523743       /data/RoonServer/Database/Core/aa363c64d5754a999e6c0ab266009d81/broker_2.db/004742.ldb                                                                                          
7ffb001e6000-7ffb003e9000 r--s 00000000 00:42 12344230683165483927       /data/RoonServer/Database/Core/aa363c64d5754a999e6c0ab266009d81/broker_2.db/004741.ldb                                                                                          
7ffb003e9000-7ffb005ec000 r--s 00000000 00:42 12842909125785145639       /data/RoonServer/Database/Core/aa363c64d5754a999e6c0ab266009d81/broker_2.db/004796.ldb                                                                                          
7ffb005ec000-7ffb007f1000 r--s 00000000 00:42 10637204629538905478       /data/RoonServer/Database/Core/aa363c64d5754a999e6c0ab266009d81/broker_2.db/004740.ldb                                                                                          
7ffb007f1000-7ffb009f3000 r--s 00000000 00:42 12629453448345076428       /data/RoonServer/Database/Core/aa363c64d5754a999e6c0ab266009d81/broker_2.db/004743.ldb                                                                                          
7ffb009f3000-7ffb00bf6000 r--s 00000000 00:42 10806192060531958031       /data/RoonServer/Database/Core/aa363c64d5754a999e6c0ab266009d81/broker_2.db/004745.ldb                                                                                          
7ffb00bf6000-7ffb00dfc000 r--s 00000000 00:42 13657839064149823964       /data/RoonServer/Database/Core/aa363c64d5754a999e6c0ab266009d81/broker_2.db/004746.ldb                                                                                          
7ffb00dfc000-7ffb01000000 r--s 00000000 00:42 9664546309201092508        /data/RoonServer/Database/Core/aa363c64d5754a999e6c0ab266009d81/broker_2.db/004828.ldb                                                                                          
                                                                                                                                                                                     
[...] number of characters limited in the post                                                                                                                                                                                                          

Native stacktrace:

    /app/RoonServer/RoonMono/bin/RoonAppliance() [0x5064e6]                                                                                                                                                                                          
    /app/RoonServer/RoonMono/bin/RoonAppliance() [0x5ca974]                                                                                                                                                                                          
    /lib/x86_64-linux-gnu/libpthread.so.0(+0x110e0) [0x7ffb85e500e0]                                                                                                                                                                                 
    /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcf) [0x7ffb858bbfff]                                                                                                                                                                                   
    /lib/x86_64-linux-gnu/libc.so.6(abort+0x16a) [0x7ffb858bd42a]                                                                                                                                                                                    
    /app/RoonServer/Appliance/libleveldb.so(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0xbf) [0x7ffb535a93ee]                                                                                                                                      
    /app/RoonServer/Appliance/libleveldb.so(_ZN10__cxxabiv111__terminateEPFvvE+0x3) [0x7ffb535683a2]                                                                                                                                                 
    /app/RoonServer/Appliance/libleveldb.so(_ZN10__cxxabiv112__unexpectedEPFvvE+0) [0x7ffb535683d8]                                                                                                                                                  
    /app/RoonServer/Appliance/libleveldb.so(__cxa_rethrow+0) [0x7ffb53567c42]                                                                                                                                                                        
    /app/RoonServer/Appliance/libohNet.so(+0x12fefe) [0x7ffafeaeaefe]                                                                                                                                                                                
    /app/RoonServer/Appliance/libohNet.so(+0xb195f) [0x7ffafea6c95f]                                                                                                                                                                                 
    /app/RoonServer/Appliance/libohNet.so(+0xb3075) [0x7ffafea6e075]                                                                                                                                                                                 
    /app/RoonServer/Appliance/libohNet.so(+0x13cf3b) [0x7ffafeaf7f3b]                                                                                                                                                                                
    /app/RoonServer/Appliance/libohNet.so(+0xbfdca) [0x7ffafea7adca]                                                                                                                                                                                 
    /app/RoonServer/Appliance/libohNet.so(+0xc0664) [0x7ffafea7b664]                                                                                                                                                                                 
    /app/RoonServer/Appliance/libohNet.so(CpDeviceListCreateUpnpServiceType+0xd5) [0x7ffafea71945]  
	[0x41848b44]

[mjw]

It looks like you’re running another Mono application, e.g. Emby. I don’t think two (different) versions of Mono play well together.

There are some threads in the forum on this topic. I’ll check this evening when I return home.

[noris]

Hi @Torpi,

Do you by any chance have Emby running on the same Core? I also think this issue could be due to having multiple versions of Mono, see: Roon Server on Linux core dump on startup

[Torpi]

Thank you @mjw, well, I will double check that but I don’t think so. I ran this on a separate server. Yes I have plex and emby but they are part of my swarm cluster, not on this separate server. But my takeaway from your post is that OpenHome is not something supposed to be part or Roon Core, right?

@noris, I don’t but I will follow the steps described in the post and have another try. Thanks for the support.

[mjw]

I was thinking of Emby not OpenHome and the thread @noris posted. Try shutting down Emby and then see if Roon starts without error. Not ideal, but it’d be useful to see if that is causing the issue.

[Torpi]

Emby is definitely not on the server, it’s in a docker on another server in a swarm. Shutting it down does not change anything.

But I think I found something interesting.

Roon Core is starting correctly, actualy the crash happens when I click “Connect” to enter my credentials. Before that, the logs are not reporting any problems.

On this bare-metal installation of Roon Core, I had dockerd (the deamon) running and it is also binding port 1900.

Could this be related? It’s not clear to me what make the app crashes when I connect.

T.

[mjw]

Roon doesn’t use port 1900, it uses 9100:9101/tcp and 9003/udp.

[Torpi]

When I listed ports on temporary environment, I had RoonAppli listening on 9100, 9101, 9150, 42165 and RAATServe on 9004, 38917. You are right, 1900 was not there.

So, on the same subnet, Emby stopped, same problem. Now on the same subnet, I have also Plex running. Not on the same machine, on another one. Stopping Plex fix the crash when I connect to Roon Core.

Plex is in a docker swarm (separates machines) and is configured on a bridge (not host) network with those mappings:

ports:
  - 32400:32400/tcp
  - 3005:3005/tcp
  - 8324:8324/tcp
  - 32469:32469/tcp
  - 1900:1900/udp
  - 32410:32410/upd
  - 32412:32410/upd
  - 32413:32410/upd
  - 32414:32410/upd

Any idea what in the discovery might make it crash on Connect?

[mjw]

Take a look at the following threads. I don’t use Docker so not much help here.

Linux Core - Crashes on Remote Login (Network Error)

Apologies for the delay, been busy and got sidetracked. I just wanted to close out this thread. I believe the issue does stem from OpenHome, even when using a docker container I got the error when running with net=host. I tried a vanilla Ubuntu install I had on another machine and everything worked fine, so I was fairly convinced there was something to this OpenHome issue. I decided net=host must be the culprit as I couldn’t otherwise see how OpenHome would be creeping into a container. I e…

RoonServer in docker can't find Meridian zones

I have worked out my issue. @Steef_de_Bruijn, you may be interested in this as well. I’ll write the issue up on your dockerfile repository. My docker host has multiple networks attached, and the container was attached to all host networks. For some reason, despite the container being able to reach the Meridian 218, roon was unable to discover its presence. My solution to this was to create a docker macvlan network on the host, connected to my main LAN, and attach the docker container only to …

[Torpi]

Martin, to me this is not related to docker but to having Plex running on the same subnet.
What works now:

Scenario 1 - working

• dedicated machine with Roon Core on bare metal Ubuntu
• multiple nodes with docker swarm running many containers. Emby running in one of those containers. Plex disabled (not running)

Scenario 1.1 - not working

• dedicated machine with Roon Core on bare metal Ubuntu
• multiple nodes with docker swarm running many containers. Emby running in one of those containers. Plex Running on one of the nodes.

-> crashes when Connecting to Roon-Core. Nota: Dedicated machine and swarm nodes are in the same subnet.

Scenario 2 - working

• multiple nodes with docker swarm running many containers. Emby running in one of those containers. Plex disabled (not running). Roon-Core running on one of the host (not in docker container)

-> working on ANY host (even the one with Emby)

Scenario 1.1 - not working

• multiple nodes with docker swarm running many containers. Emby running in one of those containers. Plex running in one of those containers. Roon-Core running on one of the host (not in docker container)

-> crashes when Connecting to Roon-Core. I tried to run Emby, Plex and Roon on 3 different host nodes that does not change anything.

Scenario 3 - working

• multiple nodes with docker swarm running many containers. Emby running in one of those containers. Plex disabled (not running). Roon-Core running in one of those containers.

-> working on ANY host (even the one with Emby)

Scenario 3.1 - not working

• multiple nodes with docker swarm running many containers. Emby running in one of those containers. Plex running in one of those containers. Roon-Core running in one of those containers.

-> crashes when Connecting to Roon-Core. I tried to run Emby, Plex and Roon on 3 different nodes that does not change anything.

My conclusion is definitely that when Plex is running on the same subnet, Roon-Core crashes on Connection.

I am using Plex to be able to stream when I am on the road or abroad traveling. I would remove it if Roon provided the functionality.

T.

[mjw]

@Torpi

Martin, to me this is not related to docker but to having Plex running on the same subnet.

Agree. Those threads look into this and suggest a solution, i.e. macvlan on the Plex container. I use Roon and Plex side-by-side on a physical machine.

@Torpi

I am using Plex to be able to stream when I am on the road or abroad traveling. I would remove it if Roon provided the functionality.

This is on the roadmap.

Why are we excited about Chromecast (or Sonos)?

There’s a lot in this thread…but a few points: I’m not going to rehash the discussion about UPnP. Our viewpoint has been made clear enough times by now. However, there is some ultimate consistency in our treatment of UPnP and API-only streaming services: we can’t make good product out of either, so we won’t. Spotify, Apple Music, and Google Play are non-starters, compromise or not. They do not make available an API/commercial arrangement that we are allowed to use. It’s not really interesting …

[Torpi]

@mjw,

Yes, I’ve read the macvlan approach but this is a constraint I would like to avoid on the swarm cluster because it needs to be configured everywhere…

I followed my first instinct and remove the port 1900/udp in the list of exposed ports of Plex. I then stopped Roon-Core, started Plex, wait for everything to be up, stream audio, then started Roon-Core: WORKS!

The only thing I am missing is that my local Plex server is not able to discover my other remote Plex server (I have a remote instance for DVR and LiveTV) but I am able to workaround this with an Nginx reverse proxy so it’s not that bad.

My conclusion is that when Roon Core started its discovery on port 1900, it gets an unattended response from Plex Server and crashes. I think it’s something that you could catch on your side.

T

[mjw]

Perhaps @support can pass on this information to the dev team. BTW, I’m a community member like you.

[Torpi]

@mjw

Thank you very much for kindly helping me!

Finally, removed the dedicated Roon-Core hardware and run everything on docker Swarm, with Roon in a container. I set a preferred node for Roon so I don’t have to unauthorized the license unless this node is down. In that case it’s a one-click quick fix since data is stored on GlusterFS and available from any node.

If anyone need information or config files, just ping me since Roon team seems pretty sensitive about sharing that on github/dockerhub.

T.

[noris]

Hi @Torpi,

Thanks for the update here and glad to hear that Roon is now working since making those changes!
I’ll be sure to forward your findings over to QA.

[Torpi]

No problem, you can contact me directly if you need more information or if you need to test anything else. Just a comment about port 1900, the DLNA server (using that port, at least in Plex documentation) is disabled on my Plex server. So I would bet it is related to the discovery. Having Plex in another subnet (not routed) seems to do the trick as well. So I would say it is related to the broadcast.

[nuwriy]

Hello @Torpi,

I wanted to reach out and let you know that we released Roon 1.7 Build 571, which includes some improvements that we believe should help here. Please give the update a try and let us know if it helps!

You can read the full release notes here:

Roon 1.7 (Build 571) is Live!

Hi everyone! The latest update of Roon is now rolling out, with bug fixes, performance improvements, and our new Recent Activity feature! Recent Activity Roon’s overview screen has always functioned as a kind of “home” screen, providing you with easy access to your recently added music and new releases recommended for you. If you liked an album enough to add it to your collection, Roon has always made sure it’s close to hand. But we realize that not everyone listens in the same way, and with …

Thanks,
Roon Support

[Torpi]

Thanks! @nuwriy, I gave a try and it seems to solve the problem. The only glitch I got with the upgrade was some endpoints were disabled (I had to enable and name all Ropieee).