Ubuntu 16.04.3 / RoonServer/Unhandled Exception error

Steven_Neighorn · March 19, 2019, 7:42pm

Ubuntu 16.04.3
Music stored on NAS
253K tracks
RoonServer V1.6 (build 401) stable

Starting about a week or so ago, crashes with the following output:

Mar 18 21:40:41 qiclab start.sh[20832]: Unhandled exception NetworkError at Os/OsWrapper.cpp:91 in thread SsdpListenerM
Mar 18 21:40:41 qiclab start.sh[20832]: /opt/RoonServer/Appliance/libohNet.so(+0x14ae50) [0x7ff712990e50]
Mar 18 21:40:41 qiclab start.sh[20832]: /opt/RoonServer/Appliance/libohNet.so(_ZN8OpenHome9ExceptionC2EPKcS2_j+0x25) [0x7ff712934b85]
Mar 18 21:40:41 qiclab start.sh[20832]: /opt/RoonServer/Appliance/libohNet.so(+0x14a01a) [0x7ff71299001a]
Mar 18 21:40:41 qiclab start.sh[20832]: /opt/RoonServer/Appliance/libohNet.so(+0x12d630) [0x7ff712973630]
Mar 18 21:40:41 qiclab start.sh[20832]: /opt/RoonServer/Appliance/libohNet.so(+0xbe8cf) [0x7ff7129048cf]
Mar 18 21:40:41 qiclab start.sh[20832]: /opt/RoonServer/Appliance/libohNet.so(+0xbf385) [0x7ff712905385]
Mar 18 21:40:41 qiclab start.sh[20832]: /opt/RoonServer/Appliance/libohNet.so(+0xb2229) [0x7ff7128f8229]
Mar 18 21:40:41 qiclab start.sh[20832]: /opt/RoonServer/Appliance/libohNet.so(+0xb29be) [0x7ff7128f89be]
Mar 18 21:40:41 qiclab start.sh[20832]: /opt/RoonServer/Appliance/libohNet.so(+0xb36e1) [0x7ff7128f96e1]
Mar 18 21:40:41 qiclab start.sh[20832]: /opt/RoonServer/Appliance/libohNet.so(+0x1414d9) [0x7ff7129874d9]
Mar 18 21:40:41 qiclab start.sh[20832]: /opt/RoonServer/Appliance/libohNet.so(+0x14a978) [0x7ff712990978]
Mar 18 21:40:41 qiclab start.sh[20832]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba) [0x7ff87c0356ba]
Mar 18 21:40:41 qiclab start.sh[20832]: /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ff87bb5541d]
Mar 18 21:40:41 qiclab start.sh[20832]: /proc/self/maps:
Mar 18 21:40:41 qiclab start.sh[20832]: 00400000-0096e000 r-xp 00000000 fc:00 84279489                           /opt/RoonServer/RoonMono/bin/mono-sgen
Mar 18 21:40:41 qiclab start.sh[20832]: 00b6d000-00b71000 rw-p 0056d000 fc:00 84279489                           /opt/RoonServer/RoonMono/bin/mono-sgen
Mar 18 21:40:41 qiclab start.sh[20832]: 00b71000-00bc8000 rw-p 00000000 00:00 0
Mar 18 21:40:41 qiclab start.sh[20832]: 01336000-01bae000 rw-p 00000000 00:00 0                                  [heap]
Mar 18 21:40:41 qiclab start.sh[20832]: 40ead000-41589000 rwxp 00000000 00:00 0
Mar 18 21:40:41 qiclab start.sh[20832]: 416f8000-41708000 rwxp 00000000 00:00 0
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff508000000-7ff508021000 rw-p 00000000 00:00 0
...
many similar lines
...
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff669fe6000-7ff66a1eb000 r--s 00000000 fc:00 2634658                    /var/roon/RoonServer/Database/Core/80213ce7a49a47a2be44e4e88688239b/broker_2.db/193211.ldb
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff66a1eb000-7ff66a3ed000 r--s 00000000 fc:00 2634657                    /var/roon/RoonServer/Database/Core/80213ce7a49a47a2be44e4e88688239b/broker_2.db/193210.ldb
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff66a3ed000-7ff66a5f3000 r--s 00000000 fc:00 2634645                    /var/roon/RoonServer/Database/Core/80213ce7a49a47a2be44e4e88688239b/broker_2.db/193209.ldb
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff66a5f3000-7ff66a7f9000 r--s 00000000 fc:00 2634644                    /var/roon/RoonServer/Database/Core/80213ce7a49a47a2be44e4e88688239b/broker_2.db/193208.ldb
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff66a7f9000-7ff66aa00000 r--s 00000000 fc:00 2634611                    /var/roon/RoonServer/Database/Core/80213ce7a49a47a2be44e4e88688239b/broker_2.db/193207.ldb
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff66abee000-7ff66adef000 r--s 00000000 fc:00 2634333                    /var/roon/RoonServer/Database/Core/80213ce7a49a47a2be44e4e88688239b/broker_2.db/193206.ldb
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff66adef000-7ff66aff3000 r--s 00000000 fc:00 2630419                    /var/roon/RoonServer/Database/Core/80213ce7a49a47a2be44e4e88688239b/broker_2.db/193205.ldb
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff66aff3000-7ff66b1f9000 r--s 00000000 fc:00 2629637                    /var/roon/RoonServer/Database/Core/80213ce7a49a47a2be44e4e88688239b/broker_2.db/193204.ldb

...
many similar lines
...
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87b000000-7ff87b004000 r--p 00000000 fc:00 84279616                   /opt/RoonServer/Appliance/Roon.Storage.CollectionDump.dll
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87b004000-7ff87b006000 rw-p 00000000 00:00 0
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87b006000-7ff87b010000 r--p 00000000 fc:00 84279596                   /opt/RoonServer/Appliance/LevelDb.Database.dll
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87b010000-7ff87b015000 r--p 00000000 fc:00 84279774                   /opt/RoonServer/Appliance/Roon.Broker.Concurrency.dll
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87b015000-7ff87b022000 r--p 00000000 fc:00 84279889                   /opt/RoonServer/Appliance/Roon.Broker.Remoting.dll
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87b022000-7ff87b033000 r--p 00000000 fc:00 84279775                   /opt/RoonServer/Appliance/Roon.Media.dll
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87b033000-7ff87b049000 r--p 00000000 fc:00 84279617                   /opt/RoonServer/Appliance/Messaging.dll
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87b049000-7ff87b07d000 r--p 00000000 fc:00 84279528                   /opt/RoonServer/RoonMono/lib/mono/gac/Mono.Posix/4.0.0.0__0738eb9f132ed756/Mono.Posix.dll
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87b07d000-7ff87b0c9000 r--p 00000000 fc:00 84279525                   /opt/RoonServer/RoonMono/lib/mono/gac/Mono.Security/4.0.0.0__0738eb9f132ed756/Mono.Security.dll
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87b0c9000-7ff87b0e7000 r--p 00000000 fc:00 84279569                   /opt/RoonServer/RoonMono/lib/mono/gac/System.Configuration/4.0.0.0__b03f5f7f11d50a3a/System.Configuration.dll
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87b0e7000-7ff87b0f3000 r--p 00000000 fc:00 84280022                   /opt/RoonServer/Appliance/Roon.Storage.Core.dll
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87b0f3000-7ff87b123000 r--p 00000000 fc:00 84279960                   /opt/RoonServer/Appliance/RoonApp.dll
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87b123000-7ff87b152000 r--p 00000000 fc:00 84279892                   /opt/RoonServer/Appliance/Metadata.Messages.dll
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87b152000-7ff87b163000 r--p 00000000 fc:00 84279714                   /opt/RoonServer/Appliance/Roon.Messages.dll
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87b163000-7ff87b1be000 r--p 00000000 fc:00 84279773                   /opt/RoonServer/Appliance/Broker.Messages.Core.dll
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87b1be000-7ff87b1bf000 ---p 00000000 00:00 0
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87b1bf000-7ff87b3bf000 rw-p 00000000 00:00 0
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87b3bf000-7ff87b776000 r--p 00000000 fc:00 84279580                   /opt/RoonServer/RoonMono/lib/mono/4.5/mscorlib.dll
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87ba4e000-7ff87bc0e000 r-xp 00000000 fc:00 4981894                    /lib/x86_64-linux-gnu/libc-2.23.so
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87bc0e000-7ff87be0e000 ---p 001c0000 fc:00 4981894                    /lib/x86_64-linux-gnu/libc-2.23.so
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87be0e000-7ff87be12000 r--p 001c0000 fc:00 4981894                    /lib/x86_64-linux-gnu/libc-2.23.so
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87be12000-7ff87be14000 rw-p 001c4000 fc:00 4981894                    /lib/x86_64-linux-gnu/libc-2.23.so
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87be14000-7ff87be18000 rw-p 00000000 00:00 0
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87be18000-7ff87be2e000 r-xp 00000000 fc:00 4981261                    /lib/x86_64-linux-gnu/libgcc_s.so.1
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87be2e000-7ff87c02d000 ---p 00016000 fc:00 4981261                    /lib/x86_64-linux-gnu/libgcc_s.so.1
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87c02d000-7ff87c02e000 rw-p 00015000 fc:00 4981261                    /lib/x86_64-linux-gnu/libgcc_s.so.1
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87c02e000-7ff87c046000 r-xp 00000000 fc:00 4981853                    /lib/x86_64-linux-gnu/libpthread-2.23.so
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87c046000-7ff87c245000 ---p 00018000 fc:00 4981853                    /lib/x86_64-linux-gnu/libpthread-2.23.so
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87c245000-7ff87c246000 r--p 00017000 fc:00 4981853                    /lib/x86_64-linux-gnu/libpthread-2.23.so
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87c246000-7ff87c247000 rw-p 00018000 fc:00 4981853                    /lib/x86_64-linux-gnu/libpthread-2.23.so
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87c247000-7ff87c24b000 rw-p 00000000 00:00 0
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87c24b000-7ff87c24e000 r-xp 00000000 fc:00 4981851                    /lib/x86_64-linux-gnu/libdl-2.23.so
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87c24e000-7ff87c44d000 ---p 00003000 fc:00 4981851                    /lib/x86_64-linux-gnu/libdl-2.23.so
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87c44d000-7ff87c44e000 r--p 00002000 fc:00 4981851                    /lib/x86_64-linux-gnu/libdl-2.23.so
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87c44e000-7ff87c44f000 rw-p 00003000 fc:00 4981851                    /lib/x86_64-linux-gnu/libdl-2.23.so
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87c44f000-7ff87c456000 r-xp 00000000 fc:00 4981572                    /lib/x86_64-linux-gnu/librt-2.23.so
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87c456000-7ff87c655000 ---p 00007000 fc:00 4981572                    /lib/x86_64-linux-gnu/librt-2.23.so
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87c655000-7ff87c656000 r--p 00006000 fc:00 4981572                    /lib/x86_64-linux-gnu/librt-2.23.so
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87c656000-7ff87c657000 rw-p 00007000 fc:00 4981572                    /lib/x86_64-linux-gnu/librt-2.23.so
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87c657000-7ff87c75f000 r-xp 00000000 fc:00 4981955                    /lib/x86_64-linux-gnu/libm-2.23.so
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87c75f000-7ff87c95e000 ---p 00108000 fc:00 4981955                    /lib/x86_64-linux-gnu/libm-2.23.so
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87c95e000-7ff87c95f000 r--p 00107000 fc:00 4981955                    /lib/x86_64-linux-gnu/libm-2.23.so
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87c95f000-7ff87c960000 rw-p 00108000 fc:00 4981955                    /lib/x86_64-linux-gnu/libm-2.23.so
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87c960000-7ff87c986000 r-xp 00000000 fc:00 4981852                    /lib/x86_64-linux-gnu/ld-2.23.so
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87c986000-7ff87c9a3000 r--p 00000000 fc:00 84279997                   /opt/RoonServer/Appliance/Base.dll
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87c9a3000-7ff87ca11000 r--p 00000000 fc:00 84279996                   /opt/RoonServer/Appliance/RoonBase.dll
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87ca11000-7ff87ca80000 r--p 00000000 fc:00 84280011                   /opt/RoonServer/Appliance/Roon.Broker.Api.dll
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87ca80000-7ff87cb06000 rw-p 00000000 00:00 0
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87cb06000-7ff87cb64000 ---p 00000000 00:00 0
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87cb64000-7ff87cb6b000 rw-p 00000000 00:00 0
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87cb6b000-7ff87cb74000 r--p 00000000 fc:00 84279903                   /opt/RoonServer/Appliance/Roon.Client.Models.dll
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87cb74000-7ff87cb75000 rw-p 00000000 00:00 0
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87cb75000-7ff87cb79000 r--p 00000000 fc:00 84279772                   /opt/RoonServer/Appliance/RoonAppliance.exe
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87cb79000-7ff87cb84000 rw-p 00000000 00:00 0
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87cb84000-7ff87cb85000 rw-s 00000000 00:14 6                          /dev/shm/mono.15812
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87cb85000-7ff87cb86000 r--p 00025000 fc:00 4981852                    /lib/x86_64-linux-gnu/ld-2.23.so
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87cb86000-7ff87cb87000 rw-p 00026000 fc:00 4981852                    /lib/x86_64-linux-gnu/ld-2.23.so
Mar 18 21:40:41 qiclab start.sh[20832]: 7ff87cb87000-7ff87cb88000 rw-p 00000000 00:00 0
Mar 18 21:40:41 qiclab start.sh[20832]: 7ffc5691f000-7ffc56940000 rw-p 00000000 00:00 0                          [stack]
Mar 18 21:40:41 qiclab start.sh[20832]: 7ffc569e6000-7ffc569e9000 r--p 00000000 00:00 0                          [vvar]
Mar 18 21:40:41 qiclab start.sh[20832]: 7ffc569e9000-7ffc569eb000 r-xp 00000000 00:00 0                          [vdso]
Mar 18 21:40:41 qiclab start.sh[20832]: ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
Mar 18 21:40:41 qiclab start.sh[20832]: Native stacktrace:
Mar 18 21:40:41 qiclab start.sh[20832]: #011/opt/RoonServer/RoonMono/bin/RoonAppliance() [0x5064e6]
Mar 18 21:40:41 qiclab start.sh[20832]: #011/opt/RoonServer/RoonMono/bin/RoonAppliance() [0x5ca974]
Mar 18 21:40:41 qiclab start.sh[20832]: #011/lib/x86_64-linux-gnu/libpthread.so.0(+0x11390) [0x7ff87c03f390]
Mar 18 21:40:41 qiclab start.sh[20832]: #011/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38) [0x7ff87ba83428]
Mar 18 21:40:41 qiclab start.sh[20832]: #011/lib/x86_64-linux-gnu/libc.so.6(abort+0x16a) [0x7ff87ba8502a]
Mar 18 21:40:41 qiclab start.sh[20832]: #011/opt/RoonServer/Appliance/libohNet.so(+0x14ae29) [0x7ff712990e29]
Mar 18 21:40:41 qiclab start.sh[20832]: #011/opt/RoonServer/Appliance/libohNet.so(+0xeefc3) [0x7ff712934fc3]
Mar 18 21:40:41 qiclab start.sh[20832]: #011/opt/RoonServer/Appliance/libohNet.so(+0x1416c7) [0x7ff7129876c7]
Mar 18 21:40:41 qiclab start.sh[20832]: #011/opt/RoonServer/Appliance/libohNet.so(+0x14a978) [0x7ff712990978]
Mar 18 21:40:41 qiclab start.sh[20832]: #011/lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba) [0x7ff87c0356ba]
Mar 18 21:40:41 qiclab start.sh[20832]: #011/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ff87bb5541d]
Mar 18 21:40:41 qiclab start.sh[20832]: =================================================================
Mar 18 21:40:41 qiclab start.sh[20832]: Got a SIGABRT while executing native code. This usually indicates
Mar 18 21:40:41 qiclab start.sh[20832]: a fatal error in the mono runtime or one of the native libraries
Mar 18 21:40:41 qiclab start.sh[20832]: used by your application.
Mar 18 21:40:41 qiclab start.sh[20832]: =================================================================
Mar 18 21:40:41 qiclab start.sh[20832]: Error
Mar 18 21:40:43 qiclab start.sh[20832]: Initializing
Mar 18 21:40:43 qiclab start.sh[20832]: Started
Mar 18 21:40:44 qiclab start.sh[20832]: Not responding
Mar 18 21:40:45 qiclab start.sh[20832]: aac_fixed decoder found, checking libavcodec version...
Mar 18 21:40:45 qiclab start.sh[20832]: has mp3float: 1, aac_fixed: 0
Mar 18 21:40:49 qiclab start.sh[20832]: Running

This is happening 6-8 times a day.

mjw · March 21, 2019, 6:37pm

Did this coincide with an update to either Ubuntu or Roon? I note that your Ubuntu install doesn’t have the latest binaries and fixes too (you’re probably not running HWE.)

Steven_Neighorn · March 21, 2019, 7:21pm

Hello Martin, thank you for the response.

I am not running HWE.

I installed the “1.6 build 401” on or shortly after Feb 20th. I had an alert on the main Roon screen and
followed instructions for update. I am sorry to say that I don’t know if the crashes started then or
sometime after. I have improved the logging notifications on the system and started seeing the
crash reports. While I do apply Ubuntu 16 patches as they come out via “apt” the server itself has not been rebooted in a few months.

Do you recommend HWE for U16.04 as a next step?

mjw · March 21, 2019, 7:32pm

First I’d try a reboot, then consider 16.04.5. I use HWE since it keeps my system up-to-date, particularly for kernel. This is particularly useful if newer applications need functionality in newer kernels.

I’m running 18.04 now, but you should be able to execute the following to enable HWE.

sudo apt install --install-recommends linux-generic-hwe-16.04

Steven_Neighorn · March 21, 2019, 11:42pm

Thanks again Martin. I will give this a try ASAP. I really appreciate your help.

Steven_Neighorn · March 25, 2019, 10:45pm

I did the HWE install, and the reboot, and sadly the problem has not gone away.

Here is the latest crash (top and bottom)

Mar 25 10:31:19 qiclab01 start.sh[41632]: Unhandled exception NetworkError at Os/OsWrapper.cpp:91 in thread SsdpListenerM
Mar 25 10:31:19 qiclab01 start.sh[41632]: /opt/RoonServer/Appliance/libohNet.so(+0x14ae50) [0x7fbd5945ae50]
Mar 25 10:31:19 qiclab01 start.sh[41632]: /opt/RoonServer/Appliance/libohNet.so(_ZN8OpenHome9ExceptionC2EPKcS2_j+0x25) [0x7fbd593feb85]
Mar 25 10:31:19 qiclab01 start.sh[41632]: /opt/RoonServer/Appliance/libohNet.so(+0x14a01a) [0x7fbd5945a01a]
Mar 25 10:31:19 qiclab01 start.sh[41632]: /opt/RoonServer/Appliance/libohNet.so(+0x12d630) [0x7fbd5943d630]
Mar 25 10:31:19 qiclab01 start.sh[41632]: /opt/RoonServer/Appliance/libohNet.so(+0xbe8cf) [0x7fbd593ce8cf]
Mar 25 10:31:19 qiclab01 start.sh[41632]: /opt/RoonServer/Appliance/libohNet.so(+0xbf385) [0x7fbd593cf385]
Mar 25 10:31:19 qiclab01 start.sh[41632]: /opt/RoonServer/Appliance/libohNet.so(+0xb29be) [0x7fbd593c29be]
Mar 25 10:31:19 qiclab01 start.sh[41632]: /opt/RoonServer/Appliance/libohNet.so(+0xb36e1) [0x7fbd593c36e1]
Mar 25 10:31:19 qiclab01 start.sh[41632]: /opt/RoonServer/Appliance/libohNet.so(+0x1414d9) [0x7fbd594514d9]
Mar 25 10:31:19 qiclab01 start.sh[41632]: /opt/RoonServer/Appliance/libohNet.so(+0x14a978) [0x7fbd5945a978]
Mar 25 10:31:19 qiclab01 start.sh[41632]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba) [0x7fbed829c6ba]
Mar 25 10:31:19 qiclab01 start.sh[41632]: /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fbed7dbc41d]
Mar 25 10:31:19 qiclab01 start.sh[41632]: /proc/self/maps:
Mar 25 10:31:19 qiclab01 start.sh[41632]: 00400000-0096e000 r-xp 00000000 fd:00 84279489 /opt/RoonServer/RoonMono/bin/mono-sgen
Mar 25 10:31:19 qiclab01 start.sh[41632]: 00b6d000-00b71000 rw-p 0056d000 fd:00 84279489 /opt/RoonServer/RoonMono/bin/mono-sgen
Mar 25 10:31:19 qiclab01 start.sh[41632]: 00b71000-00bc8000 rw-p 00000000 00:00 0
Mar 25 10:31:19 qiclab01 start.sh[41632]: 026a9000-02f22000 rw-p 00000000 00:00 0 [heap]
Mar 25 10:31:19 qiclab01 start.sh[41632]: 40722000-40732000 rwxp 00000000 00:00 0
Mar 25 10:31:19 qiclab01 start.sh[41632]: 41448000-41ba4000 rwxp 00000000 00:00 0
…
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed71dd000-7fbed79dd000 rw-p 00000000 00:00 0
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed7cb5000-7fbed7e75000 r-xp 00000000 fd:00 4981894 /lib/x86_64-linux-gnu/libc-2.23.so
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed7e75000-7fbed8075000 —p 001c0000 fd:00 4981894 /lib/x86_64-linux-gnu/libc-2.23.so
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed8075000-7fbed8079000 r–p 001c0000 fd:00 4981894 /lib/x86_64-linux-gnu/libc-2.23.so
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed8079000-7fbed807b000 rw-p 001c4000 fd:00 4981894 /lib/x86_64-linux-gnu/libc-2.23.so
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed807b000-7fbed807f000 rw-p 00000000 00:00 0
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed807f000-7fbed8095000 r-xp 00000000 fd:00 4981261 /lib/x86_64-linux-gnu/libgcc_s.so.1
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed8095000-7fbed8294000 —p 00016000 fd:00 4981261 /lib/x86_64-linux-gnu/libgcc_s.so.1
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed8294000-7fbed8295000 rw-p 00015000 fd:00 4981261 /lib/x86_64-linux-gnu/libgcc_s.so.1
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed8295000-7fbed82ad000 r-xp 00000000 fd:00 4981853 /lib/x86_64-linux-gnu/libpthread-2.23.so
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed82ad000-7fbed84ac000 —p 00018000 fd:00 4981853 /lib/x86_64-linux-gnu/libpthread-2.23.so
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed84ac000-7fbed84ad000 r–p 00017000 fd:00 4981853 /lib/x86_64-linux-gnu/libpthread-2.23.so
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed84ad000-7fbed84ae000 rw-p 00018000 fd:00 4981853 /lib/x86_64-linux-gnu/libpthread-2.23.so
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed84ae000-7fbed84b2000 rw-p 00000000 00:00 0
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed84b2000-7fbed84b5000 r-xp 00000000 fd:00 4981851 /lib/x86_64-linux-gnu/libdl-2.23.so
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed84b5000-7fbed86b4000 —p 00003000 fd:00 4981851 /lib/x86_64-linux-gnu/libdl-2.23.so
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed86b4000-7fbed86b5000 r–p 00002000 fd:00 4981851 /lib/x86_64-linux-gnu/libdl-2.23.so
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed86b5000-7fbed86b6000 rw-p 00003000 fd:00 4981851 /lib/x86_64-linux-gnu/libdl-2.23.so
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed86b6000-7fbed86bd000 r-xp 00000000 fd:00 4981572 /lib/x86_64-linux-gnu/librt-2.23.so
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed86bd000-7fbed88bc000 —p 00007000 fd:00 4981572 /lib/x86_64-linux-gnu/librt-2.23.so
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed88bc000-7fbed88bd000 r–p 00006000 fd:00 4981572 /lib/x86_64-linux-gnu/librt-2.23.so
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed88bd000-7fbed88be000 rw-p 00007000 fd:00 4981572 /lib/x86_64-linux-gnu/librt-2.23.so
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed88be000-7fbed89c6000 r-xp 00000000 fd:00 4981955 /lib/x86_64-linux-gnu/libm-2.23.so
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed89c6000-7fbed8bc5000 —p 00108000 fd:00 4981955 /lib/x86_64-linux-gnu/libm-2.23.so
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed8bc5000-7fbed8bc6000 r–p 00107000 fd:00 4981955 /lib/x86_64-linux-gnu/libm-2.23.so
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed8bc6000-7fbed8bc7000 rw-p 00108000 fd:00 4981955 /lib/x86_64-linux-gnu/libm-2.23.so
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed8bc7000-7fbed8bed000 r-xp 00000000 fd:00 4981852 /lib/x86_64-linux-gnu/ld-2.23.so
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed8bed000-7fbed8bee000 rw-p 00000000 00:00 0
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed8bee000-7fbed8c0b000 r–p 00000000 fd:00 84279997 /opt/RoonServer/Appliance/Base.dll
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed8c0b000-7fbed8c79000 r–p 00000000 fd:00 84279996 /opt/RoonServer/Appliance/RoonBase.dll
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed8c79000-7fbed8ce8000 r–p 00000000 fd:00 84280011 /opt/RoonServer/Appliance/Roon.Broker.Api.dll
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed8ce8000-7fbed8d6c000 rw-p 00000000 00:00 0
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed8d6c000-7fbed8dcb000 —p 00000000 00:00 0
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed8dcb000-7fbed8dd2000 rw-p 00000000 00:00 0
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed8dd2000-7fbed8ddb000 r–p 00000000 fd:00 84279903 /opt/RoonServer/Appliance/Roon.Client.Models.dll
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed8ddb000-7fbed8ddc000 rw-p 00000000 00:00 0
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed8ddc000-7fbed8de0000 r–p 00000000 fd:00 84279772 /opt/RoonServer/Appliance/RoonAppliance.exe
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed8de0000-7fbed8deb000 rw-p 00000000 00:00 0
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed8deb000-7fbed8dec000 rw-s 00000000 00:19 4 /dev/shm/mono.44393
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed8dec000-7fbed8ded000 r–p 00025000 fd:00 4981852 /lib/x86_64-linux-gnu/ld-2.23.so
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed8ded000-7fbed8dee000 rw-p 00026000 fd:00 4981852 /lib/x86_64-linux-gnu/ld-2.23.so
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7fbed8dee000-7fbed8def000 rw-p 00000000 00:00 0
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7ffc70c62000-7ffc70c83000 rw-p 00000000 00:00 0 [stack]
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7ffc70df7000-7ffc70dfa000 r–p 00000000 00:00 0 [vvar]
Mar 25 10:31:19 qiclab01 start.sh[41632]: 7ffc70dfa000-7ffc70dfc000 r-xp 00000000 00:00 0 [vdso]
Mar 25 10:31:19 qiclab01 start.sh[41632]: ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
Mar 25 10:31:19 qiclab01 start.sh[41632]: Native stacktrace:
Mar 25 10:31:19 qiclab01 start.sh[41632]: #011/opt/RoonServer/RoonMono/bin/RoonAppliance() [0x5064e6]
Mar 25 10:31:19 qiclab01 start.sh[41632]: #011/opt/RoonServer/RoonMono/bin/RoonAppliance() [0x5ca974]
Mar 25 10:31:19 qiclab01 start.sh[41632]: #011/lib/x86_64-linux-gnu/libpthread.so.0(+0x11390) [0x7fbed82a6390]
Mar 25 10:31:19 qiclab01 start.sh[41632]: #011/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38) [0x7fbed7cea428]
Mar 25 10:31:19 qiclab01 start.sh[41632]: #011/lib/x86_64-linux-gnu/libc.so.6(abort+0x16a) [0x7fbed7cec02a]
Mar 25 10:31:19 qiclab01 start.sh[41632]: #011/opt/RoonServer/Appliance/libohNet.so(+0x14ae29) [0x7fbd5945ae29]
Mar 25 10:31:19 qiclab01 start.sh[41632]: #011/opt/RoonServer/Appliance/libohNet.so(+0xeefc3) [0x7fbd593fefc3]
Mar 25 10:31:19 qiclab01 start.sh[41632]: #011/opt/RoonServer/Appliance/libohNet.so(+0x1416c7) [0x7fbd594516c7]
Mar 25 10:31:19 qiclab01 start.sh[41632]: #011/opt/RoonServer/Appliance/libohNet.so(+0x14a978) [0x7fbd5945a978]
Mar 25 10:31:19 qiclab01 start.sh[41632]: #011/lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba) [0x7fbed829c6ba]
Mar 25 10:31:19 qiclab01 start.sh[41632]: #011/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fbed7dbc41d]
Mar 25 10:31:19 qiclab01 start.sh[41632]: =================================================================
Mar 25 10:31:19 qiclab01 start.sh[41632]: Got a SIGABRT while executing native code. This usually indicates
Mar 25 10:31:19 qiclab01 start.sh[41632]: a fatal error in the mono runtime or one of the native libraries
Mar 25 10:31:19 qiclab01 start.sh[41632]: used by your application.
Mar 25 10:31:19 qiclab01 start.sh[41632]: =================================================================
Mar 25 10:31:19 qiclab01 start.sh[41632]: Error
Mar 25 10:31:21 qiclab01 start.sh[41632]: Initializing
Mar 25 10:31:21 qiclab01 start.sh[41632]: Started
Mar 25 10:31:23 qiclab01 start.sh[41632]: aac_fixed decoder found, checking libavcodec version…
Mar 25 10:31:23 qiclab01 start.sh[41632]: has mp3float: 1, aac_fixed: 0
Mar 25 10:31:23 qiclab01 start.sh[41632]: Not responding
Mar 25 10:31:28 qiclab01 start.sh[41632]: Running

noris · March 26, 2019, 1:09am

Hello @Steven_Neighorn,

Are you by any chance running Emby on the Ubuntu Core? I have seen an issue with having both Roon and Emby at the same time as documented in this thread (Roon Server on Linux core dump on startup).

– Noris

Steven_Neighorn · March 26, 2019, 5:13pm

Thank you Noris,

I am not running emby. I do run Plex on the same server. I am not using containers. The process list for roon and plex follows:

(notice that “start.sh” was run on March 22nd but RoonAppliance and processreaper and RAATServer all have start time stamps of 9:09AM (about an hour ago), since those are the programs that get restarted after each ABRT).

root 16061 41815 23 09:09 ? 00:14:01 /opt/RoonServer/RoonMono/bin/RoonAppliance --debug --gc=sgen --server RoonAppliance.exe -watchdogport=14798
root 16063 41815 0 09:09 ? 00:00:00 /opt/RoonServer/Server/processreaper 16061
root 16150 16061 0 09:09 ? 00:00:18 /opt/RoonServer/RoonMono/bin/RAATServer --debug --gc=sgen --server RAATServer.exe
neighorn 35008 21383 0 10:09 pts/2 00:00:00 grep --color=auto -i roon
root 41632 1 0 Mar22 ? 00:00:00 /bin/bash /opt/RoonServer/start.sh
root 41815 41632 0 Mar22 ? 00:00:29 /opt/RoonServer/RoonMono/bin/RoonServer --debug --gc=sgen --server RoonServer.exe

neighorn 35141 21383 0 10:09 pts/2 00:00:00 grep --color=auto -i plex
plex 38887 1 0 Mar24 ? 00:00:00 /bin/sh -c PLEX_MEDIA_SERVER_INFO_VENDOR="$(grep ^NAME= /etc/os-release | awk -F= “{print $2}” | tr -d " )" PLEX_MEDIA_SERVER_INFO_DEVICE=“PC” PLEX_MEDIA_SERVER_INFO_MODEL="$(uname -m)" PLEX_MEDIA_SERVER_INFO_PLATFORM_VERSION="$(grep ^VERSION= /etc/os-release | awk -F= “{print $2}” | tr -d " )" LD_LIBRARY_PATH=/usr/lib/plexmediaserver/lib “/usr/lib/plexmediaserver/Plex Media Server”
plex 38900 38887 1 Mar24 ? 00:29:17 /usr/lib/plexmediaserver/Plex Media Server
plex 38955 38900 0 Mar24 ? 00:05:04 Plex Plug-in [com.plexapp.system] /usr/lib/plexmediaserver/Resources/Plug-ins-782228f99/Framework.bundle/Contents/Resources/Versions/2/Python/bootstrap.py --server-version 1.15.2.793-782228f99 /usr/lib/plexmediaserver/Resources/Plug-ins-782228f99/System.bundle
plex 39044 38900 1 Mar24 ? 00:38:52 /usr/lib/plexmediaserver/Plex DLNA Server
plex 39045 38900 0 Mar24 ? 00:00:45 /usr/lib/plexmediaserver/Plex Tuner Service /usr/lib/plexmediaserver/Resources/Tuner/Private /usr/lib/plexmediaserver/Resources/Tuner/Shared 1.15.2.793-782228f99 32600 /waitmutex
plex 39255 38900 0 Mar24 ? 00:00:45 Plex Plug-in [com.plexapp.agents.thetvdb] /usr/lib/plexmediaserver/Resources/Plug-ins-782228f99/Framework.bundle/Contents/Resources/Versions/2/Python/bootstrap.py --server-version 1.15.2.793-782228f99 /usr/lib/plexmediaserver/Resources/Plug-ins-782228f99/TheTVDB.bundle
plex 39692 38900 0 Mar24 ? 00:00:44 Plex Plug-in [com.plexapp.agents.themoviedb] /usr/lib/plexmediaserver/Resources/Plug-ins-782228f99/Framework.bundle/Contents/Resources/Versions/2/Python/bootstrap.py --server-version 1.15.2.793-782228f99 /usr/lib/plexmediaserver/Resources/Plug-ins-782228f99/TheMovieDB.bundle

mjw · March 26, 2019, 6:55pm

Are you running Ubuntu Server or Desktop? Need to establish if you are running other Mono/ .Net applications on your system.

Steven_Neighorn · March 26, 2019, 9:55pm

Thanks again Martin. I am running the Ubuntu Server. This is on a Dell 2U R720xd. No X11 stuff, no mono apps, and no .Net apps. I believe the crashes started after the upgrade to 1.6 but I don’t know if it was 1.6 and then some ubuntu patches, or just 1.6 upgrade. I am sorry to say that I wasn’t paying
close enough attention to it. Roon is so good at restarting that I didn’t notice the crash info until I increased the level of logging and started seeing the “start.sh” output in a logcheck email.

noris · March 28, 2019, 1:36am

Hi @Steven_Neighorn,

I have moved your post over to the #support category for better tracking here. Can I please ask you to perform a test here? If you disable all of your Watched Folders in Roon and use Roon with just streaming service(s), does the same issue occur?

Thanks,
Noris

Steven_Neighorn · March 28, 2019, 6:15pm

Thank you Noris,

I have disabled the folder Roon was watching using Settings/Storage/Folders. The folder is now listed in red as “disabled”. I will update you with status. The last crash was at 8:23AM this morning, and it is now 11:15AM.

I now have 50 albums and 738 tracks, which are all from Tidal.

Steven_Neighorn · March 29, 2019, 11:54pm

Sorry to say the crashes continue. Maybe not as often but the roon services have crashed twice in the past 24 hours. Same exact errors as before. Let me know if you need the full text of the error or if you have other ideas.

noris · April 1, 2019, 8:27pm

Hello @Steven_Neighorn,

I have gone ahead and enabled diagnostics mode for your Core and what this action does is automatically upload a set of diagnostics to our servers for analysis. The report just arrived, and from what I can tell the storage locations are still active.

I am thinking that this crash might be media related, so as a test I was hoping that we can remove this variable from the equation. At the start of the log traces I am seeing these two media folders appear:

/mnt/media/music/0-9/10,000 Maniacs/Our Time in Eden/10,000 Maniacs - Our Time in Eden
/mnt/media/music/0-9/22 Brides/Beaker/22 Brides - Beaker

Can you double check to make sure that all of your storage folders including these two are disabled as a test as to rule out possible media-related issues?

Thanks,
Noris

Steven_Neighorn · April 1, 2019, 8:54pm

Hi Norris,

I had disabled the media, and just had the tidal account active. The server kept crashing. So I re-enabled, but I will turn the media watch folder OFF again so you can debug. I sent the “sorry to say” msg 3 days ago. I will turn that off right now. Thank you.

Steven_Neighorn · April 1, 2019, 8:56pm

OK, the /mnt/media folder is disabled. The only music the server sees now is my stuff from Tidal. 50 albums and 738 tracks. Please confirm that you see this too and let me know what else you need. I will
leave the media mnt disabled until you tell me otherwise.

noris · April 1, 2019, 9:34pm

Hello @Steven_Neighorn,

Thanks for disabling that location. Yes, I do see that it shows as offline in the diagnostics now.

Please reboot your Core and run this configuration for a day or two and note the exact local time (e.g. 5:33PM) of the next crash with the media locates turned off.

I will then take a look around your reported timestamp to see if there are any traces that could indicate the cause for the behavior.

Thanks,
Noris

Steven_Neighorn · April 2, 2019, 6:22pm

Crashed/restarted at 8:20AM PDT

root 17881 1 0 Apr01 ? 00:00:00 /bin/bash /opt/RoonServer/start.sh
root 17895 17881 0 Apr01 ? 00:00:06 /opt/RoonServer/RoonMono/bin/RoonServer --debug --gc=sgen --server RoonServer.
root 34264 45598 0 11:18 pts/2 00:00:00 grep --color=auto Roon
root 47098 17895 5 08:20 ? 00:10:20 /opt/RoonServer/RoonMono/bin/RoonAppliance --debug --gc=sgen --server RoonAppl
root 47100 17895 0 08:20 ? 00:00:00 /opt/RoonServer/Server/processreaper 47098
root 47205 47098 0 08:20 ? 00:00:23 /opt/RoonServer/RoonMono/bin/RAATServer --debug --gc=sgen --server RAATServer.

Let me know if you need more info from the server. thank you!

noris · April 3, 2019, 3:23pm

Hello @Steven_Neighorn,

Thanks for submitting that info and for trying the setup with storage disabled. I can confirm that the diagnostics report reached out servers and I have asked QA to take a look to see if they can identify any possible sources of this issue. I appreciate your patience until I hear back regarding the analysis.

Thanks,
Noris

noris · April 4, 2019, 6:57pm

Hello @Steven_Neighorn,

I just heard back from QA and they noted that this issue might be due to your networking interfaces on the Core. They have seen similar cases previously where turning off all except one network interfaces resolved the issue. Can you give this a try on your end and let me know if this changes anything with regards to the issue?

Thanks,
Noris