Roon stopping randomly

[fschmeis]

Hi @noris,
just to clarify: Have installed 1.7, problem remains:

• Had the situation, that Roon crashed right after starting a track, could reproduce that behavio; had to restart the NAS to get rid of this
• After restarting the NAS I had several crashes after 5 to 20 min of music listening

What can I do? What info do you need to fix the issue?

Thx in advance

Regards
Frank

[noris]

Hi @fschmeis,

Thank you for confirming this behavior is the same on 1.7.

I have been discussing your case with the QA team and to be upfront with you, we’re not exactly sure why this seems to randomly occur.

It looks like your QNAP might be running out of RAM, it seems to hit a RAM threshold at one point and RAM appears not to be increasing past this, even though it is not at max capacity that the QNAP has installed.

I cant’ say for sure if other apps on the QNAP are using the rest of the RAM and possibly preventing Roon from using any further RAM.

One thing that QA did notice is that you have a significant library size and it might just be that the QNAP is not able to keep up with such a large database.

If you are so inclined, they have suggested to reduce the library size by at least half and verify if the same problems occur.

To do this, you would need to create a Roon Backup and then try setting the current database aside. Afterwards, try importing at most 50k tracks and verify how it performs.

I look forward to hearing your thoughts. Thank you.

[PixelPopper]

I have been working in “domestic computing” for long enough to know that when a drive system is a full as yours is I would be increasing capacity as a matter of urgency to avoid problems.

Suggestions from @support to reduce the size of your library to try to prove the source of the problem confirms my suspicions. Good luck in your investigations.

[fschmeis]

Hello @noris,
thanky for talking to the QA-team; I ordered RAM (to upgrade from 8 GByte to 16 Gbyte) and an addition WD red 4TByte disk and will report back whether the crashes disappeared, will also track the memory usage when the system crashes
Cheers, Frank

[fschmeis]

hello @noris,
RAM extension succeeded:

Will report what happens … additional WD red 4 TByte on the way …
Cheers, Frank

[fschmeis]

Hello @noris,
after installing the additional 8 Gbyte (resp. substituting the 8 GByte by 16 GByte) Roon crashed just 3 min later, 11 pm local time in Germany, 2nd of December

The already observed behavior after the crash: CPU goes up for appr. 3 minutes:

Usage of Memory:

I have deactivated nearly all other applications, there is no VM running or any other RAM consuming process:

Logfiles can be found here: https://www.dropbox.com/s/j5icfrhukqbdxde/download_2019-12-02_23-17-25.zip?dl=0

Should I run the multi media stream add on or should I stop it?

Cheers
Frank

[noris]

Hi @fschmeis,

Sorry to hear that the new RAM hasn’t helped. I think it’s best that we focus our efforts to a previous point I raised:

If you are so inclined, they have suggested to reduce the library size by at least half and verify if the same problems occur.

To do this, you would need to create a Roon Backup and then try setting the current database aside. Afterwards, try importing at most 50k tracks and verify how it performs.

I also took a look through the logs and I’m not seeing anything jump out at me, the above test would hopefully reveal if this issue is due to the database or elsewhere. Note: Do not restore your database from a backup, please just try importing the tracks as fresh.

[fschmeis]

Hello @noris,
okay, if helpful I will do so during this weekend. Just to be sure: What we had done appr. 1 year ago (it is the same problem all the time) was to connect another core with my library:

Hello @dylan,
so far the (old) notebook running the core and connected directly to the router runs without any stoppages.
As the QNAP had stoppages both connected to the managed switch as well as connected directly to the router it seems that the root cause is not the managed switch but the installation of Roon on my QNAP.
So, how to proceed?
Next step could be: I connect the notebook to the managed switch (I don’t expect any stoppages, this would verify that the Roon installation on my QNAP is the cause and not the managed switch)?

At least the library itself doesn’t seems to be the reason for the stoppages, to me it seems the core running on my QNAP is the reason, maybe in conjunction with the size of my library; therefore: If it will help you to identify and hopefully fix the problem I am faced with for more than one year I will reduce the size of my library this weekend …
Cheers, Frank

[noris]

Hi @fschmeis,

so far the (old) notebook running the core and connected directly to the router runs without any stoppages.

I do recall performing this test, but the laptop and the QNAP surely have different hardware and the specs of the laptop might have allowed a larger database.

to me it seems the core running on my QNAP is the reason, maybe in conjunction with the size of my library

Yes, this is our current theory here as well. It would be good to set the current database aside verify if the same issue happens on a fresh library with under 50k tracks in the database.

Thank you.

[fschmeis]

hello @noris,
did add another 4 TByte drive, extended the existing volume (took appr. 36 h), QNAP performed a resync of the RAID-group (took another day), then copied appr. 25.000 tracks somewhere else, reset the RonnDB location and started Roon again:

will report what happens …
Cheers, Frank

[fschmeis]

hello @noris,
1st of all: since running the roon core with appr. 25.000 tracks I didn’t had any crash anymore
I am just not yet sure what the reason is: when re-configuring roon last weekend I detected a “bug” in my configuration: ROONDN was located on the same RAID group as my library (the SSD was “idle”)

as far as I remember crieke and I moved the ROONDB away from the SSD to identifiy the reason for my crashes but the crashes didn’t disappear, to say it the other way around: I had the crashes already when the ROONDB was still on the SSD

nevertheless: if you don’t mind I would now go back to my large library but move the “old” ROONDB to the SSD and look what happens … okay?

[noris]

Hello @fschmeis,

Thank you for the update here, glad to hear that changing the RAID settings appear to have helped!

nevertheless: if you don’t mind I would now go back to my large library but move the “old” ROONDB to the SSD and look what happens … okay?

I would only perform one of these changes at a time so that we can narrow down which change helped. I would suggest going back to the SSD but still keep the smaller library. And then if the system is still stable only afterwards try the larger library.

Thanks!

[fschmeis]

hello @noris,

I would suggest going back to the SSD but still keep the smaller library. And then if the system is still stable only afterwards try the larger library.

this is how the system is running since monday: on SSD and using the small library

[noris]

@fschmeis - Thanks for clarifying. In that case, yes, you can go ahead and move the old database back to the SSD which is currently hosting the library. Do let me know how it goes!

[fschmeis]

@noris: to give you an update: Roon is running much more stable (just two crashes in den meantime); couldn’t test thoroughly as I was very busy and actually have a QNAP issue …

one question is still open, do you have a recommendation (I assume that one crash was forced by parallel indexing of the multi media stram add on while using Roon):

Should I run the multi media stream add on or should I stop it?

[noris]

Hi @fschmeis,

Glad to hear that the system is running in a more stable fashion!

one question is still open, do you have a recommendation (I assume that one crash was forced by parallel indexing of the multi media stram add on while using Roon):

We’re not aware of any reports of Multi Media stream add-on causing issues with Roon, you can try both variations with/without the module, but it should be fine to leave it as-is.

[QuinnT (BANNED)]

I am having this issue (between roon and linn dsm/3) on and off for a long time, restarting roon did not help, so I decided to reboot the linn, and it fixed it.

[fschmeis]

@noris: just to give you an update:

and actually have a QNAP issue …

since beginning of 2020 my QNAP is in the netherlands to be repaired, the backplane is damaged …

@QuinnT: thx for your hint

so I decided to reboot the linn, and it fixed it.

unfortunately that doesn’t help in my case …

[fschmeis]

Roon Core Machine

QNAP TVS-671 with 16 GByte of main memory, QNAP firmware is the current version: 5.0.0.180, CPU is a Intel i5, ffmpeg : 3.3.6, QPKG-Version : 2021-05-18, running Roon Server Version 1.8 (build 831); there are 5 x 6 TByte (WD red plus) disks in RAID-1 where the music is stored and one Samsung SSD 850 Pro 256 GByte where the RoonServer and Database is installed

Networking Gear & Setup Details

Router: FRITZ!Box 6591 Cable, GBit Switch: Netgear ProSAFE JGS524PE, Wifi: FRITZ!Repeater 3000 (connected via Ethernet to the Router); iPAD as control point via Wifi, Windows 10 Notebook conntected via Ethernet as control point

Connected Audio Devices

LINN Klimax DSM via Ethernet

Number of Tracks in Library

appr. 350.000

Description of Issue

Since september 2018 I am faced with randomly crashes of Roon Server (see history of this thread); at the beginning it was not tooo often, with my growing library it crashed more often but once Roon was up and running and survived for some minutes it was “rather” stable; most of the crashes occured just after Roon Server has been restarted; but during the last months it became worse and since my library grew to appr. 350.000 tracks things are really bad, Roon isn’t usable anymore

9th of October, 6:05 pm, started to play a track of a local album, after 21 sec: CRASH

Roon Server takes something like 5 min to recover from the crash, during this time the CPU of my NAS is loaded appr. 50% with “RoonAppliance”; then the Album Art in the Control Point on my iPAD reappears

9th of October, 6:17 pm, Album Art in Roon Control Point on my iPAD reappears, CPU in NAS goes down, restart the track, Roon play the track this time for 3 sec, crash again …

I downloaded all logfiles as a Zip (Button in Ronn Server in “App Center” of QTS of my NAS) and had a look at the logfile “ROONSERVER_QNAP_LOG.txt”; at the timestamps when the crashes occured is written:

09.10.21-18:19:22 — =================================================================
09.10.21-18:19:22 — Native Crash Reporting
09.10.21-18:19:22 — =================================================================
09.10.21-18:19:22 — Got a SIGSEGV while executing native code. This usually indicates
09.10.21-18:19:22 — a fatal error in the mono runtime or one of the native libraries
09.10.21-18:19:22 — used by your application.
09.10.21-18:19:22 — =================================================================
…
09.10.21-18:19:22 — Could not exec mono-hang-watchdog, expected on path ‘/share/CACHEDEV1_DATA/.qpkg/RoonServer/RoonServer/RoonMono/etc/…/bin/mono-hang-watchdog’ (errno 2)
09.10.21-18:19:22 — Error
09.10.21-18:19:24 — Initializing
09.10.21-18:19:24 — Started
09.10.21-18:19:27 — Running

So I thought, may be something is missing in my RoonServer installation on my NAS, therefore I pushed the button “Replace the current Roon Server Application with the latest one from the Roon Labs web site” in “Roon Server” in “App Center” of QTS of my NAS

tried to restart the track, again: immediate crash

PLEASE HELP

[noris]

Hi @fschmeis ,

@fschmeis

I downloaded all logfiles as a Zip (Button in Ronn Server in “App Center” of QTS of my NAS) and had a look at the logfile “ROONSERVER_QNAP_LOG.txt”

Can you please upload the entire contents of the log files you downloaded, including RoonServer logs? Perhaps those contain further clues, thanks!