Roon server not completing start-up: continaully attempts restart

Hi @Raymond_Daniel,

I’ve gone ahead and re-opened your old ticket.

I spoke to QA regarding the old files which had issues on your end, and they weren’t able to reproduce the behavior, so I don’t believe these are the correct offending files here.

I’m not seeing your Antipodes communicating with our diagnostics systems, could you or @Tony_Devitt please access the Roon logs on this Core and send me a new set?

Hi @noris

Would you like me to share the logs via Google drive or would you prefer I send them, perhaps to the same link as last time?

Also, is there something @Tony_Devitt or I can do so that my Antipodes connects with your diagnostics?

Hi @Raymond_Daniel,

You can certainly upload the logs to the previous location I sent, just please make sure to upload them as a separate .zip archive and let me know once you have done so and I’ll check the folder.

A reboot of the Core usually resolves issues with Cores communicating with our diagnostics servers, so I would give that a try. If the communication is still not established, I would check for any firewalls on the router which may be blocking access to *.roonlabs.com addresses.

Hi @noris

Logs uploaded.

I suspect diagnostics are not connecting because Roon Server will not start. I can check the firewall later today, just to be sure.

1 Like

Hi @Raymond_Daniel,

Thanks for sending those logs over. I took a look through them and I’m not seeing much information, it looks like RoonServer is trying to start and when it tries to mount your NAS001\Backup drive the log ends. Could you try temporarily powering off your NAS001 location to see if this is a factor in the issue?

Hi @noris,

Just tried cycling with variations. Neither restarting Roon Server or bouncing the Antipodes, with the backup NAS on or off or disconnected from the network, makes a difference.

When the system first started with this type of behaviour, it occurred only occasionally and was resolved by ensuring the backup NAS was turned on. The frequency of occurrence gradually increased over roughly two years (fortnightly - weekly - every few days) and could usually be resolved by turning on the NAS and restarting Roon Server or in come cases bouncing the Antipodes. Now, turning on (or off) the NAS seems to have no effect.

It seems odd that reinstalling Roon Server seems to resolve the issue, yet the behaviour starts recurring within a few days. I thought it might be a setting I change after the reinstallation is tested (backup, Tidal) that might causing the issue… but I don’t have enough into to make even a vague correlation. Probably unlikely anyway.

No progress at my end.

Hi @Raymond_Daniel,

Thanks for this bit of info, this confirms my suspicion that the NAS has something to do with the issue.

Have you tried reinstalling Roon Server and having the NAS completely disassociated with Roon? As in don’t even add this share at all to Roon?

I wonder if that would allow a longer stable connection and something regarding the NAS or structure of the backups folder is causing issues with the Core booting up.

Possibly there are some more dodgy rip files on the NAS similar to those I found on the Antipodes hard drive?

@Tony_Devitt, Had not thought of that. I’ve deleted the files from the backup NAS, rebooted the NAS then rebooted the Antipodes. No change - still looping in the same way.

Only immediately after reinstallation of Roon Server. I have usually reconfigured backup soon thereafter. I’d be happy to try, though Tony may need to do that.

Hi @Raymond_Daniel,

What happens if you disconnect the NAS and reboot your Antipodes server (leaving the NAS offline)? Does the Antipodes still fail to start properly?

Hi @noris,

Sorry for delay in getting back to this - tough work week.

With the backup NAS now disconnected from the network, neither bouncing the Antipodes or restarting Roon Server has made a difference: Roon Server is stuck in the start-up loop. Could removing Roon Server’s backup mount help matters? Not sure how to do this without it starting and being able to access it.

Hi @Raymond_Daniel,

No worries about the delay. Apologies on our end as well due to the holiday weekend.
Can you please send me a new set of logs? I’d like to get the log-set in front of QA, maybe they can see something I missed when I took a look.

Hi @noris,

Done and message sent with link to shared resource.

I believe I have found a cause for this problem.

I had the same symptoms show up on another Antipodes server, only the second time this has happened. After completely rebuilding the software, to ensure it was not a software problem, and still having the same problem I eventually found these files in one of the album folders, that should contain only flac files:

image

When I removed these files and cleared Roon cache Roon Server started normally. When I copied them back into the folder Roon Server reverted to continuously restarting. I was able to reliably reproduce this behaviour.

Specfically it is the VLC file that causes this. I suggest that @Raymond_Daniel will have to try to check your music library for .vlc media files and remove them.

1 Like

Hi @Tony_Devitt,

Thanks for this. I’ve never used that type of player nor remember ripping to VLC format - doesn’t mean those files couldn’t make it onto the system though. I searched a few different ways and didn’t find any VLC files on the Antipodes.

Note: all my music is maintained in the FLAC folder on the Antipodes - I assume this is okay.

Tony, what is the file extension on that file that will be opened by the VLC Media Player application? It looks as though it could be .crf format? Thanks.

Hi @Tony_Devitt,

Thanks for the follow-up here. I too would be curious to know what it says to the right of the VLC Media File description, what is the exact extension of the problematic file? Also, to clarify - the .M3U/.M3U8 files didn’t cause issues, but just the VLC file with the .CXX extension?

I also just received further feedback from QA and it looks like the Roon logs don’t show much info, it just looks like RoonServer is crashing when loading the storage backend. If there are no such VLC/.CXX files on @Raymond_Daniel’s Core, then System Level logs would be helpful to troubleshoot further.

It seems I spoke too soon, apologies to all. Yesterday I was able to copy the file, it is a .cue file, to a folder in the music library, restart Roon Server, and see it restart repeatedly, and then remove it, clear cache and see Roon Server start normally. I repeated this three times successfully.

However today, after rebooting the Antipodes server, the error behaviour has returned, although the .cue file and the M3 files are no longer present in the music library. And copying the .cue file to a different server does not cause the error on that server.

So problem not solved yet I am afraid.

The system journal only shows for each server restart:

… where ‘Error’, 18 seconds after the ‘Started’ message, is the only notification.

Will continue to investigate … really thought I had nailed it :frowning:

After spending several hours on this I am thinking that the problem may be caused by lack of processor power on the Antipodes server. The server on which I have the problem dates from 2015 and has had no hardware upgrade since then. CPU is: Intel® Atom™ CPU N2800 @ 1.86GHz (family: 0x6, model: 0x36, stepping: 0x1).

I have set up an Antipodes CX with the identical music library, and it runs the same system and application software as was rebuilt in the past couple of days on the older server.

After reinstalling the Roon Server software Roon Server on the older Antipodes loads the music library first time OK, including the .cue file, and can be used. But when it is restarted it immediately goes into the repeated restart mode. On the other hand Roon Server works flawlessly with the identical music library and has no problem with restarts.

I have saved the Cache folders straight after the initial music library scan, and then after the first failed restart, in case they hold any clues.

The server I have been testing with has to go back to the owner. He in fact does not use Roon on it.

Firstly, thanks to @Tony_Devitt and @noris for the time and effort to troubleshoot the issue. Very much appreciated.

I’ve found no .cue, .crf or .c* files (by name and type) in the music folder and sub-folders of the Antipodes.

Thanks again all.

1 Like