Database backup issues

Ralf_Karpa · October 19, 2023, 1:17pm

Roon Core Machine

Mac Mini 2011 Model i5 16GByte

Networking Gear & Setup Details

Ethernet connection via Netgear switches, no VPN

Connected Audio Devices

NAD M10V2 via Ethernet

Number of Tracks in Library

approx. 100000

Description of Issue

I am experiencing problems with my backups. Until beginning of September my daily scheduled backups worked flawless. Every night the server did run this successfully. Since about 6 weeks it started to fail from time to time. The only message I see is:

But there is no additional info on the reason, no log, nothing I am aware of. Where can I find out what causes the fails? Backups are essential and I am feeling VERY unsecure of this.

I also tried to backup manually but the result was the same:

Mikael_Ollars · October 19, 2023, 1:47pm

Just a curious question here, but from your screen shots it seems you are backing up to the same share where your media files are located?
I’d firmly recommend a completely separate location for the database backups. And just checking, you are aware that backups does NOT include your media files, just your database, settings, edits, playlists and such stuff?

Ralf_Karpa · October 19, 2023, 1:53pm

Yes, I am aware of what is included in the backups. And that is my main concern as I am still editing my stuff and do not want to loose those edits because of a non working roon database.

Ye, I am using the share where the music resides as this is a Synology-NAS running on RAID-6 and that is bi-weekly backed-up on another Synology-NAS running on RAID-5.

Mikael_Ollars · October 19, 2023, 2:04pm

Problem is, your backups and database will be scanned by Roon at the same time as your backup is taking place and that will cause conflicts and file locks.
I strongly suggest you create another share on your NAS (or use another existing rather than “music”) and direct your backups there.

And i’d like to offer a piece of advise also;
It is generally better to adjust your files metadata, to allow Roon to identify and categorize the filesautomatically, than to edit in Roon.
It will save you from frustration in the event of change of playback software or consumer of your files (thinking of UPnP servers etc)

Ralf_Karpa · October 19, 2023, 3:52pm

Only the NAS folders with music are visible to roon. The backup folder is in another folder, only for that. I did not have any issues until about 6 weeks ago and did this kind of backup for > 2 years. Anyway, I also tested a local backup (as in my second screenshot) and that also failed. The 4th attempt went through.

I edit my metadata in the files first (adding all what is possible like lyrics, composers, version of the album, recordinglocation and -date, etc.). After everything is written to disk, I rescan the album, check whether identification causes trouble with composer (in that case I use a different version or leave it unidentified) and finally throw away the metadata provided by roon which is not correct (wrong person assigned or whatever other stuff comes in) or where are duplicates which are basically the same person. So after around 7000 albums prepared and maintained like that, I can state that there is a maximum 5% where roon provides correct metadata.

Mikael_Ollars · October 19, 2023, 4:13pm

Then i think we must ask @support to have a look at your logs.
I see two different scenarios: Your backup activity disturbs your database (the same share-issue) but that should be visible through logs.
The other option is network instability as it seems to work occasionally.
This should also be possible to see through the logs.

Regarding the metadata, you seem to know what you’re intent on accomplishing, but i still think you are fighting windmills.
(Or you have a very unique and eclectic collection of music, as my library indicate Roon get it right 95% of the time, and the rest are my job to report to Roon)

Stampie · October 19, 2023, 4:17pm

It might be worth having a look in the Logs folder to see if there is a more useful error message at the timestamp the backup was due to happen.

The only issue I’ve encountered with backup related to Roon losing a connection to my NAS share - by re-adding it in the Backup location window in Roon I was able to fix it. However, as your problem is intermittent, I doubt this would be of any help (I only mention in case you fancy trying it).

Also, just another access related check - if you select ‘clean/restore’ via the 3 dots in the scheduled backup window - does it correctly display a list of all your backups? I use this to delete some old backups I no longer need.

Good luck tracking the root cause down

Ralf_Karpa · October 19, 2023, 4:45pm

Thanks for your input, Stampie. I had a look at the log-files of the roon-server and was able to find the entries where the backups failed. The 2 backups stop at a different progress (10% vs. 30%). Further than that, I am not able to really understand the logs properly.

Nightly unattended run

10/19 01:13:11 Trace: [backup] uploading files. bytes transferred: 177766923/1788515518 (9%)
10/19 01:13:11 Trace: [backup] uploading files. bytes transferred: 179869871/1788515518 (10%)
10/19 01:13:11 Warn: [backup] failed to sync: putfile hash: AttachedDir:/Users/rkarpa01/Library/RoonMounts/RoonStorage_edd12c747c81ca48607327083b405492939114f6/Roon/6b952f04-ee94-dca4-0478-648ead8aa47a/fc/ed/fcedebc8aab8c46967a41c904704278edfea197f69e41daece739f8b2ed6694a: Result[Status=UnexpectedError, ErrorText=System.IO.IOException: Is a directory : '/Users/rkarpa01/Library/RoonMounts/RoonStorage_edd12c747c81ca48607327083b405492939114f6/Roon/6b952f04-ee94-dca4-0478-648ead8aa47a/fc/ed/.tmp-fcedebc8aab8c46967a41c904704278edfea197f69e41daece739f8b2ed6694a'
   at System.IO.RandomAccess.WriteAtOffset(SafeFileHandle handle, ReadOnlySpan`1 buffer, Int64 fileOffset)
   at System.IO.Strategies.OSFileStreamStrategy.Write(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.Strategies.BufferedFileStreamStrategy.WriteSpan(ReadOnlySpan`1 source, ArraySegment`1 arraySegment)
   at Roon.FileSystem.AttachedFileSystem.<>c__DisplayClass15_0.<PutFile>b__0()
   at System.Threading.Tasks.Task`1.InnerInvoke()
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
   at Roon.FileSystem.AttachedFileSystem.PutFile(Stream srcfile, IFileSystemDir parent_, String destname, Boolean overwrite, CancellationToken canceltoken)]
10/19 01:13:11 Warn: [broker/backups] failed sync on FileBrowser.Entry: \\epilogue, music : \Roon: Result[Status=UnexpectedError]
10/19 01:13:11 Debug: [broker/backups] on done, auto: False
10/19 01:13:11 Debug: [broker/filebrowser/volumeshare] Use dispose, id: \\epilogue\music
10/19 01:13:13 Debug: [broker/locations/directorystoragelocation] ev_subdirectory, path: /Users/rkarpa01/Library/RoonMounts/RoonStorage_edd12c747c81ca48607327083b405492939114f6/iTunes
10/19 01:13:17 Info: [stats] 10258mb Virtual, 2881mb Physical, 2375mb Managed
10/19 01:13:17 Trace: [broker/accounts] [heartbeat] now=18.10.2023 23:13:17 nextauthrefresh=18.10.2023 23:58:18 nextmachineallocate=19.10.2023 02:58:17

Daily run while watching the progress

10/19 13:29:46 Trace: [backup] uploading files. bytes transferred: 4871760931/16028496549 (30%)
10/19 13:29:46 Trace: [backup] uploading files. bytes transferred: 4871769391/16028496549 (30%)
10/19 13:29:46 Warn: [backup] failed to sync: putfile hash: AttachedDir:/Users/rkarpa01/Library/RoonMounts/RoonStorage_edd12c747c81ca48607327083b405492939114f6/tmp/RoonBackups/6b952f04-ee94-dca4-0478-648ead8aa47a/d4/56/d4563b08f127614791cd8cde1d7838c7ff9720efb209cff900483d146e6a8ed4: Result[Status=UnexpectedError, ErrorText=System.IO.IOException: Is a directory : '/Users/rkarpa01/Library/RoonMounts/RoonStorage_edd12c747c81ca48607327083b405492939114f6/tmp/RoonBackups/6b952f04-ee94-dca4-0478-648ead8aa47a/d4/56/.tmp-d4563b08f127614791cd8cde1d7838c7ff9720efb209cff900483d146e6a8ed4'
   at System.IO.RandomAccess.WriteAtOffset(SafeFileHandle handle, ReadOnlySpan`1 buffer, Int64 fileOffset)
   at System.IO.Strategies.OSFileStreamStrategy.Write(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.Strategies.BufferedFileStreamStrategy.WriteSpan(ReadOnlySpan`1 source, ArraySegment`1 arraySegment)
   at Roon.FileSystem.AttachedFileSystem.<>c__DisplayClass15_0.<PutFile>b__0()
   at System.Threading.Tasks.Task`1.InnerInvoke()
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
   at Roon.FileSystem.AttachedFileSystem.PutFile(Stream srcfile, IFileSystemDir parent_, String destname, Boolean overwrite, CancellationToken canceltoken)]
10/19 13:29:46 Warn: [broker/backups] failed sync on FileBrowser.Entry: \\epilogue, music : \tmp\RoonBackups: Result[Status=UnexpectedError]
10/19 13:29:46 Debug: [broker/backups] on done, auto: False
10/19 13:29:46 Debug: [broker/filebrowser/volumeshare] Use dispose, id: \\epilogue\music
10/19 13:29:48 Debug: [broker/locations/directorystoragelocation] ev_subdirectory, path: /Users/rkarpa01/Library/RoonMounts/RoonStorage_edd12c747c81ca48607327083b405492939114f6/FLAC

Yes, whenever it was a successful backup, I can see them in the restore-panel. I use auto-rotation using 10 recent backups.

Stampie · October 19, 2023, 5:59pm

Hopefully support can interpret the log errors and provide a solution.

Ralf_Karpa · October 25, 2023, 5:45am

Is anyone able to read the above logs and give some advise?

Is there a way to get an email response from the core whenever a backup fails?

Ralf_Karpa · October 31, 2023, 6:25pm