Scheduled Backup failed on network drive, directory not available

[Hans3d (Hans)]

Roon Core Machine

Roon Rock on a NUC 10i5FN
External USB SSD for Music storage

Networking Gear & Setup Details

Router: AVM Fritz.box 6690 with external USB drive as Network location (NTFS)
Netgear switch
all connectet over ethernet cable

Connected Audio Devices

NAD M33 over ethernet
Focal Arche over USB from the NUC

Number of Tracks in Library

22.000 tracks

Description of Issue

I have scheduled 2 Database Backups: one is backed up on the external USB Drive and works just fine, the other is to a Network share (a USB SSD plugged into the AVM FritzBox) and this keeps failling: Backup directory not available

I have recreated the Backup Task, added a new share (now with the IP instead of the Server Name) and created the Backup Folder within Roon and manually run the first Backup successfully.
But the next day i tried to force another update to that directory and failed.

The Folder is accessable from my Windos PC and I can also browse and create Folders within Roon, just the backup quits with a fail.

Would be nice to get this running ;o)
thanks

[Hans3d (Hans)]

here is what I could find in the Log files:

02/17 10:19:21 Debug: [broker/backup] start backup, force? True
02/17 10:19:21 Trace: [broker/backups] doing backup now
02/17 10:19:21 Trace: [broker/backups] computing list of files to back up
02/17 10:19:21 Debug: [broker/backups] preparing sync on FileBrowser.Entry: \\192.168.178.1, fritz.nas : \T7Shield1TB\RoonDB_Backup
02/17 10:19:21 Debug: [broker/filebrowser/volumeshare] Use volume, user: broker/backups/startbackup, id: \\192.168.178.1\fritz.nas
02/17 10:19:21 Info: [broker/filebrowser/volumeshare] Volume's removable changed: False
02/17 10:19:21 Trace: [backup] preparing backup on dir AttachedDir:/roon/sys/storage/smbmounts/RoonStorage_24edaf998dd2af48574971ef4e9c1bfd19a4257d/T7Shield1TB/RoonDB_Backup
02/17 10:19:21 Trace: [backup] migrating backups on dir AttachedDir:/roon/sys/storage/smbmounts/RoonStorage_24edaf998dd2af48574971ef4e9c1bfd19a4257d/T7Shield1TB/RoonDB_Backup
02/17 10:19:21 Trace: [backup] getbackupdirs, rootdir: AttachedDir:/roon/sys/storage/smbmounts/RoonStorage_24edaf998dd2af48574971ef4e9c1bfd19a4257d/T7Shield1TB/RoonDB_Backup, machineid: 0acadc73-1f72-d5d5-0260-967e26d88e96
02/17 10:19:21 Warn: [backup] failed to syncprepare: getfile manifest: AttachedDir:/roon/sys/storage/smbmounts/RoonStorage_24edaf998dd2af48574971ef4e9c1bfd19a4257d/T7Shield1TB/RoonDB_Backup/b_20230216215758_74a8a41e-4ca9-4883-b68f-ab8dcb95c4ba: Result[Status=UnexpectedError, ErrorText=System.IO.IOException: Stale file handle : '/roon/sys/storage/smbmounts/RoonStorage_24edaf998dd2af48574971ef4e9c1bfd19a4257d/T7Shield1TB/RoonDB_Backup/0acadc73-1f72-d5d5-0260-967e26d88e96/xx/b_20230216215758_74a8a41e-4ca9-4883-b68f-ab8dcb95c4ba'
   at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirectory, Func`2 errorRewriter)
   at Interop.CheckIo(Error error, String path, Boolean isDirectory, Func`2 errorRewriter)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize)
   at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize)
   at Base.IO.LongPathFile.Open(String path, FileMode mode, FileAccess access, FileShare share)
   at Base.IO.LongPathFile.OpenRead(String path)
   at Roon.FileSystem.AttachedFileSystem.<>c__DisplayClass14_0.<GetFile>b__0()
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
   at Roon.FileSystem.AttachedFileSystem.GetFile(IFileSystemDir parent_, String name, Stream outstream, CancellationToken canceltoken)]
02/17 10:19:21 Warn: [broker/backups] failed sync prepare2 on FileBrowser.Entry: \\192.168.178.1, fritz.nas : \T7Shield1TB\RoonDB_Backup: Result[Status=NotAvailable]
02/17 10:19:21 Debug: [broker/backups] on done, auto: False
02/17 10:19:21 Debug: [broker/filebrowser/volumeshare] Use dispose, id: \\192.168.178.1\fritz.nas
02/17 10:19:21 Info: [broker/filebrowser/volumeshare] Volume's removable changed: True
02/17 10:19:30 Info: [stats] 27307mb Virtual, 973mb Physical, 635mb Managed, 321 Handles, 82 Threads

[MamaTried (Paul)]

@Hans3d

UnexpectedError, ErrorText=System.IO.IOException: Stale file handle :

Hello, I’m just a fellow user but it looks like Roon doesn’t see the mount from your NAS as being properly “clean” mounted. It has been my experience you will need to clean the filesystem. a fsck -y or a reboot are two options. Hopefully others will chime in with a better method if it exists.

[Hans3d (Hans)]

today I tried to reformat the external SSD with exFat, but unfortunately my Router OS does not support this File System (yet, there is an update coming that should support exFat…some day). So I ended up reformatting again to NTFS. This should rule out any problems with a messed up filesystem at least.
But also on a freshly formatted Drive (with all files copied back) I still get the same error.

[Hans3d (Hans)]

I have made some minor changes: first I moved all the Musik Files on the USB Drive into an ‘Musik’ folder, so now there is no more Backup Folder INSIDE the music files, Backup (on the USB) and Musik are properly seperated in their own folders.

unfortunately my second Backup to the network share still wont work.

When I delete the existing Backup on the Network Share from my Windows PC and then force a new backup in Roon to that location, the backup runs just fine!

But trying a second run then fails with the directory not available error…

[connor]

Hi @Hans3d,

Just for due diligence, do you have any network security software enabled on this Core? Do you experience any connectivity issues on the network?

The team will investigate logs more thoroughly, but the trace you posted suggested the drive isn’t available for the duration of the backup. We’ll respond shortly with more information.

[Hans3d (Hans)]

Hi Connor!
thank for answering.
No there is no extra security tool running. The Core is running ROCK on a NUC, doesn’t get more basic ;o)
Also i can’t find any Network issues. The Network share is accessible from my windows PC and all Android devices and also ROON can access it with no problems. I can easily create a new folder from within ROON on that network share.
And when the bakup folder on the Network share is empty (when i delete any old backups there) the backup runs just fine.
But then its stuck and wont do another backup.

interestingly when i start a backup, usually i get the ‘snapshotting database’ screen for a few seconds, before the data is copied to the backup location. On this Network share I don’t see this screen. It says ‘preparing’ and then pretty fast ‘backup failed…’

I had a look inside the backup folder again after another failed attempt to run a backup.
the only files that show an updated date are the ‘roon_backup_root’ , ‘roon_backup’ , and ‘name’ files, none of the many 2 digits folders is touched or changed.

[connor]

Hi @Hans3d,

The team looked into this issue a little further and found traces of a stalefilehandle error when attempting to mount the internal storage location.

Digging deeper, the symptoms are broadly consistent with this documented Ubuntu error: mount - Stale file handler when mounting CIFS SMB network drive from Fritz router - Ask Ubuntu. There’s a reported workaround here.

Please let us know if this information is helpful, and we’ll investigate recent diagnostics from the weekend in the meantime.

[Hans3d (Hans)]

@connor

t us know if this information is helpful, and we’ll investigate recent diagnostics from the weekend in the meantime.

Hi and thanks again for investigating!

I tried giving the full Linux mount command line into the ‘Add network share’ page of ROON
(
mount -t cifs //fritz.box/fritz.nas/T7Shield1TB -o rw,noserverino,username=user,password=password,workgroup=WORKGROUP
)

And to my surprise the Network share was succsessfully created. I dont know if all the parameters around the actual path where ignored though.

anyways, the result is still the same: Backup failed, directory not available…

[Gianluca_Milza]

I have basically the same issue with a Fritz Box 7590 (latest firmware), with a Samsung USB SSD attached to it, exFat formatted.
I use Rock on a NUC nuc10i7fnh.
And to add: i also see that changing the folder for backup but reusing the same share (so, without reconfiguring it), always works, at least at the first backup.

[Gianluca_Milza]

It have searched for it and found that is a rather common error, on fritz box and other NAS solutions.
I found a more extensive discussion here and one specific solution here..

Both of those (and other i found) in the end recommend to use the noserverino mount flag on the client.

Without changes in Rock/Roon this is unfortunately not possible, as i don’t think we can touch Rock smb filesystems mount options. I didn’t really understand if there is something that can be done server side, certainly not in the Fritz Box configuration (only option is enable smbv1, but you don’t want that and in any case Rock probably will connect using a recent protocol version).

It is rather unfortunate, now i will lose regular backups.