Preserving Library (Metadata) when moving RoonServer from ROCK to other OS

Support
1

When moving your library between different Roon installations (e.g. between Mac, Windows or Linux or just to a new PC) that quite often involves moving the music files to a different path in the new installation, if it’s residing on the same hardware as the RoonServer.

The recommended method to preserve database entries (manual edits, likes/disklies etc.) when moving your music library is to:

  • deactivate the storage location on the source installation in Roon settings
  • backup your library database
  • move your music files to the new path on the new installation / PC
  • restore your library database backup to the new Roon installation
  • edit the path of the deactivated storage location to point to the new path in Roon settings
  • activate the storage location again

This way Roon should recognize the music files as “already known” and not as new files so all metadata from the backup is kept intact.

This however doesn’t work when moving away from a ROCK installation that keeps music files on the “internalStorage” (i.e. the 2.5" SATA SSD in the NUC) as such a storage location is not present in any other installations, only in ROCK.

After restoring the database to the new non-ROCK installation (Linux in my case) there’s no option to edit the deactivated storage location “InternalStorage” to point to a different path. There’s simply no storage location at all until you add a new one. This way, the music files are treated as new and all metadata of the library backup is lost.

So my thinking was, if it was possible to recreate the mounting point used for “InternalStorage” in the ROCK installation in a target linux installation and whether RoonServer would be recognizing this as the original “InternalStorage” location this way preserving the metadata in the library database.

However, the mounting point for “InternalStorage” in the ROCK filesystem is not known. Externally it is represented as “\Data\Storage\InternalStorage” via SMB to access it over the network, but this doesn’t have to be the internal mount point, this could be quite different using internal symlinks to cryptic mount points created by the OS automounting detected drives.

Does anyone have any information on this?

Or is there a much simpler solution to preserve the metadata in your library database when moving from Rock to a different OS?

Note: When keeping the files on an USB drive attached to ROCK, this doesn’t happen, as this storage location can be manually edited and therefore “moved” to the new installation.

1 Like
3

Hello @Roland_von_Unruh,

Thank you for reaching Roon support.

Mounting point for the internal storage is “/dev/sda1 on /roon/sys/storage/mounts/InternalStorage”

We’ve discussed this with our senior QA and development teams. The team is investigating some possibilities here and, as soon as that investigation is complete, we’ll be sure to follow up ASAP.

You have our apologies for the trouble here, and we’ve greatly appreciated your patience as we continue investigating this tricky issue. We’ll be in touch as soon as we can.

2 Likes
4

Thanks for the quick response and investigating the issue!

Based on your information I tried both mounting the SSD holding the music files to
/roon/sys/storage/mounts/InternalStorage/dev/sda1
and
/roon/sys/storage/mounts/InternalStorage
because I wasn’t quite sure how to understand your feedback on the actual mount point.

The procedure was to stop roonserver, mount the drive, restart roonserver and restore the backup from the ROCK installation.

In both cases, the storage location wasn’t automatically detected as ROCK Internal Storage and had to be manually added as storage location.
The music files were treated as new and the original data was still left in the library as music files “not connected to a storage location” in the library cleanup dialog.

So obviously no success with the approach of trying to recreate the original ROCK storage location on the target linux system.

5

Everything in Unix and hence Linux is represented as a file. The hardware device is represented (as a file) as /dev/sda, the first partition is /dev/sda1.

This device file gets mounted into the file system as /roon/sys/storage/mounts/InternalStorage and it’s then accessible in this path.

That’s why a mount command in Linux needs to provide both the device file to be mounted and the path to mount it to, i.e., in general

mount -t ext4 /dev/sda1 /mnt

Thanks for bringing up this issue, very interesting.

7

@Suedkiez : THX for clearing this up for me, still learning in linux :slight_smile:

@Support: I wonder whether this problem is not limited to ROCK InternalStorage but might also relate to other OS’es default watch folders that can’t be edited (just activated/deactivated) like:

  • User Music folder in Windows
  • User Music folder in MacOS

For example if you have a RoonServer on a Windows Machine keeping your Music Files locally stored in the default watch folder (like: c:\users\user_xy\music) as a source installation and want to move to another OS (Mac or Linux or RoonOS) as target, you might run into the same issue of not being able to reassign the storage location to the new folder on the target system since the OS-specific default watch folder of the source system isn’t available on the target system and can’t be edited.

1 Like
8

For Watched Folders that you have added yourself, you can edit the filepath using the “3 dots” menu and the then “Browse” button in the “Edit strorage location” screen. That is a standard procedure in migration and Roon will use the new location for the Watched Folder.

9

Agreed, especially if you keep your music files externally on a USB drive or a NAS, there’s no issue migrating the RoonServer to a new system - even different OS - and keeping your metadata library.

The problem is with the OS-specific default watched folders that can’t be edited and don’t transfer to a new system through backup if the target system is a different OS.

10

I’ve moved from Linux to ROCK, and ROCK to Linux, and most recently, to a container, and every time the storage paths were recreated in Roon. (macOS was also in the mix at one time.)

I usually see a warning triangle against the old folders, so delete them before adding the new locations.

Not once did I loose my Roon metadata. I’m constantly updating FLAC metadata, and so long as the file modification time is preserved, Roon has no problem matching my library with the database.

The most important thing you must do, outside Roon, is preserve file attributes when copying. Anything that changes the file stream, e.g., changing FLAC compression, will change the file import date, but other than that, metadata should remain unchanged.

1 Like
11

In my scenario described above i’m accessing the exact same files on the internal 2.5" SATA SSD inside the NUC when moving from ROCK to Linux (simply by swapping the M.2 SSD holding RoonOS against a M.2 SSD with the Linux installation).

No music files are being copied around, so all file attributes are identical. It’s just the logical paths that change from ROCK (System default watched folder “InteralStorage”) to Linux (manually added watched folder to the mount path of the SSD).

Nevertheless the Linux RoonServer interprets these files as new and not as the files already known and identified in the backup restored from the ROCK installation.

I even tried the following in the ROCK installation itself without success:

  • deactivate the watched folder “InternalStorage” (this can only be activated/deactivated, not edited or deleted)
  • the music files show as “connected to a deactivated storage location” in the library cleanup dialog.
  • Manually create a watched folder pointing to the same files via network share (//rock/Data/Storage/InternalStorage) as it’s not possible to access the RoonOS filesystem directly for this
  • the files are treated as new and imported/identified again
  • the original data still shows as “connected to a deactivated storage location” in the library cleanup dialog.

This wasn’t surprising and was to be expected, tried it anyway.

The idea behind this was to create a manually editable watched folder to the music data with my library metadata attached inside ROCK before migrating and that this folder would be transferrable to a new system via backup (i.e. shows up in the new installation with a warning sign and could be edited to the new folder path).

12

Using the ellipsis, you should be able to edit or delete any storage location.

13

Not possible for “InternalStorage” on ROCK.
I can share a screenshot in the evening to illustrate.

Edit: See screenshot by @Geoff_Coupe. Also, the Location cannot be deleted via the “…” menu, only activated/deactivated.

14

The problem is that, unlike other Watched Folders, the Internal Storage of ROCK (and presumably Nucleus) does NOT have a “Browse” button on the Edit storage screen, so you can’t edit the the filepath…

image
image1477×1056 68.5 KB

3 Likes
15

Okay, I never used internal storage, so haven’t encountered this.

When I return home at the weekend I may fire up my ROCK, and experiment. However, this does appear to be unwanted behaviour. I get why you shouldn’t be able to navigate the Roon OS filesystem, but this should not affect the music library.

I wonder if Roon OS drops some hidden files (or certain permissions) in the storage, and these are still present because the drive was moved rather than the library?

16

RoonOS seems to leave the files in the storage untouched and just reads there, not write. Either way those hidden files would be there as well on for example an external USB drive with your music files that you carry over to a new RoonServer and shouldn’t cause any issues or even help RoonServer to identify the files as “already known”, not the opposite IMHO.

BTW:
Ownership of the Internal Storage is root:1001 (user:group) in RoonOS as can be seen with ls -l when the drive is mounted in the Linux installation. User and group are allowed to read/write.

AFAIK 1001 (or any other number instead of a name for that matter) indicates a group id that is not known/assigned to a defined group in the new Linux installation. Most likely it’s a specific group in RoonOS for accessing StorageDevices.

So i (recursively of course) adjusted group ownership in the Linux installation to a known group to enable writing new files locally and via SMB share.

And of course before moving back to RoonOS I reset group ownership to 1001 to avoid any issues with RoonOS.

17

Yet, a USB drive on Roon OS doesn’t have any restrictions or issues when moving to another non-Roon OS setup.

I don’t understand why internal storage should behave any different. Most likely, it has something to do with moving the drive intact to another server.

Have you thought about adding a USB drive to the ROCK, and moving some of the files, and seeing how this behaves? As usual, backup first, preserve file attributes etc.

18

Hi @Roland_von_Unruh,

Thanks for your continued patience. I believe the next step here is to escalate your case to our developers for a closer look. I’ve submitted a ticket on your behalf, and we’ll follow up with any updates as soon as we have more information to share.

20

Yes, because it’s a manually added watched folder. I also don’t understand why InternalStorage behaves different and so does Roon as it seems judging from @Daniel’s post above.

I don’t want to mess around anymore with my ROCK installation, however I just have tried the following with success:

  • setting up a spare MacMini M1 as RoonServer
  • copying all music files to the users music folder which is the system default watched folder for MacOS
  • restoring a backup of the ROCK RoonServer on login

This way, my metadata coming from the backup was associated with the files in the music folder!

So most likely RoonServer treats system default watched folders special from manual added folders and associates those over different OS (at least ROCK → InternalStorge to MacOS → MusicFolder).

The culprit might be here, that in the Linux environment there is no system default watched folder maybe because there’s no Linux Roon Remote App running in a user environment that could be used to point to a specific users home/music folder or similar.

So a possible approach to solve this might be to define a system default watched folder for the linux server distribution, that could be placed in the roonserver user home directory and serve as a mount point for your actual music files stored on another drive. This way, this folder could be treated similar to system default watched folders of RoonOS/ROCK, MacOS, Windows and the library would be easily transferrable between all four.

21

This is good to know.

Since server and control are separate apps, even on macOS and Windows, it shouldn’t matter what OS is used. And, whilst preventing access to the Roon OS filesystem may be important, this should be handled by the OS and not the Roon app.

Anyway, let’s see what comes back from the devs.

22

Hi @Roland_von_Unruh,

Thanks for letting us know what additional steps you’ve tried. I’ll make sure that information is added to the ticket so the team has the full picture as they continue the investigation. We’ll keep you updated as soon as we have more to share.

23

I have found a fairly complicated solution to this.

As written above, I was successful in setting up a RoonServer on MacOS and transferring the library through a backup of the ROCK installation on the new server with the music files in the system default Music Folder.

As a second step, I stopped RoonServer on MacOS and moved the music files to a different folder.

After Restarting RoonServer, I added this folder as a new watched folder to RoonServer and the files were correctly identified as “already known” with the existing metadata reattached, leaving 0 files to cleanup on the library cleanup dialog.

I deactivated the (empty) system default Music Folder and made another backup of the MacOS RoonServer library now with all files connected to a manually editable Storage location.

Now I swapped the M.2 SSD with RoonOS in the NUC for the M.2 SSD with the Linux installation (DietPi for native PC), cleaned up the Roon installation (deleting the roonserver folder holding the database and settings) for a fresh start and mounted the 2.5 SSD as “/mnt/music_disk” as well as adjusting group ownership to allow for r/w access.

After that I started RoonServer on the Linux machine and opted for restoring the new Backup from the above mentioned MacOS installation having the metadata connected to a manually added storage location. The location as expected showed up in Settings with red warning text and could be reassigned to the new mount point of the actual music files.

This way, the library metadata was finally successfully re-associated with the music files, again leaving 0 files in the library cleanup dialog.

Edit: it might also be possible to achieve this by attaching an USB HDD to Rock and move the files form internal storage to the external USB over SMB access while RoonServer is stopped as @mjw suggested and add the external storage location in settings after completion.

The point is the same, it is necessary to transfer the files from the system default watch folder to a manually added one on the source system before migration to a target linux system.
And of course one would have to move all files back on the internal storage again afterwards…

1 Like