ROCK storage migration problems: backups and more ... long

Marco_Sassone · April 5, 2020, 4:21pm

Hi everyone, firstly, wishin’ you’re all safe out there.
Thankfully, in these tragic times, here I’m facing trivial problems with computer setups …

Due to lockdown, I decided to do some maintenance and to streamline my Roon setup.

I wanted to silence my perfectly working NUC7I5BNH and I placed it into an Akasa Plato X7 fanless case (found the last one available on Amazon …).
It works and looks great.
Troubles started when I wanted to migrate my 600GB library from the old external (NTFS) USB drive to an internal drive. Mind you, I followed the guidelines and [FAQs] and HowTo and for every issue I encountered, I researched extensively in the Roon Community.
I also followed this thread
and found myself in this situation.

Issue 1
My initial storage setup looked like this, with three folders in the external USB Drive, showing like three different ROON Optimized Core Kit:

I used the ext4 driver from Paragon in order to mount the Roon formatted internal drive in my iMac and copied all three folders of my library from the USB external over to the internal.
I expected to simply disable and edit the old locations of each folder, pointing to the respective new ones on the internal drive and be ready to go. No way.
They never showed up and the drive always showed as single drive … terrified to lose my database as I found out any of my backups were restorable (see more on this in issue 2), I finally disabled all and then hit the ‘Enable’ button for the internal drive and left working overnight, fearing of ending up with thousands of duplicates and not a database to restore.
I ended up with the following situation:

Is it this how it is supposed to be?

I’m not sure whether to delete the old disabled locations (on the old USB external drive ) or keep them at least for a while, until all settled.

Issue 2
No matter what I tried, I wasn’t able to restore any of the backups. They all show as ok and restorable, but each time I tried, from the Roon preferences or unlogging and from the login screen, I always get this:

The backups reside on two USB sticks exFAT formatted (8GB and 16GB). None on both of them seems restorable. I tried to reformat one of them in a Windows 7 PC (my main computer is a 27"iMac running Catalina).
I also followed the workaround @noris suggested here Terminal workaround.
No luck.
The only thing I can add is that all backups originally resided on another external drive ( size 1TB: NTFS or exFAT formatted I can’t recall). Because its size was overkill and wanted to use it as a media drive, I decided to pass the backups on the USB sticks.
Every time from ROON I try to make a new backup to the stick, it works, and all of them show up in the list of available to restore. But none work.
Are my backups useless?
Am I missing something, as I don’t find

Issue 3
Around a year ago I reported a problem of out-of sync grouped Airplay endpoints.

in which @john helped and the situation got much better.
Unfortunately, I have to report the issue is sometimes, much less, but still happening.
Just worth reminding that two of the three Airplay endpoints (all Airport Express + 1 AppleTV 3rd gen) show a different IP than the network I set up and expect to see( 10.0.1.xxx).

Thanks for any help

stay safe
Marco

noris · April 7, 2020, 5:40am

Hi @Marco_Sassone,

I’ve moved your post over to #support. You should post any support inquiries here as this is the only section of Community regularly monitored by support staff.

Did you follow our Preserve Edits Guide when you made the migration?

It is important that Roon does not see two copies of the same audio files at the same time. Since you have duplicates, I fear you may have accidentally had Roon perform the import improperly, so let’s focus on issue #2 for the moment.

If you try restoring one of the backups to your Windows 7 PC, does it restore as expected there?

Marco_Sassone · April 7, 2020, 11:32am

thank you @noris for your reply and routing my post to the right forum.

Absolutely yes, I never had more than 1 drive seen by Roon at one time.
My current situation is like in the below screenshot, and all my edits are back and no duplicates, so no problem there, I think.

So I suppose this is how it should be. That is, from the Storage window the Internal Storage folder structure is not visible.
And when I import music directly into ROON by dragging it over its window, it ends up in the “Roon Imports” folder, only viewable via LAN, as opposit as it was before with the external USB drive, where I could choose the folder where to store it.

In any case, my imports are done by syncing the ROCK storage with a ‘master’ drive I have attached to my iMac. The sync is carried with Chronosync. This practice has always worked.

I think I’m missing something here…
I assumed it would be irrelevant from which Roon control you instruct the ROCK to perform a restore, as this will be performed ONTO the ROCK database residing on the M2 SSD (in the case of my NUC).
Whether I issue the ‘restore’ from my iMac, iPad or other should not matter, as it’s the ROCK that has problems reading the backups that are attached to its USB port, right?
Besides, the Windows 7 Eee PC laptop I was referring to is something I resurrected (just in order to ‘chkdsk’ some NTFS external drives that started becoming unreadable once I power off my beloved OPPO BDP-93 they were attached to). It is sooo damn slow that I wouldn’t use it for anything else.

Please let me know if I am missing the bigger picture

noris · April 7, 2020, 6:42pm

Hi @Marco_Sassone,

That’s great news.

Yes, when you restore the ROCK database, it doesn’t matter which remote you restore it from, but my thinking here was to perform a bit of a different test, and this is to see if the backup restores properly on the Windows 7 machine when it’s acting as the Core.

That way we can find out if ROCK is having issues restoring any types of backup or if the issue is with the backup itself. To switch the Core to the Windows 7 PC you can use these instructions:

Open Roon on the other PC you wish to try as the Core
Roon Settings -> General
Disconnect
On the “Choose your Core” screen, press “Use this PC”
If asked to Unauthorize, you can go ahead and do so. You are limited to one active Roon Core at a time but you are free to switch between them as often as you’d like.
Verify if the same behavior with restoring occurs on the different PC

Marco_Sassone · April 8, 2020, 3:25am

Thank you @noris,
I followed your instructions (not so easy as the EeePC laptop is soo outdated and painfully slow it needed some work to make it run acceptably and once I installed Roon, it told me the screen resolution was too small to run so I had to connect the PC to a TV screen, etc…)
To cut it short:
On the EeePC desktop, there was already a copy of the ROON backups residing on the USB stick usually plugged into the NUC. I did the copy before exFAT formatting it with Windows 7, thinking the problem could have been because I formatted it from my iMac.

Well, ROON was able to see all the backups from the folder on the desktop, presenting the option Delete or Restore. I tried to Restore the last backup I did in the NUC and ROON started for a split second, then showed the same alert: “Restore Failed. Please Check Your Backup Folder and Try Again”.
Basically the same behavior of the ROCK (NUC).

I then tried to plug the USB stick (with exactly the same backups as in the desktop folder).
Windows was able to see the drive (E:) and the backup folder.
But ROON wasn’t! In the Find Backups -> Browse, the E: drive didn’t show up!
I tried to run chdsk E: /f but there was no problem reported.

This is beyond me.
Hope you could give me a solution on how to have backups ROON can actually restore.

At this point I think I could start afresh formatting the USB pen, as the old backups seem useless. Do you think I could try making a new backup from the NUC on the newly formatted USB stick and restore it straight away, basically to the same state, just in order to see if it works? Is there no risk I mess up something?

thanks

PS: I don’t know if this could be of help, but the only thing that struck me as strange was how long it took to copy the USB with around 10 backups on the iMac and on the PC too, which weights 4,77GB with 16.600 files in 16.400 folders …

noris · April 8, 2020, 3:47am

Hi @Marco_Sassone,

Thanks for giving that the old PC a try. Can you please archive a set of your Roon logs from the restore attempt on the EeePC using these instructions and send them to me via a shared Dropbox / Google Drive / Send.firefox.com link? I’d like to see if there are any clues in there before we proceed with further troubleshooting on the NUC.

Marco_Sassone · April 8, 2020, 8:45am

Hi @noris, I’ve just sent you the dropbox link in a message (PM)
thanks

noris · April 8, 2020, 5:20pm

Hi @Marco_Sassone,

Thanks for sending the log over. It looks like your PC was able to locate the backup and start it but it failed due to missing files from the backup. Can you please share a screenshot of the contents of the USB drive? This should have a _roon_backup_root_ and _roon_backup_ file like so, can you share the screenshot that lets us know if these files are present in the backup?

Marco_Sassone · April 8, 2020, 8:57pm

Hi @noris, these are two screenshots of the content of the USB stick (a Sandisk Cruzer 16GB).
I took 2 screenshots as there are 256 items in that folder and the roon_backup file shows at the end of my list (i have my Mac showing folders first).
Hope this helps…

noris · April 10, 2020, 4:25am

Hi @Marco_Sassone,

Thanks for the screenshot. It looks like all the files are present, so it is very curious why the backup is failing. Let’s do this:

Copy the current backup off the flash drive (save it to your desktop or somewhere else)
Format the flash drive as an exFAT drive (do not do a quick format when asked)
Plug the USB back into ROCK again
Create a new backup on the flash drive
Try restoring it on your Windows 7 PC again

Marco_Sassone · April 10, 2020, 3:09pm

Thank you @noris ,
the USB thumb’s behaviour was quite erratic when I tried to make a copy of it (mounting, not mounting, giving some errors, etc.).
So I formatted it (not quick, and it was no quick at all ) then performed a backup from the ROCK.
This time I turned the NUC off before unplugging the USB thumb.
I restored this last backup on the PC and this time went all right.
As it seemed all fine, I didn’t start and authorize Roon on the PC though, fearing to complicate things.
It looks that the solution was to format (instead quick-format) the USB thumb, as in one of my numerous attempts, I tried to format it, but it was ‘quick’ format.

I believe the USB thumb was corrupted because it happened on two thumbs, not just one.
Probably because I just used to unplug them from the NUC without turning it off beforehand? (and because there’s no ‘eject’ button on the ‘rock.local’ window in the browser).

Is it then good practice to turn off the NUC (rock) before unplugging external drives? If so, would it be worth including it in the ROCK instructions?

Can we consider it ‘case closed’?

best

noris · April 10, 2020, 5:58pm

Hi @Marco_Sassone,

Glad to hear that the backup worked as expected this time around!

Stopping RoonServer via the Web UI may also be enough before ejecting the drive.

It does sound like the backup went through as expected but without testing the database, it’ll be hard to say. Unauthroizing Cores shouldn’t be an issue, you are limited to one active Core at a time but you can switch between Cores as often as you’d like. If you really want to be thorough and verify that the backup is working as expected I would:

Try the database on the Windows 7 PC after authorizing it as the Core
Try setting the current ROCK database aside, start with a fresh one and restore the backup you crated on ROCK. Here’s how to do this:

Stop RoonServer from running in ROCK’s WebUI
Navigate to your Roon’s Database Location
Find the folder that says “RoonServer”
Rename the “RoonServer” folder to “RoonServer_old”
Restart the RoonServer in the WebUI to generate a new Roon database folder
Restore your database from the backup

If these tests work as expected, then yes, we can conclude that the backups are in fact intact and contain your database. If there are any issues, you can always rename Roon_old on the ROCK back to Roon and get your old database back (without the backup) if that makes sense.

Marco_Sassone · April 11, 2020, 2:46pm

thank you @noris,
for some reason, I didn’t receive the usual email alerting me of your reply, so I’ve just found it now when I went to check the thread.

Anyway, I skipped the step 1) PC restore, as the TV screen was busy (kid playing…) and went straight to step 2: restore from the backup on ROCK.
Everything went well: my Roon is up and running and all my edits are there.
Now what do I do with the “RoonServer_old” folder?
Shall I delete it and carry on or shall I reinstate the old situation ( Delete the new RoonServer folder and rename the RoonServer_old to RoonServer)?

Lastly, I re-read instructions here : https://kb.roonlabs.com/Migrating_To_ROCK
I initially started using Roon on my iMac as a Core, subsequently I migrated it to the NUC ROCK.
I can’t recall deleting the OSX Database.
I went now in the Library folder on my iMac and found a Database folder weighing 1,3 GB.
Turned off Roon Server Software from GUI, then renamed the Database folder.
Started Roon Server Software and found a new Database folder, now 13 KB only.
As I suppose this is the way it should be, as there’s no Core on my iMac anymore, I am going to delete the old Database folder in my iMac once for all (and regain space)
Can you confirm this is ok?

thank you
Marco

noris · April 12, 2020, 2:32am

Hi @Marco_Sassone,

That’s great news, this confirms that the backups are intact.

It’s up to you. If you continue using the backup for a few more days and don’t notice any issues you can remove the RoonServer_old folder. It’s a direct snapshot of your old database.

If the iMac is now just in Client mode and that database lives on ROCK, then there shouldn’t be any issue with doing a Roon_old on the iMac. It was 1.3GB because it contained the contents of your old database, with all the edits, artwork, ect.

It sounds like you renamed your ROCK database again? If this was before you did the backup it shouldn’t be 13 KB anymore, since you re-imported your Roon database contents.

Marco_Sassone · April 12, 2020, 3:06am

no no, forget the ROCK.
I meant this regards my iMac which I now use as a Remote. After I renamed the Database on the iMac, and restarted the Roon Server, it created a new 16KB database.
This new 16KB Database sits along with the old database, which I believe was the snapshot of Roon when it used to run as a core on the iMac (hence the 1.29GB size).
Is it safe to delete the Database_mac_old?.

PS: I still didn’t receive an email notifying me of your reply.
And yet the thread shows I’m Watching it and should receive notifications because I created the topic …

noris · April 13, 2020, 4:10pm

Hi @Marco_Sassone,

Yes, the old 1.29GB folder is very likely your old iMac database. If you have save the information into a backup and restored it to your ROCK, then you don’t need the old iMac database anymore and it’s safe to delete.

Are you seeing the message notification on the Community website when you open it up? Is it just the emails that are not working?

noris · April 13, 2020, 9:31pm

A post was merged into an existing topic: Cannot play to airport express group in sync

system · April 15, 2020, 9:31am

This topic was automatically closed 36 hours after the last reply. New replies are no longer allowed.