Deeper Library Clean

Recently I stored the RoonBackup on the same external drive that holds my library files. It meant Roon took forever to refresh the database as there were the 100s of thousands of backup folders present. I’ve since deleted the backup files but think it may still be lingering as: i’ve only got 80,000 tracks on file but the latest rescan is up to a count of 220,000 files assessed and is taking ages to scan. The standard cleanup area is showing zero items to be clean out though.

So am I right that the old backup data is still playing havoc, and is there a way to clean it properly?

WOW, What that? Like 8,000 albums!!! Wonder where the duplication is from? How many albums do you have saved from streaming services?

No streaming services. It’s perplexing me.

What’s your Core running on? Is it a ROCK NUC?

I’ve had a similar issue occur twice in the past couple of years on my ROCK NUC. The Internal Storage folder hierarchy somehow gets recursively repeated in an endless succession, producing thousands of ghost mirrors of my library. See here:

Ah ok. I’ve recently moved it to the work desktop (form mac to PC). Not sure if that has anything to do with it.

Did you get a solution form @support? Or you just reinstalled and it hasn’t happened since the first 2 times?

I’ve passed on the logs to Support for analysis. It’s only happened the two times, once last December, and once last month. Whatever it is, it must be rare.

Ah ok, yes still recent. It’s all above my head so i’ll just do a rain dance and see what falls out. =)

Hello @Hugo_Sharp,

Thanks for contacting us regarding this issue. I would agree with your theory that the old backup could be the one causing the issues here, have you removed the backup from your watched directories now?

What platform are you on currently and which platform did you experience this issue with? You mention that you recently moved from Mac to PC, but were these hosting the Roon Core or were they just Roon Remotes?

The issue that Geoff mentioned only appeared on ROCK from what I’ve seen and not on Windows/OSX so I’m thinking this behavior is just due to the backup files starting to be scanned but never finished scanning. Have you tried rebooting your Core and the behavior is still the same?

Thanks,
Noris

Hi @noris,

Yes the backup has been removed from the watched directory (external drive).

I run core and player as the same program while at work. Recently i moved that core from mac to windows. I can’t recall if i had the issue on mac also. I think i may have but that it was mush faster so i never saw it as a real issue. Everything has been slower on windows (see other post linked below) so I think it’s since moving to windows that it’s become a problem.

Are you asking if i’ve rebooted the machine hosting the core? Yes i’ve done that and no effect.

Additionally, i’ve been trying to restore the backup from the work PC on my server at home and it’s took 14hrs to do 25%. At that point i had to use Roon at work and revoked the licence. Could this issue have been transferred to the backup and now making restores very slow? Just deleting backups in explorer takes 2 hours.

Also, is it potentially related to this recent issue i’ve had? Editing Folder Structure in Windows

Thanks
Hugo

Hi @Hugo_Sharp,

Thanks for providing that information. I would like to take a look a closer look at the diagnostics from your Roon devices to see if there is more info that we can gather, but before I enable this feature for your account I’d like to make sure I have the appropriate information, can you please confirm:

  1. The issue with the adding 200k tracks to library was on the PC at work, correct? Can you confirm the exact model of this Core and approximately when you experienced this behavior (date and time)?

  2. The current Core that you are using at home is a Core/Streamer combo? What is the exact model of this Core/Streamer combo?

  3. Do you recall approximately when you tried to restore the backup to your server at home (date/time)?

  4. How did you transfer the backups from the work PC to the home server? Are you using a USB drive, an external hard drive or Dropbox? Was the backup created directly to the flash drive or was it created on the PC and then copied over?

Please let me know the above when possible and then I can go ahead and enable this feature.

Thanks,
Noris

Thanks @noris,

  1. It’s been a noticeable issue since the switch to the work PC (around the 19th March). It’s an intel i7-7700 cpu (MEC95).
  2. At home it’s an Innuos Zenith II.
  3. I’ve been trying many times to restore to the Zenith at home. Most have but 2 have worked. I don’t know the dates though.
    I’ve been using dropbox backups created directly to dropbox.

Thanks
Hugo

@noris it’s all really falling apart. Whole albums have lost their link to the original files. They still have the correct path to file and the files are there but playback fails. So i just can’t play that album as it won’t re-import it. Also i can’t delete the corrupted album, the best i can do is hide it.
Other albums have this issue but for just a few tracks so i get multiple entries for the album. Again i can’t delete the errored ones, just hide them.

Hey Hugo,

Have you spoken about this issue with Innuous? We’ve seen a few issues related to backups on these devices and their team has been quite helpful, and should have more insight than us since we don’t have access to this gear, unfortunately.

Their support team will be able to figure out what’s going on here, but one thing I’d recommend is using the Dropbox desktop app to download the backup archive to a local drive, since the Dropbox API can be a bit slow – things will go a lot faster if you have the archive saved locally as you troubleshoot this issue.

If you’re stuck, definitely let us know and @noris and the team can help, but I think Innous should be able to expedite this. Let us know how it goes, and sorry for the trouble here!

Thanks @Mike sorry to clarify, my issues are with the core on a PC (from a Mac), not the Innuos.
Side note: Main library is on the PC and then i restore backups weekly to the Innuos so i have the same metadata at work (PC) and home (Innuos).

I haven’t yet restored the PC core from a backup of itself. I was waiting for @noris to gather data. Should i do this now?

Hi @Hugo_Sharp,

You can feel free to go ahead with the restore from the PC, I have gathered the diagnostics needed. Please let us know how the restore goes.

Thanks,
Noris

Hi @noris I restored the db (deleted db and then restored from a fresh backup). Hasn’t changed anything though. Still many hundreds of thousands of “tracks” being reviewed each library scan (when only 80,000 real tracks exist).
Also some albums say they can’t be found but the folder and flacs are still there and named the same. Nothing i do rectifies and i can’t even delete the entry in Roon. I’ve had to hide many many albums and files now as i can’t delete anything that has lost its link to the flacs.

Where to next? This is a real problem that’s making Roon unusable.

Thanks

image

Still. It gets a lot higher. 89,000 tracks actually there. I can’t rename any album folders on the external hard drive otherwise Roon now looses them forever.

@support so how do i purge these phantom entries that are slowing down the library scans and stopping me functionally managing my library?

Thanks

Hi @Hugo_Sharp,

Just to make sure I understand you correctly here, this issue is occurring for your Work PC on MEC95, correct? It sounds like something might still be wrong with your database here, how have you cleared it out the last time?

To properly clear out an old database and remove any possible sources of this issue I would advise using these steps:

  • Exit out of Roon
  • Navigate to your Roon’s Database Location: https://kb.roonlabs.com/Database_Location
  • Find the folder that says “Roon”
  • Rename the “Roon” folder to “Roon_old”
  • Restart/Reinstall the Roon App to generate a new Roon folder
  • Restore your database from a backup before this issue started

Can you also confirm that none of your other cores exhibit this behavior, neither the Mac nor the Innuous?

Thanks,
Noris