Mass Deletion of Duplicates and Partial Duplicates

Support Metadata
22

Hi @Jeffrey_Robbins — Apology for the delay on the response here!

We did some testing on our end with this media and we weren’t able to get this behavior to happen to us — The duplicates were grouped appropriately as versions and hidden with the correct setting.

Moving forward, we are hoping you can try something that should help us better understand what you’re experiencing. Please see the steps outline below:

  • Make a copy of all of the Toys in the Attic albums somewhere outside of your watched folder.
  • In Roon, delete all of these albums. It is important that you have the copies saved outside of your watched folder because this will delete the ones in your watched folder from your drive.
  • Reboot your Core.
  • Move the copied files back into your Roon watched folder.
  • After they all import, share a screenshot of how things look. Are they now grouped together? Do you see the same issue?
23

Hi @dylan. Result is that a single version of the album appears and all versions are contained within. So, removing from watched folder, deleting, rebooting, moving the files back to the watched folder and allowing time for a scan produced the desired result. Excellent!

But, I have 30+Tb of music on my NAS and cannot possibly go through this process for each album. For starters, I don’t have a second NAS to hold that much data. So, what do we do to get this to work in situ? JCR

24

Thanks for confirming this worked for you, @Jeffrey_Robbins.

I’m going to need to speak with the team about our next steps here. One option here would be to start with a completely fresh import, but in doing so you would lost your Roon-specific data — Playlists made in Roon, history, etc. If you’re mostly interested in getting things up and running and you aren’t worried about that data, then we could move forward with that. Alternatively I can follow up with you after discussing with the team further and we can give you some alternate next steps.

25

Yes, talk to the team. I really don’t want to lose all my Roon-specific stuff. Thanks. JCR

26

@dylan, just checking in on whether the team has some ideas of what I should do, short of manually lolonkong each album back together. Thanks. JCR

27

Hi @Jeffrey_Robbins,

Apologies for the delay here. I met with the team to discuss this and have some next steps.

First, I wanted to verify — Do you have any backups of Roon from before you made changes to the watched folders?

My understanding is that this specific issue didn’t exist prior to making those changes. There were duplicates, but not the same as what we’re seeing here, correct?

Next, the team was hoping you could start a fresh Roon database, at least temporarily, and import all media fresh. This will allow us to verify that, as with the Toys In The Attic example, things do import fine when in a fresh state. If things don’t import fine and you’re seeing the same issue, there may be problems with the specific files or even the database location, so this would be a good test in helping us narrow that down.

To start with a fresh database, please do the following:

  • Make a Backup of your current Roon Database
  • Close Roon
  • Navigate to your Roon’s Database Location
  • Find the folder that says “Roon”
  • Rename the “Roon” folder to “Roon_old”
  • Restart/Reinstall the Roon App to generate a new Roon folder
28

I do backup the Roon database to my NAS each night at 2:00a. But, unfortunately, it is set only to save the prior 10 backups, so 11/2 is the earliest one left — and that’s after we did the watched folder change up.

Now, I can certainly startup a new database run as you note. I assume it will take 24-48 hours or more to complete? Might there be a way to extract my playlists, history, etc. from a current backup and merge it into the new database run once complete? Or can you think of a way to recover the Roon backups that are older (I’m assuming not, but thought I’d ask). Thanks. JCR

29

Well, @dylan, it went quite fast. But, the results are most unsatisfying, to wit: we lost a lot of tracks on the creation of a new database:

Old Roon Database:
Albums 33,669
Tracks 211,391
Artists 5,863
Lyrics 97,454
Reviews 37,262
Images 44,888

Newly Created Roon Database:
Albums 21,334
Tracks 71,239
Artists 3,866
Lyrics 37,083
Reviews: 25,809
Images 31,686

Taking our Toys In The Attic example, and focusing on albums -mp3, the only version that appears is the 1993 SBM Remaster (and the TIDAL version under it). Gone are all of the other versions.

Since we do have a backup and where we are at, I’m clearly not going to keep, I’m going to do a forced rescan of the watched folder. Currently, it shows 57,313 tracks imported. I will report back. JCR

Update: the scan was over 2.5MM files, but found not a single additional track. I’m now going to do a backup of this new database, save this new Roon folder as old2 and restore my original, at least until you next weigh in. Thanks. JCR

Second update: everything is back to normal, as in where it was before we started – and the only album grouped together seems to be the Toys In The Attic that we removed and reimported. Your sage advice next, then? JCR

30

Hi @Jeffrey_Robbins,

When you created the new database that had fewer albums, what was the watched folder path that you added?

31

Same as before — just the main J:/ folder on my NAS. JCR

32

Hi @Jeffrey_Robbins,

Did you completely remove the files from Settings > Storage or did you just disable them? Are they still showing in Settings > Storage? If so can you share an updated screenshot?

33

Not sure of your question. I did nothing to files. I.followed your prior instructions to rename my RoonServer folder on my core server to _old and then, after killing and restarting RoonServer on the core server, I ran a whole new scan on the same folder on my NAS — J:/. And you know the results were not good. So, I changed the name of the new RoonServer folder on the NAS to _old2, then changed the name back of _old to eliminate that suffix and got back to where we started without incident.

No where in there did I “remove” files or folders. So, I am unclear what you are asking. JCR

34

Meanwhile, I decided to try a fresh database again and see what happens. The result is amazingly similar to last time:

First Newly Created Roon Database:
Albums 21,334
Tracks 71,239
Artists 3,866
Lyrics 37,083
Reviews: 25,809
Images 31,686
Tracks Imported 57,313

Second Newly Created Roon Database:
Newly Created Roon Database:
Albums 21,333
Tracks 71,314
Artists 3,867
Lyrics 37,387
Reviews: 26,006
Images 31,702
Tracks Imported 57,313

I’m not going to save this second database, as it is obviously coming out the same as the first time – but very much missing a lot of albums and tracks. That’s a whole different problem, I surmise.

If your question is what did I do with my music folder in Storage/Settings, I disabled it each time I ran the database anew, figuring you would want me only to run the database against the single parent folder. JCR

35

Hi @Jeffrey_Robbins,

Apologies, I should have been more clear here — By this I meant back when we first started discussing your original issue. You had the following watched folders:

image
image.jpg727×647 73.4 KB

You disabled all of the watched folders except for [J:] Media — You can either disable folders or remove them entirely, and I was just curious whether or not they were disabled or removed.

On your existing database, can you share a screenshot of this page once more?

36

Settings%20Storage
Settings Storage.jpg796×699 55.5 KB

Here you go. When I started a brand new database, it was the same, except that the imported tracks was the 57XXX number instead of the 199XXX number here.

Well, @dylan, we seem to be in an odd spot. The original database has all my albums, but won’t group them together. And twice now, a start over rescan cuts substantially short the album count. The next move? Thanks. JCR

37

Hi @Jeffrey_Robbins,

Can you send us the original database? Ideally you can zip up the entire Roon folder and share it in a in a PM via a shared Dropbox link. Once we have it I can pass it along to the team to take a look.

38

Will do. So as not to repeat last time, where the share link was posted in the forum here, to what address do I share the link privately with you and the team? JCR

39

Send it via a Private Message to Dylan - click on his Avatar icon, and then on the blue Message button to create a PM to him.

40

OK, done. Thanks. JCR

41

I’m fascinated to find out what happened here…