B1207 After database update library maintenance is reporting missing files [Ticket in]

Hey @Early_Access
So my system just updated to 1207, and took ages to update the database. It is working now but Library Clean-up shows this;

It didnt show that before.

My album count has also dropped.

How do I identify the files (presumably Qobuz titles?) that are now missing?

1 Like

I just checked here - 369 files to be deleted when the figure was zero before the update. Hmm.

Michael

1 Like

I’m at a cool 147 to be wiped.
If it is garbage that is newly found. Yay thank you.
If not. Not as much thank you.

I like the idea of being able to verify which 147 files myself as well.

1 Like

Hi, @Michael_Curtis. @Tim_Rhodes, @MamaTried, thank you for your reports! We are investigating this case internally at the moment, please, do not clean up these files. I will post an update here when investigation is done.

Thanks!

–
Ivan

3 Likes

Argh! Too late! I cleaned them up yesterday, although I do have backups from before this update that I could restore from…

I have left it. I’ve spent far too long curating the details and tagging my files to randomly whack 147 unnamed files.

Indeed. I have had this message for a longer time, not sure how long, for 492 files. That may be Qobuz deletions or whatever, but I have never dared clicking the Clean Up Library button. There should be a List Files button so that I can check before deleting anything. Even if it’s inevitable Qobuz deletions I would like to know what they are

1 Like

I skipped 1207 and went to 1208. Before updating 262 missing, after updating and without doing anything to the library, 394 missing. I agree with posts above, a list of what actually is considered to be missing would be very handy. My bet is Qobuz pulled and re-entered releases, but I will not touch the button to actually delete :grin:

1 Like

Certainly I have no local music files missing as I compared what I have on the NUC with the two backups I have here. Hard to say if Qobuz titles are now missing though. I did an export of Qobuz titles quite some months back so although that wouldn’t be up to date it may give me pointers as to some of the albums that may be missing.
In the past on the few occasions I’ve physically deleted files I’ve noticed that in library cleanup there is one file per track that needs deleting, so I’m guessing I may have thirty or so Qobuz albums missing. I do have database backups from before the update although I don’t know if restoring would help at all.
It’s not the end of the world - wish I hadn’t run cleanup but it’s not worth worrying about.

Michael

So now I have 1082 suddenly and I obviously won’t click the Clean up button now as long as this issue isn’t clarified. However, even without the current uncertainty I would never want to click that.

Even more now :slight_smile:

I don’t know if this is helpful to contribute, or, not.

I’ve never dared touch the ā€˜Clean Up Library’ button, so don’t know what was there, previously, like there’s, but it does seem that there’s a lot of files previously indexed that I’m rather surprised about.

I seem to recall ā€˜starting again’ with my Roon library very early on, which may account for the rather large amount of files Roon is reporting below

I don’t know if it’s related, but the reported numbers changed after I had to restore from last night’s backup because I inadvertently deleted an album of local files (with edits. By the way I am pretty sure that Roon showed me a streaming version and I removed this from the library, but then it actually removed the local version). The files were, however, still there on the storage drive after this.

After the restore, it is now reporting 708 deleted files as opposed to the 1082 from 2 days ago (see a few posts up).

More concerning, it reports 3286 files not associated with a storage location, and the Home screen indeed shows 37860 tracks instead of the normal 41.146 (or so, definitely above 41K) . That wasn’t there 2 days ago (again, see above)

However, this is on a ROCK that only ever had the one internal storage location that it is still using, and as far as I recall it shows the same number of files it always did. (I thought I posted a screenshot recently somewhere but can’t find it)

Turns out that after the backup restore it failed to auto-login to Tidal, causing the 3287 files ā€œnot associated with a storage locationā€.

The ā€œdeletedā€ files changed to 742 now, the third different number after 708 yesterday and 1082 3 days ago :slight_smile:

1 Like

Hey @ivan

I did as you asked and did not clean up my library to retain the 437 ā€˜deleted files’

However yesterday 1212 was installed on to my core and remotes and the picture has now changed;

So now the number of deleted files has reduced a lot, but is still non-zero (it was zero before all this happened).

Any ideas as to what is going on here?

Hi, @Tim_Rhodes. The problem with deleted file is that Qobuz sometimes pulls down albums completely due to different reasons or pulls down and edition of an album and then puts back another version of this album but it’s album id changes. In the latter case such album will still be marked as favourite on Qobuz but a link to it will lead to a 404 Not Found page.

We will still show such album but none of the tracks will be available since. Here’s the example from my library:



So, when you first reported this problem deleted files appeared because Roon was not longer showing album which exhibited this behaviour. The fix was to restore this behaviour and make it align with production #1202 which also had a problem with that part but it exhibited itself in a bit different way. To sum up, the ā€œgood newsā€ here is that deleted files are not something which is still available and Roon randomly removed it but rather stuff that Qobuz pulled down and we cannot really control that part :frowning:

Thanks!

–
Ivan

1 Like

Thanks Ivan…

Is there a simple workaround to finding these albums?

I suppose the missing piece is how to identify the albums thats have been removed/relisted by Qobuz (or TIDAL etc)? There ought to be a way either with an exportable list on the Library Maintenance page or by being able to focus on ā€˜Unavailable’ tracks/albums.

2 Likes

I agree with that, I will raise this topic locally with our product team:

Thanks!

–
Ivan

2 Likes

@ivan - we’ve been asking for this for years…

https://community.roonlabs.com/search?q=unavailable%20%23feedback%3Afeature-suggestions

2 Likes

As per the UI, the files could have been deleted in Roon, in the file system, or by the streaming service. So this could be accidental deletions of local files as well, not just Qobuz removals.

Therefore, it would be helpful (for all 3 categories) if we could export a list before clicking cleanup and losing all information of what these files were.

1 Like

@ivan this would be a great first step, and has been requested before.

I would hope it doesn’t stop there, and that Roon can work out something with Qobuz to alert users when withdrawals or substitutions happen or have happened, with options to replace the removed version with a replacement where such exist. My experience is that the bulk of these dangling pointers amount to a very straightforward substitution, often amounting to a difference in bit- or sampe rate only, if even that.