Clean Up Library should list the "deleted" files it proposes to remove

Feedback Feature Suggestions
44

“The Roon software absolutely knows which files are slated for deletion, otherwise it wouldn’t know which entries needed to be deleted from the database.”… I never said otherwise. Roon, as it clearly indicates under Library Cleanup, indexes every single entry you make in Roon. In all likelihood, it’s via something like a hexadecimal number, as in hex, you can index over one million entries with just five hexadecimal digits. Neither you, nor I designed the Roon software, so neither of us knows for sure what happens…it’s pure conjecture. But the logic of efficiency leads me to suspect that when you add something to Roon, say “Money” by Pink Floyd, it adds both the file information “Pink Floyd” + “Dark Side of the Moon” + “Money” plus “index number” (Let’s say 1a2b3) to create an entry. Now, when you delete it, for efficiency’s sake, it makes sense that when “Money” is deleted that everything except the index number 1a2b3, and possible amending that number with a 1,2 or 3 (the cleanup group). Does Roon do it that way?? Who knows, but from an efficiency standpoint, it makes 100% sense. You can track that entry from it’s inception to it’s ultimate demise with just with a short index number.

another reason why a system like mine makes sense, take this example… I delete all 44.1/16 entries as I have decided to go all hi-res. Let’s say that’s 2500 entries. Once deleted, I do a backup. I know from experience, that the database does not have the file information, as I have had to restore from that backup (I auto backup every midnight, and 2-3 times have had to do a restore early in the morning), and it did not put the songs back into my library. But, the first group of Library Cleanup items shows 2500. The only logical way that I can think of for that to happen is via something similar to what I suggest, i.e., only the index number survives deletion. Once I delete those 2500 through Library Cleanup, then the 2500 indices are lost. I wish a Roon software designer would explain the mechanics of this for everyone, but I bet my way doesn’t vary too far from theirs.

45

I THINK I have come up with a simple solution. What if Roon dumped all that info into a Roon folder on your hard drive, say CLNUP1, CLNUP2 and CLNUP3, each folder corresponding to the same numbered Library Cleanup group. So, if you delete 15 songs via Roon, it goes into CLNUP1. You disable your Christmas folder, the tracks show up in CLNUP3. Does this make sense?? As all this would be on your hard drive, it will not, if I’m correct, affect how efficiently Roon servers run, and any performance hits would not fall on everyone.

46

Now I’m beginning to wonder if you even understand what this feature request actually is. Please, you might wish to re-read this thread before commenting further.

47

I think I understand the thread far more than you do, and why it is at least impractical, and probably impossible.

48

OK, Neil, here’s a simple question: In the screen shot above from mikeb, how does Roon know that there are 41 database entries that need to be cleaned up?

50

By the indices of those entries!

51

Then logically disprove anything I have said. YOU CAN’T!!

52

Fine. Displaying a number in hexadecimal does not in any way influence the amount of space that number consumes in the underlying storage. The number 255 in decimal is FF in hexadecimal and 11111111 in binary. They’re all precisely one byte in size. That’s a fact and it doesn’t even require logic to disprove.

But, beyond that, what do you suppose is contained in the “index of that entry” in the Roon database? It necessarily includes the track name, artist name, file location (or reference to the track entry on Qobuz or Tidal), and all the other metadata associated with that track which used to exist on disk but no longer does.

Roon has a database in the core which references a file which no longer exists on disk. In this case, there are 41 such database entries. Currently the Roon app displays only the total number of such entries.

This request is to allow users to see the other metadata from those entires – the song names or the full path to those files which have disappeared.

How would this be “probably impossible” to accomplish?

54

Wrong. The amount of space to catalog index, title, album, artist, etc. does exceed that needed by the index alone, by a significant amount, and is needed only for display purposes prior to deletion, etc. Where did I compare binary to hexadecimal?? I was comparing the amount of space needed to hold the index to the whole kit and caboodle. And, again wrong. the Index does not necessarily include 'track name, artist name, file location" etc. The index IS the file location. All the other information is only needed for display purposes. Once a file is deleted, Roon no longer needs any information but the file location / index. Anything more than that will slow Roon down. The most sensible thing for Roon to use after deltions have happened in your 41 entries in the index only. That can tell Roon exactly where each of those 41 entries is located. It does not need any of the other information to accomplish this. I understand that some would like Roon to keep all this information, but to do so would make Roon bloatware.

55

Right, and you implied that using hexadecimal is a valid way to reduce the size that an index requires. That’s patently false. But it’s not your most glaring misunderstanding.

As Mike rightly observes, if a missing file somehow reappears, Roon clearly has not lost any of the metadata associated with that track. You can try this for yourself if you don’t believe us. Until the user performs the clean up operation, the database entry for the affected tracks are still un-removed, un-abridged, and exist in their full glory in the Roon database.

Currently the Roon app merely shows the user a count of the number of these entries. In the example above there are 41 database entires – containing the “whole kit and caboodle” in the database. But all we can see is how many of these stale database entries exist.

The feature request is to allow the user to see the other information about these database entries, in order to assist in diagnosing the reason why the actual file (or link to streaming track) is broken.

This is false because this situation can arise for reasons other than a file’s deletion. It can be a misconfiguration or permissions failure on a linked NAS device where the file lives. In this circumstance, the remedy should be to repair the NAS and not delete the entry in the Roon database. However currently it is not possible to know if this is the situation because the Roon UI doesn’t expose the details.

Similarly, the stale database entry might be because that track has been removed from Qobuz or Tidal. In this case, the user might wish to obtain that album or song from another source rather than just having that track silently disappear from the library and playlists. Again, this is not possible because Roon does not provide any details about which tracks are affected.

Once the user performs the “Clean up Library” process, the entry is removed entirely. Metadata and any index. Everything. There is no step in this process where the information about the track is removed yet, curiously, an “index” of the record is kept. There would be no value to that to anyone.

It’s really not that complicated and I honestly struggle to see how you’ve so completely misunderstood what we’re saying. How can I explain this better so that you can understand?

56

“I’m no Roon programmer (obviously), but the database entry is definitely kept after the file is lost - because when the file re-appears, possibly with a different path, the associated data is immediately viewable again.” … In these cases, the file is probably not lost, but in all likelihood, re-indexed. If I move files from folder A to folder B, I expect a re-indexing. And, yes, it would reappear shortly. If a file is lost or deleted, I would expect the counter to be incremented under my hypothesis of how I think it works. Under it, two states exist. Index with data, and index without data. A lost or deleted file would change the state from state 1 to state 2, but the index remains the same. Clean Library recognizes that a location now has no data, and increments the counter. Now, I have indicated before that this is all hypothesis, as I’m not on the Roon software team. But, I have been into computers since 1969, being a computer science student at UNC. We no longer do poke and peek for memory usage, but using my 51+ years in computers and sound logic, my hypothesis makes more sense that anything else I have seen here to the contrary. Could I be wrong? Dang straight, but if Roon isn’t more aligned with my thinking that y’allses, it should be. The goal of Roon isn’t to be everything to everyone, but to maximize the experience for as many as possible. And to accomplish the goal you guys want, in my educated opinion, would be counter productive to Roon. And make the experience so painful, that a lifetime member like me might abandon ship.

57

There’s no need to compare resumes and I think we’d all do well by not assuming where ours ranks among the others in this thread. This is not a complicated feature request and is clearly possible to accomplish within the existing framework of the Roon database.

Again, this is really very simple:

  1. How does Roon know that a track in the database needs to be cleaned?
  2. What does Roon know about a track in the database that needs to be cleaned?
  3. Is there information about a track in the database that needs to be cleaned which Roon knows but does not currently display to the user?
  4. What is removed from the database once a user cleans it?

Answer those four questions and I think you’ll be aligned with the thread. This request relates only to item 3.

Please believe me when I say: If you think this then you are misunderstanding the request. Truly and honestly. I don’t know how to say it more clearly.

58

Sorry, but logic is missing from your argument. First, I never implied that hexadecimal was a way to reduce the size of the index. I just used hexadecimal is an example as it has been used in computers since I started taking computer science in 1969. That in no way implies that hexadecimal is the only way, or even the best way, or doing it. If a file magically reappears, it was never “lost”, but just reindexed to express it’s new location.

“Once a file is deleted, Roon no longer needs any information but the file location / index”…Huh?? Nothing you said after that has one iota to do with my comment. I never said or implied anything to do with misconfigurations or permissions. I said DELETED files. Regarding Qobuz and Tidal files disappearing, through 1.7, such files showed “unavailable” in the tracks list. Whether there is a better way to do that, I don’t know. That is why I didn’t broach upon that. Without knowing all about the inter-platform intricacies, it’s anybody’s guess on that.

“Once the user performs the “Clean up Library” process, the entry is removed entirely. Metadata and any index. Everything. There is no step in this process where the information about the track is removed yet, curiously, an “index” of the record is kept. There would be no value to that to anyone.”…Excuse me?? Virtually everyone would benefit, as Roon would run slower having to retain all that information than if it only has to retain the index (file location). Again, all I have said pertains to DELETED files. Not once have I said it implies to group 2 or 3.

59

Which of this statements do you think is incorrect? Speaking of the screenshot from mikeb above:

  1. Roon is aware of 41 songs in its database which reference files which the server can no longer find on disk
  2. Roon’s database contains all the usual metadata about those 41 songs: artist, track name, file location, etc.
  3. The “Clean Library” function informs the user that 41 such songs exist in the database.
  4. If the user chooses to clean the library, those 41 database records will be deleted.
  5. If the user has not yet run “clean library” and 1 of those files somehow re-appears on disk (because its absence was transient or the result of a misconfiguration) that song will no longer need to be cleaned and the corresponding count will drop from 41 to 40.

Do you agree that all of these statements are accurate?

If yes – then this feature request is simply to amend number 3 and allow the user to inspect the entire database entry for those 41 songs and not just be told how many of them there are. There is no performance impact to this change, it is simply an addition to the user interface.

60

  1. How does Roon know that a track in the database needs to be cleaned?..By an index number without any attached data, in all likelihood IF the tracks are in the first group.

  2. What does Roon know about a track in the database that needs to be cleaned?.. if it’s group 1, hopefully NOTHING except it’s index / location. Anything else is BLOAT!!

  3. Sometimes. My Christmas tracks only show from 12/1 thru 12/26, as they reside on a dedicated drive disabled the rest of the year.

  4. All data plus associated indices.

Remember, the more data a software has to keep track of, the slower it gets. Definitely a case of less being more performance.

61
  1. I bet this is wrong if you deleted the song via Roon, in which case, I bet only the index survives, (Group 1)
  2. With group 1, I bet the index only remains in the database.
  3. We know the indices will be deleted. anything else is conjecture.
  4. Only applies to group 2 & 3. A deleted song will not miraculously reappear.
62

This cannot be correct and it’s easy to test if you don’t believe me.

Roon stores quite a bit of information about a track in the database. You’re able to override the track name, artist name, your metadata preferences, and add tags to each track in the database. These edits that you perform are stored in the Roon database, they’re not contained in the underlying file.

Imagine you have a track in your Roon database and you fix a misspelling in the track name, and you make a few “Prefer file” or “Prefer Roon” adjustments to the metadata preferences. And you add a few tags. All that is stored in the Roon database.

Now, delete the file from disk or move it to another location that Roon cannot see. I don’t mean deleting the track from Roon, but the actual file on disk.

This is now a entry that needs to be cleaned. Roon will report that there’s 1 song that needs to be cleaned.

Your view is that at this point, there is nothing in the Roon database except an index number and the full path to the file, right?

Now undelete the file, or move it back into place. Guess what? All of those edits and tags are still there. Why? Because they exist in the Roon database. This is why the Clean up Library feature exists. Because you’re right, all that data takes up space in the database and if we know for sure that the file is never coming back then it’s best to delete the entry in the database.

But sometimes that file is missing for other reasons. And we do expect it can come back (like your Christmas songs). Or if the file is missing because of a technical problem that can be resolved (like a NAS that’s offline). In those cases we absolutely do not want to clean those entires from the library because we’d lose play counts and tags and metadata edits. If we clean those entries and then the file comes back, we would experience the loss of that data.

Because of this, it would be really great if Roon would provide more information about what songs it is seeing. Because that information could lead to finding and correcting whatever problem has led to the file not being visible to Roon where it expects the file to live.

Do you understand now?

63

I think perhaps I didn’t explain this clearly before. If you delete the song via Roon there is no reason for an index to survive because the index contains nothing of value. That’s what I meant when I said there was no value to the user to have an index without any of the other data about a track.

If you delete a track via Roon I’m sure it deletes everything including any index. There’s no advantage to keeping the index in that circumstance.

This is not true. A “deleted” song might not actually be deleted. It might be on a USB drive that has lost power. Or it might be on a NAS but the file sharing permissions have been changed and Roon can no longer read the directory on that mounted volume. Or the operating system might have a new bug that gets confused when it encounters filenames with umlauts. There are a near infinite number of reasons why Roon might no longer be able to see a file it expects to see which are transient problems outside of Roon’s control and can be remedied by the user, restoring Roon’s ability to see the file.

64

If it reappears, it’s either group 2 or group 3. Group 1 won’t reappear, with the possible exception if Tidal or Qobuz puts a song back into their system with the exact pointers all the same. I have never seen this happen, but I can’t say that’s impossible. Many times I have deleted songs from my library, and group 1 has correspondingly increased.

65

This is not correct but I’ve run out of ways to explain it. Best of luck to you.