This could be treated as a roon software or feature discussion. Because of the errors it caused me, I preferred to list it in support.
This post is related to 2 recent posts,
Those posts are focused on specific problems. I thought the topic deserved a more comprehensive treatment.
Much of this is surmise. I’m new to roon and its a closed system. I’m confident enough in this analysis and the tests I have run to present this, but reader beware. Having some sort of “theory of operation” in the knowledge base would be illuminating.
For my purposes, I’m going to treat roon as having 3 scan types. (this may be incomplete, oversimplified, or incorrect. I’ll stop with the caveats, they apply to the whole post.) The 3 types are the initial scan, the rescan that users can force or schedule, and the on-demand individual album rescan.
Initial scan: the initial scan’s purpose is to populate the database. When roon was developed, some clever folk(s) realized that there were music industry databases with a great deal of information that could be linked to user music files. Perhaps the core of roon’s secret sauce is this linkage and how it is done (very well, IMO). User files can have many different file organizations and naming conventions, many errors, etc. A very clever algorithm was developed to use user library file and metadata information to associate user directories with albums (maybe roon can do clever things with really disorganized music collections of individual tracks, I dunno). A key ground rule is not to change the user filesystem. The end result is a mapping of a location on a user disk with an identifier for an album that can be linked into large music databases. Really that would be all roon would need to show albums, metadata, reviews; enable users to control playback; etc: the user has the data, roon knows what it is, and where to find it on disk. However, users are a prickly lot with lots of different preferences. roon’s setup ‘general’ and ‘library/import settings’ give the user some control about how information regarding their library is presented: do you want to see your genre’s or roon’s industry standard genres? etc. But other than user preferences, once roon has mapped an industry recognized album to a user file location, roon has all it needs (simplified first order). Initial scan has done its job. Takes a while and is computationally demanding.
Individual album scan: In the album view, under edit, a user can choose to rescan an album. This is not a bulk operation, its one by one, so not very helpful for rescanning many albums at a time. However, at least in my tests, this rescan is pretty thorough and seems comparable to the detailed initial scan. Certainly its more complete than rescan (below), since it picks up changes rescan does not.
Rescan: by default, rescans are run every 4 hours or when the server restarts. For a big collection, if done inefficiently, this could be a bottleneck and performance hit. My initial scan took a couple of days, so a scan comparable to that is not going to be repeated every 4 hours, nor should it need to be - the goal is mostly to check the locations, availability and major changes, not do a “from-scratch” analysis like initial scan. So rescan has short cuts. Its a bit harder since roon has committed to not writing on the user music disk. Heres my main point in this long-winded missive: because of the short cuts, as user data changes, roon’s database view will drift out of sync with whats actually on disk.
I have only been using roon for a month or so, but I have found 2 somewhat different cases with this same root cause:
- if you edit a music tracks metadata, roon may or may not notice. Major changes will be incorporated on the next rescan: for example, I changed the genre on a peter, paul and mary album from folk to rock. roon picked this up properly on rescan. But I changed my custom ROONALBUMTAG from “unrated” to “rating = 9”, and rescan failed to make this change.
If I do an individual album scan, the change is picked up, but if I had made a bulk change, I would have to rescan every album, or do a full library initialization, to get roon back in sync with my changes. - Under settings, if you change the “show hidden tracks and albums” setting from the initial setting when your library was incorporated into roon, a rescan does not pick this up. I suspect scanning every duplicate album individually would work, as would a full library rebuild. I showed that merely moving an album triggered rescan to pick up duplicates that it otherwise was missing, since the move likely triggered a more comprehensive scan, similar to individual album scan (which if done on an album and its duplicates would likely also work).
These are just the examples I happened upon. I can’t look at rescan and see what it does, and testing is tedious. Short of a full library rebuild, I know of no way for a user to initiate some sort of “deep scan” to find library changes not picked up by rescan. Perhaps most of the discrepencies between the user library and the roon view are minor, but maybe not. In my case, not seeing “duplicate” albums was a frustrating problem. There may be other user issues in the forum whose root cause is the incomplete rescan.
Possible ways to help this problem:
- knowledge base article explaining scan better, especially any known limitations of rescan.
- a user-accessible control enabling occasional user “deep scan” to ensure library/roon sync. this would be more than rescan, but need not repeat all the analysis of the initial scan. Changes generally will not require re-identification in the industry databases. Even a large library could probably be “deep sync’d” overnight.
- a mod to rescan to do a “deep scan” on a portion of the library on each rescan, so over multiple rescans all changes get picked up.