Over the past 18 months, my database has grown to ~26GB. I have 7200 albums. It’s working fine and I back it up routinely to two locations.
Does the backup routine run db maintenance and/or compaction to remove white space or can this be done manually to ensure the db is a lean as possible?
I do run the Library cleanup every few weeks; mostly to remove Tidal albums/track that become unavailable…
I would recommend not touching any of the database folders as that can cause issues and possibly lead to corruption. The database size may vary depending on how many edits you have made, and if it’s not causing any issues other than taking up additional storage space I would leave it as is and let Roon perform the backups as normal.
No intention of touching the db folders. I was just curious if any maintenance routines are scheduled, or should we just trust the db is always going to be OK?
In my experience as a 30 year IT veteran, databases go sideways for numerous reasons and they all require maintenance. I do keep two separate backup locations and have restored for new hardware builds several times without issue.
Yup, this is what I would recommend. We’re always seeking out higher quality artist photos and album covers, so growing database isn’t too surprising or unexpected here.
The library maintenance feature is more targeted at use cases like “i’ve deleted 3000 old MP3 albums, and I don’t want my database tracking content I no longer own”.
In short, it’s not something you need to do periodically.
I suspect doing an export and re-importing to an empty folder would do the trick. Yes I agree - 30 years in IT for me too, and databases always need maintenance. There should be some kind of compacting routine. I’d say if we look under the hood, it’ll be some standard DB format, and that DB will have some standard tools - because they need that and are readily available. What I’m not sure is why Roon are so scared of making that accessible through the GUI.
Roon’s database makes me nervous, for two reasons. Firstly, because there isn’t a way to check the integrity of the current database. Secondly, because if there is a problem it’s also probably being replicated when the database is backed up (at least I assume that’s the case).
My ‘solution’, such as it is, is to export and then restore every month or so. At least then I know that the latest backup is viable should I need it.
Looking at Roon’s credits, the underlying database technology appears to use LevelDB. This was written by engineers at Google and inspired by their proprietary Bigtable database. Not saying that things can’t go wrong, but this is a robust system.
That said, checking recoverability of backups before you need them is always a smart plan.
Given your point about the robustness of the technology my paranoia is probably unjustified, but I’m happier when I check than when I don’t
I have ZFS snapshots running, so that basically covers it. Backups are there as well.
Unfortunately, Roon’s DB system does not seem to be as robust as you might expect. Many users report issues related to Roon being suddenly unable to load the DB (sometimes, a new build is directly causing the issue). Roon support’s response consists of blaming your hardware for corrupting the DB and suggesting to restore an uncorrupted DB backup. This means that even the most cautious user with 60 days of backup cannot be 100% sure that restoring will suffice to make Roon operational again, as you might have backed up corrupted data without knowing. Would an export/import function to/from a basic DB format be a solution?
One would expect so. Definitely yes.
One point to mention is we probably don’t know what Roon is doing during a backup. If there already an integrity check is included we should be fine.
I do not think there is any integrity check, unfortunately. Which is why roon support tells you that (when the problem occurs) you may have to restore from old backups as the most recent ones are likely to carry the same corrupted data that make DB loading fail.
Fair point. Maybe a bit more transparency re uncritical technical details by Roonlabs would be helpful so users know about risks they take for the work they invest.