You know, this thread is one of the reasons that I was attracted to Roon in the first place. Not only is it a great piece of software for managing and playing digital music, but the people behind it are smart, knowledgeable, and discuss the innards and design choices in their public forum with us… Thank you.
11 Likes
Yes this thread is a higher gain for the community with each post.
I never suggested it was, that’s relational 
Your post is interesting but it does leave me wondering, if you knew all that why neglect some of the other software engineering Rs, Resilience/Recoverable? That’s rhetorical, things look different in hindsight. I’m about to spend a day triaging bug lists on a few projects, I’ll probably be asking similar questions of myself. Mistakes happen, not making the same ones again is the best response.
1 Like
We resolved the error in 880 about corrupt backups.
Making the db resilient to hardware errors is not a goal.
1 Like
I set up a backup schedule as suggested by @killdozer , but there is only room for two digits so 180 or 365 days is not possible
I have two backup schedules. One is every night at 3am and keep 30. The other is once a month and keep 12. Backing up to 1 TB USB HDD.
Roon backups are incremental, so if an early increment gets corrupted presumably subsequent backups are no good. Is it worth giving users the option of non-incremental backups?
1 Like
I guess it depends on what gets corrupted. I’ve been assuming that the latent corruption is present in the live database, and then transferred to the backup. If this is the case non-incremental backups wouldn’t make any difference.
True, but there has also been talk on these threads of SSD errors giving rise to corruption so that’s more what I had in mind - if an increment got corrupted, would everyone after it be no good? If so then at least the option of full backups might be worth considering.
Yeah, I hadn’t though about that. What we probably need is some clarification from Roon as to exactly where, when and how corruption can occur. I know there have been a variety of discussions (power outages, hardware failure etc), but it’s dotted around in various threads.
2 Likes
Manually initiated Backups are always full backups. I just ping myself a reminder ever Sat night to start the backup before bed.
1 Like
I kinda like the idea of the computer doing stuff for me automatically, especially on a Saturday night!
Guys, give us something to work with… OK, you messed it up by giving us an integrated backup tool, which does not recognize data corruption or inconsistencies. Understood. But now, we are unable to run new backups and can’t get rid of the data corruption.
So please give us something! I expect that Roon is working on this issue with highest priority and at least give its paying customers some workarounds. For example:
- Let us export playlists and re-import them
- Let us export playing history and re-import it
- Let us export our favorites and re-import them
2 Likes
I think you’re narrowing in on it here. The real problem, I think, is not hardware storage failures or other things of that ilk. It’s that you have several classes of data combined.
For instance, you have cached data. The identification of albums and tracks, artist bios and pictures, the results of scanning the LAN to find devices, all of that. LevelDB is a fine way to store cached data; it was designed for that and as you say, Chrome uses it for that. But cached data by definition is not primary data – it’s an efficiency hack. If it goes away, it can be reconstituted, at the expense of a little overhead.
Then you also have historical data. What albums I’ve listened to, when. Past searches. That kind of thing. This is not cached data, because there’s no way to reconstitute it. But it’s not terribly vital data; if it went away, users might grumble a bit but it would build up again automatically in the background.
A third category might be called “precious data”. This would be things that users entered explicitly, like playlists or favorites or corrections to Roon’s metadata or DSP settings. This is stuff that Roon can’t recover without the user’s direct intervention. And this is the kind of data that requires more robust persistence. Typically this is achieved with redundancy, by writing this data in multiple storage areas so that if one fails, the other copies continue to exist. Sometimes it’s even written by different pieces of code, so that if there’s a bug in one externalization procedure, the same bug might not be in other versions of the externalization.
I’d suggest you look at the data you’re storing in your database and classify it according to which category it belongs to. The three I list above might not be all there are, by the way. Then give each category a persistence solution appropriate to its value.
5 Likes
I let Roon do the first few and keep the really old ones on a separate box away from the NUC. More near-line than online with its own software administering the retention.
This is a valuable thread with lots of great info. Thanks to all. @danny you provide some great info here that eased my mind and educated me. Thanks everyone
1 Like
Hi @danny
Are you exploring other DB technology instead of levelDB? Could you share the rationale of this choice for Roon?
In a perfect world, what would you like to achieve on this topic?
No.
Performance.
In a perfect world? No database at all. Purely cloud storage.
Just focus on the stuff that can’t be dynamically rebuilt, that’s the stuff that matters.
3 Likes
I use OneDrive for all my Roon backups (and almost everything else) so as well as local copies my stuff is all backed up in the cloud. Can I sleep worry-free?