Online backup strategy (and making it work)

I’m aware of and convinced by conventional thinking about backing up a music library–that you need several backups, at least one off-site. Right now though, I’m focused on an online backup. It occurred to me that there’s something very basic here that I don’t know.

In backing up a smaller file, like Roon’s database, it makes sense to keep several independent copies, going back a few revisions. (Backup) But for something like a whole music library–mine is a bit more than 2TB–that strategy is very expensive. The alternative is an incremental backup, which you keep a single version in the cloud, which is merely adjusted as the local library changes, to mirror the local library. (This is called “sync,” right?)

That makes a lot of sense except–if your local library becomes corrupted, or if there’s a major inadvertent deletion–anything like that–wouldn’t this be directly reflected in the backup? The backup wouldn’t know the difference between unintentional and intentional changes. So then your backup mirrors the corruption of the local library.

Please share thoughts. Pragmatically, that the primary library is on a Synology NAS. This is what I’m backing up–to the cloud–and not what I’m backing up TO. Synology NASes comes with many tools for backup, some for backing up to the Synology, others for backing up the Synology. But before I can move forward, I need to make some basic decisions.

The backup is incremental but each step is known and can be traced back. Unchanged files have only one copy, for each changed file the older and the newer version are retained. (Up to the configured number of backups to keep). So you can go back to an earlier state.

This is different from a straight mirror configuration, where the secondary location really does mirror the primary location in its present state.

1 Like

This sounds great–but I need to make sure I understand. Do all “sync” programs work this way?

They should. The QNAP HBS 3 Hybrid Backup Sync works that way but I hesitate to say that all do. Surely, there’s documentation to consult.

I keep a nightly backup permanently attached to my Synology, and my music collection is currently about 2.4 TB.

I keep another backup in a fireproof safe in my house, updated every couple of months or so once I get my collection into a certain shape.

I have another two copies offsite, updated a few times a year as well.

All of these are physical copies, and physical drives are relatively cheap, easy to recover from, and easy to manage… I find this far easier to deal with than any cloud based backups.

I log all changes and backups in a text file that is also backed up with my music.

I like having these disparate backups available, because on rare occasions, I can refer to one of them to validate whether or not I have a corrupted file or it was corrupted out all along, etc.

Given that I’ve been collecting digital music for over 40 years, I don’t find the risk of any gap in these backups to be particularly troublesome.

So I suppose, @Jim_Austin, I’m probably not answering your question, because I deliberately avoid online backups. But it is my perspective, and I thought it might be useful to share.

The HBS can do both, simple 1:1 sync/mirror and incremental backups. They are different types if jobs. If I recall correctly, it can do backups of data that is already on the NAS, but external disks can only be mirrored. So I set up a mirror task to pull the music files off my ROCK to the NAS, and then a backup task to back up the NAS copy.

Exactly what I do every day.

Since you are using Synology, you may want to consider Borg backup.

This is a deduplicating archiver, so is space efficient and very good at keeping archives small. For instance, it only backs up parts of changed files, so if you edited the metadata of a FLAC file, it would only upload the changed blocks rather than the entire file.

Restoring files is lightning fast and reliable.

There are GUIs for Linux, macOS, and Windows that can be used to browse the archives.

I recommend Hetzner Storage Box for remote backups. 5 TB costs me around £9 pcm.

Thanks for that. All of my library is on a NAS and backed up to other NAS, so I was unaware of the restrictions for local libraries.

Have you considered using Synology’s HyperBackup in combination with an affordable S3 compatible online storage service? Personally I’m using iDrive e2, but I believe BackBlaze B2 is somewhat in the same price range. I use it not only to backup my music and the Roon database backup files, but also photos and documents.

The backup runs on a daily/nightly basis. I have it configured to keep daily versions of the last 30 days, weekly versions of the last 13 weeks and monthly versions before that. HyperBackup uses incremental block level backup, and keeps only unique versions. That means that if a file is not changed between backups, there will be no extra backup space used. Furthermore, minor changes to a file, such as changing metadata in a 100MB audio file will not backup the entire file, but only the blocks of the file tha have actually been modified, thereby greatly reducing the actual storage required.

I would think not or you could be duplicating files massively . Take a scenario where you make a change to metadata , you now have 2 copies of that file, you make another change , 3 files etc

I use SyncBack Pro , which copies any changed files , once copied removes older versions of that file. So your BU can be very small depending upon how many new files you add or files you change. You finish up with a MIRROR .

We are dealing with a music library not legal or financial documents , if legal etc I would agree keeping multiple versioned copies (so that changes can be monitored between versions) but still I can see no reason for keeping multiple copies of the same versioned documents in the same backup.

I keep my main library on my Main PC and mirror to an SSD in my NUC/ROCK , then I mirror again to 2 x USB HDD as back ups . The only thing I don’t do is to keep a copy off site.