Crashplan has always worked well for file/folder recoveries but I’ve never had to do a full drive recovery. Sounds like I’ve been dodging a bullet. I am currently evaluating iDrive and will also look at BackBlaze as alternatives. They all seem to have issues on one front or another. Given that my primary concern is for music, it would be interesting to see what Roon might come up with.
@danny - I used the CP for Home family plan for years and then switched to CP for Small Business when they offered 1 year at a substantial discount for prior Home use customers. That year’s now up so looking at alternatives which is why this thread caught my eye.
If you want real backups, you really have to use something like crash plan or back blaze. Any other options have a lot of holes in their strategies. For example - files do corrupt and the average time for that to be notices is 120 days. So unless you’re keeping ‘points in time’ backups like the online services do - e.g. you have multiple tapes or something you’re going to lose this data. ZFS file system goes a long way to rectify this because of it’s self healing and snapshotting but you’re a brave man if you rely on that. Crashplan keeps unlimited copies for unlimited time, I’ve done the calculations and it is still much cheaper than doing it any other way - assuming you want proper backups of course.
And yes, it is a lot of data in the beginning, but once it’s uploaded, it’s only a little data ongoing - so you can use different strategies to combat that in the beginning if that’s a problem for you - like run the first backup in a different location that doesn’t have throttling or call your ISP and tell them up front, pause it at so many GB’s until next month etc. That’s my 2c, but it’s 2c I’ve learnt from quite a few years of IT experience.
My Roon server is an Ubuntu server. I run HashBackup and back up daily to B2 cloud. I spend about $2.50 a month on B2 storage. But, I stream probably 75% of my music now, after buying Roon, so my music files are less than 400GB. I don’t know how big my Roon backups are, but I’m certain the bulk of my B2 storage is the actual FLAC/ALAC stuff. I haven’t bought an album for download in over a year.
Backup solutions are becoming less and less necessary as users drift away from local files to streaming only, in fact, some of our own Roon community have abandoned their physical media altogether.
I would imagine that most users physical media acquisition has declined as their streaming use goes up. And this will continue.
One issue, for me, is that if the backup is for “music files only”, then my backup life become complicated. I will have one backup subscription for music files, and then, another for my other files. I would rather just one. And as for price, it would need to be cheaper than the other services because I would be using two backup solutions.
I have no doubt that there are war-stories with respect to every available backup solution. Crashplan seems solid but you don’t know what you don’t know until it’s too late.
With respect to files that get corrupted over time… I have had files corrupt or go missing somewhere along the way. But detecting such has been totally hit or miss. Fortunately, when it has happened, my CP backups had an earlier version that was fine. What tools are available that will proactively look for such problems? While I run Roon on NUC/ROCK, the drive serving that to the NUC is an external USB drive that is duplicated from my primary storage managed via Win 10.
My personal music library is about 2TB so initial backup can be time-consuming but not unreasonably so. Switching away from Crashplan is a concern simply because I can’t be certain that my current local library doesn’t have a few corrupt (or missing) files. Losing a few years of CP version backups could be a bigger loss than I might anticipate.
There are a few things I’ve seen that run manual checksums over data, but the only thing I’ve seen that works properly is ZFS. I do believe ZFS can be used on Windows, Mac, Linux or BSD, so I suspect you’ll be covered. It’s also remarkably simple to interact with, however you do have to learn some commands. If you want a GUI, the best thing for ZFS at the moment is still TrueNAS, however it’s supposed to be coming to Unraid in some form which may rock the applecart a little.
There are different versions of this and you need different tools for formats you care about. This just demonstrates an effective approach. Most of my collection composes FLAC files and JPEG artwork. For a FLAC library there is reference validator provided by Xiph.org Foundation. The jpeginfo tool provides validation for JPEGs. Other tools are available but these are both open source and available to anyone. Here’s a demonstration on a single album:
cfw@roon:/music/ssd/Aphex Twin/Selected Ambient Works 85-92 (Remastered)$ date
Wed 31 Mar 21:27:17 BST 2021
cfw@roon:/music/ssd/Aphex Twin/Selected Ambient Works 85-92 (Remastered)$ flac -t *.flac
flac 1.3.2
Copyright (C) 2000-2009 Josh Coalson, 2011-2016 Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY. This is free software, and you are
welcome to redistribute it under certain conditions. Type `flac' for details.
01 - Xtal.flac: ok
02 - Tha.flac: ok
03 - Pulsewidth.flac: ok
04 - Ageispolis.flac: ok
05 - i.flac: ok
06 - Green Calx.flac: ok
07 - Heliosphan.flac: ok
08 - We Are The Music Makers.flac: ok
09 - Schottkey 7th Path.flac: ok
10 - Ptolemy.flac: ok
11 - Hedphelym.flac: ok
12 - Delphium.flac: ok
13 - Actium.flac: ok
cfw@roon:/music/ssd/Aphex Twin/Selected Ambient Works 85-92 (Remastered)$ jpeginfo -c *.jpg
cover.jpg 355 x 355 24bit JFIF N 21201 [OK]
folder.jpg 355 x 355 24bit JFIF N 21201 [OK]
cfw@roon:/music/ssd/Aphex Twin/Selected Ambient Works 85-92 (Remastered)$ date
Wed 31 Mar 21:27:36 BST 2021
cfw@roon:/music/ssd/Aphex Twin/Selected Ambient Works 85-92 (Remastered)$
If you look at the time checks an album takes 20 seconds, whole collections can be checked in manageable timescales.
This was all run on my NUC core while playing music.
Yeah, reference implementations can be like that, generally built for accuracy, maintainability and reliability.
Performance isn’t usually the aim. It’s also trivial to multi-thread.
I’d only run this once a year or so, just to be sure, or to work out if I could change backup service.
It’s a lot quicker than playing the collection but it’s equivalent to a decode.
Earlier in this thread I said that I wouldn’t personally be interested in this service but I have occasionally considered online backup of ALL my files. In considering this service it’s not clear to me why anyone would be interested in a backup of just their music files. Am I missing something?
Maybe a value added function, less played files (let the users set the limit, 3 months to a year without access, as a whole album) will move up to cloud storage to free up local storage for new music (kind of infinity drive as long as the the cloud drive expanding on demands), but the indexing still here and accessible via cloud as long as internet connection is available, very useful for people not using NAS (I have NAS, but just for backing up ROCK server periodically, then I switch it off to reduce drive wear and energy consumption. Then using storage attached to ROCK server, so the main traffic still remains locally.
So the internet traffic will not be super busy and gentle to cloud server.
The old Google Play Music had problems, but it had a concept that I really liked: cloud music storage with local caching. I could pick some playlists, ask for them to be downloaded to my phone, and then go on a trip to places with limited cell coverage and still have that music available for local play. I’d be very interested in a service that allows me to store all my local music in the cloud, which serves both as a backup and as a streaming source for different locations, with all the Roon database information being transparently shared across all of my Roon systems. Because there’s not anything like this, I’m currently carrying a server with a 4TB SSD between the two locations I spend my time at. I’d rather have two separate Roon servers at the two locations (and pay Roon Labs for two licenses ) but that then I’d have two essentially unsynchronizable Roon systems.
I believe you, Craig. But can someone from the Roon team confirm this (@Danny)? If this is the case, I would like this to be fixed before a solution for music backup is developed.
According to the KnowledgeBase regarding Backup:
Your Roon database is full of information about the music you love. In addition to your library’s metadata, it also contains any edits you’ve made, your play history, all of your Roon playlists, your favorites, your settings, and more. All of this information is stored in your Roon Database.
Cover art and artist photos are edits I’ve made so I do want them to be included in the backup.
If that data is not included yet, I would open a feature request. And I would suggest to edit the KnowledgeBase accordingly.
RAID is for high availability…not for backup. If the computer writing to the drives writes bad data for some reason, you data is toast. If you or someone else deletes a file, by accident or on purpose, it’s gone.
No, I am not interested in Roon creating a music file backup solution. You guys are not experts at that. Stay in your lane, please! Put resources into making the your software function better. Or put resources into fixing bugs or adding features like proper handling of box sets.
+1 for cloud locker but it must be a service that is more than just a backup. If this were also the solution to offer a mobile service, or more exactly “play Roon outside your local network”, with the possibility of play / sync / download / manage the library and cover all cases in which you might want to use Roon "outside home ", then it would be +2
Bug fixing and developing new features like this should be 2 separate areas which must work in parallel.
The ability to backup the music location to another drive/device or network location would be good - similar to backup of the database in terms of locations.
But I would rather have have integrity checking of the database before and after backup to avoid corrupted backups.
I suppose backup would be nice to have for many - a turnkey solution that required nothing more than the roon core to operate.
Myself, I store music and other data on a local nas, with dropbox sync from the nas. Works brilliantly, so I don’t really need an extra backup feature in Roon.
For me, backups are complicated and there is no one size fits all. So I can easily imagine a solution that is specific to my music that doesn’t have to cover everything else. Of course, if Roon is going to backup my music directories that it also scans, then I would assume it would include everything in those directory trees includes non-music files as well as the Roon backups and any artist artwork I add.
The challenge will be the scope of additional functionality versus the cost of providing it. Got me curious…
This idea has value to me but I’d say the cost would actually have to be less than other cloud solutions assuming that it is just for the music library. Primarily because OneDrive (something I use to backup my library right now) can be used for anything and is integrated into everything (iOS, Android, web etc).
The only way this would be worth paying the same as other cloud storage is if it unlocks caching to devices. If I can play music from my iPhone on Roon without connecting to the network (just pulling local files down from the Roon cloud) then it would be worth paying £5 - £8 (size of library dependent).
) but that then I’d have two essentially unsynchronizable Roon systems.