Backing up my music library - Best practice

I recently tried backing up the 6.5 terabytes in my music library using Carbon Copy Cloner for mac to a new LaCie 10TB external drive attached to my Mac.
After several days it appeared to have completed with several errors but after analysing the errors in CCC the software crashed and I had to restart the Mac.
After re-starting the LaCie became read-only and I was only able to reformat it APFS. Support at Carbon Copy Cloner advised me that the errors were due to another program using the files while they were being backed up. I presume Roon is the culprit here and perhaps I shouldn’t have been using Roon while I was backing up the music files.
The LaCie distributer Scorptec computers kindly replaced the drive and now I have a fresh backup drive pre-formatted Fat32.
Can somebody please advise about the best practice for backing up my music files from Nucleus.
Here is the directory of the internal files and the attached drive with my music folders. Any advise or support about this would be appreciated.

This is helpful information @Paul_Elliott! I wouldn’t have thought about trashing a drive because of concomitant use.

One important thing to understand about backups of such a large size is that the first one takes a long time, subsequent backups don’t. The whole database has to be written. Also, backup programs generally aren’t as fast as simple file transfers, so it will take even longer. Once the primary database is written to backup, subsequent backups will be incremental and won’t take very long.

Because the backups aren’t designed for speed, it’s best to schedule them for off hours.
Whenever possible, the first backup should be done with an ethernet connection. When I have to transfer large amounts of data from a computer on my network to my NAS, I do it through the NAS rather than via my laptop. That way it happens via wire rather than wifi.

Another general good practice is to avoid moving music into your library while Roon is running. Roon can run into issues if it’s trying to match music for the database while new music is being added. I haven’t found it a problem for the occasional album, but it’s not good practice to add large chunks of music to the library when Roon is running. Turn Roon off, then move the music.

1 Like

HI Scott G,

Thanks for the info.
OK stopped Roon running, I’ve plugged the LaCie external disk into the back of the Nucleus and selected the music files from Nucleus and copied them over to the LaCie. I decided not to use Carbon Copy Cloner this time.

Wish me well, hope I don’t have to wait too long before I can use Roon again.

I hope it goes smoothly.
I know it sounds odd, but I suspect this may go slower than you think. They way I’ve always thought about this, if you have a PC via Wifi and drag the files from your Nucleus to your LaCie in the back of the Nucleus (via the PC), it all goes through the PC which is wireless. But if you plug the LaCie into your PC, turn off the wifi on the PC and plug it into ethernet, the transfer from Nucleus to LaCie is via ethernet.
@BlackJack is a real wizard in things like this. Perhaps he can comment?

Thanks for the info.

I’ve got 2 Macbook pros without ethernet to the Macs. One has a USB 3.1 link to Ethernet via a USB hub.

This would definitely speed up the transfer. The external drive should be connected to the Mac/PC to avoid sending the data twice over the network. As a Nucleus is a small device, it should be possible to move it temporarily if needed to place it somewhere where wired Ethernet is available.

Note: 6.5TB won’t get copied in an hour. I would expect something around 16 to 18 hours over Gigabit Ethernet if all works well.

See also:

The backup stopped due to an error:

"The Finder can’t complete the operation because some of the data in 125728_the_best_exotic_marigold_hotel_iqpod_ipad.mlev can’t be read or written (error code -36)

Any ideas why I would get this message, as I don’t have the the best exotic… movie and I’m only copying music from the Nucleus to the external hard disk?

As I’m not a Mac user, this is the best I can come up with:

https://www.google.com/search?q="The+Finder+can’t+complete+the+operation"+"error+code+-36"

MacOS has different filename conventions than Linux, which the Nucleus uses. Things can get funky when trying to use Mac to backup Linux filesystems, and I’ve learned the hard way – I use a Mac as my normal computer, but a Linux server for my Roon Core. It gets even worse if the target drive has yet different filename restrictions, FAT32 in your case.

To avoid those problems, all files and folders need to have names compatible with the common denominator or all the filesystems and operating systems involved. Which is a pain if the files were ripped from CDs with a program and operating system that has yet a different idea of allowable filenames.

Thanks for all your help. Fernando_Pereira what do you suggest as I’ve 6.5TB of music files to backup from my Nucleus to the LaCie?

List of error codes https://krypted.com/lists/comprehensive-list-of-mac-os-x-error-codes/ shows -36 as ioErr = -36 I/O error so it sounds like a disk data error when reading that part of the drive.
I have had this myself on OSX when trying to backup and needed a hard drive replacement from Apple (which was free as there was a recall on my iMac).

I’ve found that SyncTwoFolders is fairly bullet-proof for copying large amounts of data from one drive to another. You can’t clone a drive - you need to specify source and target folders (rather than copying a drive to a drive) - but it does work well.

https://throb.pagesperso-orange.fr/site/ind_JS.html?Prg_S.html&Prg_ApplisRB.html#SyncTwoFolders

I’m currently using it to copy just under 5TB from my old drive to a new Seagate 8TB drive. It’s been running for about 12 hours and I reckon it has another 10 to go, but it’s been flawless so far. I also use this software for incremental backups.

Thanks David, I just had a look at this program, unfortunately my music is spread over several folders, so it’s probably not the best program for me.
I think I’ll give chronosync a look.

Chronosync is a really solid program, well maintained. But the issue I mentioned with potential filename incompatibilities does not go away with it (from experience).

When I used to manage my music files from macOS, I used various methods to ensure track and folder filenames were acceptable to macOS, even though they were stored on a Synology NAS that uses Linux filenames.

These days, I just use Linux rsync between my NAS and my core NUC running Ubuntu, but that’s not possible with the plug-and-play Nucleus.

I have my music organised into two main folders so I only need to run it twice to back up everything. If you’re using a whole bunch of folders then I guess it’s probably an inconvenient solution.

I use the built-in program rsync to copy to and from my ROCK to a local HD. The advantages of using rsync is that it compress the data which speed things up a bit, and that it’s incremental which means that after the first sync it only focus on changed files. Open up a terminal and execute a command like this one below to copy from Rock to a local HD:

rsync -av --progress --delete-after “/Volumes/Data/Storage/InternalStorage/” “/Volumes/Music_Backup”

If you have several folders, you can issue several commands like:

rsync -av --progress --delete-after “/Volumes/Data/Storage/InternalStorage/Jazz” “/Volumes/Music_Backup/Jazz”
rsync -av --progress --delete-after “/Volumes/Data/Storage/InternalStorage/Classical” “/Volumes/Music_Backup/Classical”

1 Like

Thanks for your suggestions, I already had Carbon Copy Cloner and so I’m using that now.
We’re nearly halfway through copying and keeping fingers crossed.