Migrating Music Files to Nucleus Internal Storage

Roon Software Discussion Nucleus
1

Hi, I have been running RoonCore and storing my music files on a Synology DS716+II NAS (celeron CPU and 2GB RAM) for more than a year. Obviously the NAS is not up to the recommended specs by Roon, but for some reason it has performed smoothly even I do upsampling too.

Although there has been no issues, I recently bought a Nucleus with the intent to replace the NAS for, in order to benefit from its no fan design and potentially lower electrical noise. I installed a server grade SSD drive inside Nucleus, and formatted it on the RoonOS interface through the web browser. However, after I connected a USB drive to Nucleus and tried to move my music files (~500GB, which is not big at all) to the internal drive, it showed a remaining time of “2 DAYS”! I let it run for about 1 hour, and only 5GB of data was copied and pasted over, extremely slow. To be specific, I use Finder in my Mac Mini to see Nucleus’ internal drive and the attached USB drive as network share drives, and sent my “copy and paste” command over this way. I speculate that the extremely slow transfer speed is caused by that the data does not move directly via USB connection from the external drive into the internal storage, but instead the data travels through the home network to the main router and then travels back to Nucleus before pasting to the internal drive.

My speculation above is based on my previous Q&As with Synology tech support. There are two ways to transfer files between an external USB drive and the NAS internal drive - (1) use Windows or MacOS to see the NAS drive and the attached USB drive and send command that way, which takes long time to transfer data; and (2) use the Synology “File Station” app in the Synology OS interface and send command that way, which makes transfer very fast at normal USB transfer speed. Synology told me that approach (1) is not a preferred way to transfer large files, because data unneccesarily travels through the wifi network although the drive is physically attached to the NAS via USB. Approach (2) is recommended, because the Synology app prevents the unneccesary travel but forces data directly transfer via USB,

If I am not mistaken, what i experienced with Nucleus is the same as (1) with Synology NAS. In order to allow directly data transfer from an attached USB drive in to an internal drive inside Nucleus, Roon has to add a function in the RoonOS interface to allow such direct data transfer (this will help backing up music files too). Unfortunately, I did not find such function in RoonOS interface yet.

I have seen people recommending transferring files between the NAS and Nucleus, which I tried and it still required 2 days to transfer 500GB. Transferring large files over ethernet does not seem efficient. By the way, my NAS is connected to the same switch that Nucleus and my streaming DAC are both connected to.

It seems to me Roon recommends internal storage for music files inside Nucleus, which makes sense to me and i would prefer (since my music library is not big, why would i want music storage on a NAS separately). Also want to mention that unlike a NAS, Nucleus with internal storage drive does not automatically broadcast itself on the home network. Instead, I need to type in the network path to connect to it (and it took me some trial and error before gaining access to the USB drive attached to Nucleus, which initially said i do not have the authority to visit). Anyway, it is very inconvenient to use Nucleus as a network storage. This is disappointing to me.

Can anyone advise if I missed anything? If my understanding above is generally correct, is it possible that Roon can add such direct copy and paste function in the RoonOS interface? I really want to get rid of the NAS if all possible, but now i have no choice but continue using the NAS for music storage and music file management, such as importing newly purchased music files, directly editing metadata, routinely backing up the music files, etc. Thanks in advance for your help!

2

Something’s not right… When I bought my Nucleus, I installed a 4TB SSD inside the Nucleus and transferred about 2.5TB of files from my NAS to the Nucleus in about 4 or 5 hours. I did the copy using my Mac, all connected via Ethernet.

A couple of things. Is everything on Ethernet? The copy process is going to transfer the files through the computer you are using to do the copy, so if it’s on WiFi, then it will be very slow compared to Ethernet. Also, why are you connecting a USB drive to the Nucleus? It would be faster to actually copy from the NAS since the USB transfer rate (and probably the drive) would not be as fast as using the NAS. Also, check your Ethernet cables, make sure they are at least cat 5e. If you have older cat 5 cables, you will limit your transfer rates.

In short, copy from the NAS to the Nucleus, have everything on Ethernet, (preferably, the same hub) and you should be able to transfer 500GB pretty quickly.

1 Like
3

In addition, you might be better off using a file sync software such as carbon copy cloner to the do the copy rather than just using the finder. This will also allow you to continue using the NAS to organize all your new files, and then copy them periodically to the Nucleus. Carbon copy cloner will just copy new files and files than have changed. Very easy to use and fast copies. In this way, the NAS is your reference, and the Nucleus is a synced copy. You will then have the NAS as the source, and you can use synology hyperbackup to create true back ups to a USB drive attached to the NAS. That’s the best solution if you want redundancy and backup to ensure you won’t loose your files if you accidentally delete something, experience a drive failure, or if files become corrupted over time.

1 Like
4

Thanks a lot!

The reason I tried to transfer the files from the USB drive to the Nucleus internal storage was that I had good experience doing so with my NAS previously (less than 30 minutes for 500GB). Now I have realized that, after my experiment and with your comments, this way doesn’t work well with Nucleus.

I did not try copying from NAS to Nucleus, because my NAS was previously configured as RAID 0 with two 1TB SSD, and I took out one of them and put it into Nucleus, so the NAS was not functioning at the time. So the only copy I had at the time was just the external USB drive, which doesn’t have Ethernet connection.

Now I have put everything back to the NAS, using it for music storage, while Nucleus has no internal drive right now and is only running the RoonCore. The NAS, Nucleus, and my streaming DAC are all wire connected to the same audiophile grade switch (forming a little hub/local music network, so this part should be robust). And all connections are using WireWorld CAT8 audiophile grade Ethernet cables. Based on what you told me, I will buy a new SSD for Nucleus and try again transferring files from the NAS to Nucleus. Couple of hours are acceptable. Thanks for letting me know.

May I ask you couple more questions?
(1) The SSDs I am using in my NAS right now are Western Digital WD Red SA500 (1TB), which is a server grade SSD that is designed for running 24/7. Do you think I can just repurchase the same model for Nucleus, or I should look into other options that might be faster but still server grade? If yes, what would you recommend?

(2) I noticed you mentioned that “the copy process will go through my computer via WiFi…”. Does that mean even though both my NAS and Nucleus are wire connected to the same switch, if I send the “copy and paste” command through MacOS Finder (my Mac is located in a different room via WiFi), the data has to travel from the NAS back to my Mac via WiFi before going back to Nucleus? I think this is similar to what happened between the external USB drive, my Mac, and Nucleus in my initial failed trial.

If I understand the above correctly, I will log into the NAS’ interface and use its own file management app (File Station) to send the copy and paste command, which should force the data transfer directly between the NAS and Nucleus. Do you agree this is the exact steps I should do it?

(3) if the speed between the NAS and Nucleus are acceptable, then I will use the NAS as the original or the backup (doesn’t matter anymore) for the Nucleus as you suggested. Yes, there are plenty of backup software options with Synology, which I like a lot. And I will configure the NAS in RAID 1 instead, so that both SSDs in the NAS will have the exact copy for redundancy. Am I getting this right?

I think I have been previously spoiled by the convenience using the NAS for both RoonCore and file storage/management along with an external USB drive. I just need to adapt to the new combo, Nucleus + NAS now, and I can ditch the USB drive if everything works out well.

Thanks again!

5

Right, if the Mac you are copying on is on WiFi, that’s the reason it is so slow. It’s because the files need to pass over WiFi from the NAS to Mac and then again via WiFi from Mac to Nucleus. You could connect from the NAS to the Nucleus and copy via File Station. Then everything would travel via ethernet in one pass.

As for SSDs for the Nucleus, I would recommend Samsung EVO or QVO. No reason for server grade if you are going to also have the NAS for redundancy. They are pretty cheap in 1TB sizes. I would get at least double the size of the files you have now to allow for growth, and you want to leave SSDs at least 25% free. Better to go bigger if you have doubts.

For the NAS, yes, RAID 1or SHR for redundancy, however, you should also have an attached USB drive or two for backups using Hyperbackup. The RAID will not help you if you accidentally delete, change or have a corrupted file. You want a true backup that lets you recover or copy from a previous version.

6

Thanks for confirming! Great point on the true backup using an external USB drive (will continue using the one that I have been using).

One more specific question that came to my mind - is Nucleus with internal storage supposed to automatically broadcast itself on the network and File Station can see it immediately?

I asked this because in my initial trial, Finder in my Mac could not automatically see Nucleus (unlike NAS that goes automatic), but I need to type in the SMB network path and sign in as a “guest” before seeing it, and it did not stay after I rebooted the Nucleus or rebooted my Mac. I may have set my expectation too high that Nucleus should show up like a NAS, but it did not. More importantly, I just called Synology tech support and asked if I can access Nucleus via Ethernet and File Station, if Nucleus doesn’t automatically show up in File Station, the tech support rep said no and added that File Station is designed to show its internal storage pools and drives connected to it via USB. Things on the home network would not show up in File Station. I am not sure if this is true, as I do not have a drive inside Nucleus right now to try. If File Station cannot see Nucleus, how do I avoid going back the slow Mac route (smb)?

Synology did confirm that I should avoid doing the smb/Mac approach if I can, but he could not think of any other way to do it inside Synology. :man_facepalming:t2:

Thanks!

7

I bought a Samsung EVO SSD and installed it in Nucleus tonight. I was able to see it in Finder on my Mac (connected to home network via WiFi). However, as what Synology told me earlier, I could not see Nucleus in File Station app in Synology DSM, so I was not able to copy and paste directly between the NAS and Nucleus. I also tried HyperBackup, and tried to set the backup destination as Nucleus internal storage, but HyperBackup cannot access Nucleus’ IP address.

This left me no option but do copy and paste in Finder. As expected, this is very slow again as data travels from the NAS to my Mac in a different room via WiFi before going back to Nucleus. I did the transfer folder by folder, and it took one hour to transfer 15GB of files. So my 500GB music files will take ~33 hours to complete. 15GB per hour is very slow and unacceptable.

I understand the link between the satellite router/network switch and my Mac Mini is very weak (WiFi), but I cannot change that. My Mac Mini is sitting in a room a little far away from the music room, and between I cannot simply run ethernet cables. I guess people who can do this with reasonable speed because their computer might be wire connected to the NAS and the Nucleus. Unfortunately, I do not have a laptop PC or MacBook to connect to the switch to shorten the route, but even i have, I would not want to do that everytime I want to transfer files into Nucleus. The NAS is there physically connected to the Nucleus via ethernet, why can’t they just transfer files directly between them without traveling throughout the home WiFi network?!

Please help. Thanks.

8

Moment 22 accomplished… :wink:

Seriously though, there is absolutely nothing wrong with the write speeds of the Nucleus/ROCK with internal SSD storage. I both read and write to my ROCK storage at well over 100MB/sec when both parties are on Gb ethernet.
With the snail-pace speeds you are getting i suppose you re on 2.4Ghz WLAN with the lousy duplex speeds it offers. So, basically your network is not up to snuff…

Now, what can be done here?
First up;
Attach your previously mentioned USB-drive with your library to your Mac rather than the Nucleus. Your file transfer will be one way, and should at least reach about 10MB/sec. Not optimal, no, but if you had better reach of your WLAN you could utilize the 5Ghz band which provide far better speeds and primarily way better duplexing (traffic in both directions)

But this will only solve this prticular scenario. You need some on site professional help with sorting your network. Sorry, but thats a fact…

9

Thanks Mikael! I have to admit that my WiFi network is weak, this is the fact. I have not updated it as i have been waiting for AX mesh routers to become popular and cheap (preparing for the next gen). Sorry for being too cheap here, my bad.

I was disappointed by Nucleus because Synology can provide a File Station app which keeps the file transfer locally between the attached USB drive and the NAS and transfers 500GB of files within 30 minutes. File Station completely helped data transfer avoid going through my weak WiFi network, such a happy solution for people who may not have robust WiFi network. Why can’t Roon, which is so good at networking, be able to add a simple file transfer button/function in its RoonOS interface, which will be a great help for the initial file migration between a USB drive and internal storage? This is still hard for me to understand.

Anyway, I think your suggestion to connect the USB drive to my remotely located Mac Mini is a very good suggestion, to keep the file transfer one way. Such a good idea! Yes, my Mac Mini is connected to WiFi via the 5Ghz band, and the satellite router that connects the NAS and Nucleus via ethernet cables also connects to the main router via 5Ghz band. I am going to try that now.

Thanks Mikael!

10

Okej, great! Let us hear about your progress.

And, i see what you’d like to accomplish by using the File Station app on your Synology.
I dont use Synology, but QNAP NAS, and in it’s File Station i can “map” a network drive, such as the internal storage of my Nucleus, and perform copying through that interface.
However, in my QNAP’s case that method is way slower than the plain vanilla SMB-transfer offered by a Win/Mac PC on wired LAN.
Still, i’ts not the Nucleus fault that your Synology doesn’t discover network shares! :wink: That is a security measure and inherent to non Windows/Mac operating systems.

Anyways, you can try and map your Nucleus from your Syno using the
path:
smb://<your nucleus ip-adress)/Data/Storage/InternalStorage/
If requested use “anonymous” and “guest” for user and password.

11

Fortunately and finally I was able to mount Nucleus internal drive in Synology File Station using CIFS protocol “\Nucleus IP address\Data\Storage\InternalStorage”…after mounting it, copying the files from Synology NAS in to Nucleus became direct and sooooooooo much faster.

I totally agreed that it is not Nucleus’ fault if Synology doesn’t discover network shares (glad that it does now). What if I do not have a NAS at all, but bought Nucleus with the intent to run RoonCore and store music files. In that scenario I (or other customers) may only have music files in a USB drive. What I suggest Roon is to add a button in Nucleus’ OS interface to allow direct file transfer between the internal drive and a connected USB drive (no IP address, but directly connected to Nucleus via USB). With that, the initial migration does not rely on the smb protocol, which helps people like me who may not have good home WiFi connection.

All good now, thanks again Mikael for your help!

1 Like
12

This is not a bad idea, and could be very useful. Anyways, glad it worked better this way! :slight_smile:

13

Screen Shot 2021-04-06 at 9.05.45 AM
Screen Shot 2021-04-06 at 9.05.45 AM912Ă—550 38.7 KB

Screenshot above shows how i mounted Nucleus in File Station, using CIFS protocol (not sure what it is :)). I did not try NFS protocol, which probably works too. This way I got 500GB moved over well within 45 minutes, problem solved!

Thanks Mikael again!