Import issues caused by SMB file naming constraints?

I’m currently evaluating Roon. I have loaded up a Synology NAS with about 300 albums so far and then had Roon ingest it. I’m using Roon Server on a headless Mac Mini (fully maxed out).

My file structure is ‘similar’ to the first scheme you described except I’ve put the artist at one level up. Using your example:

Music/
Miles Davis/
The Complete Columbia Album Collection/
01-01 Track.flac
01-02 Track.flac

02-01 Track.flac

I have a number of problems with the ingestion. I have split albums all over but in no consistent way. Some albums worked perfect and some ended up in Roon as multiple albums. Secondly some albums are totally missing tracks altogether, even at the individual CD level. e.g. 02-01 Track.Flac; 02-04 Track.flac (tracks 2 and 3 are MIA). I tried the “Fix Track Groupings” and in some cases I can find the missing tracks, while other cases they just don’t exist in Roon.

I found these problems prevalent in one part of my library - my live concert collection. One Artist has over 260 albums on my NAS. As you might imagine there are several instances of the same song. But I checked the meta data of some of the troubled Roon albums and the meta data identifies the album consistently and appears to be good.

Note that the other parts of my Roon library appear to have ingested successfully. It is this concert collection section that has really gone poorly. Note that these are not bootlegs, but released concerts. Roon can even find the correct album covers for most.

Any ideas on where to look? If Roon can’t handle my live concert collection it is a deal breaker which would be a shame because I’m loving other aspects of the product.

I’ve learned something more on this problem of split albums and missing tracks.

If I open two Finder windows on my music library (Synology NAS), one with SMB protocol and one using AFP, the results are very different and have very similar pattern to what I’m seeing in Roon.

The AFP Finder window shows all folders and files in the names used when I created them, also matching the folder/file names I see when viewing via DSM File Station. Everything perfect. However when I view the same files via a Finder window opened with SMB the results are very different. I see many folders and files in encrypted names, old style DOS 8-character names (at least it looks like that e.g. ABCD$EF$). And here is the tie - the files that have the converted 8-char file names are the very same files that are the missing tracks in Roon.

It’s obvious that roon is reading the NAS files through the same SMB channel, not seeing the encrypted file names as music files and therefore not registering them.

Now the question is why. As this problem is happening in Finder w/o Roon it is bigger than just Roon, however because Roon requires SMB it is suffering from this.

Looking at my NAS setup it offers 4 flavors of SMB to choose from. SMB 1, 2, 2 w/ MTU and 3. By default Synology defaults to SMB-2. I don’t know if changing SMB flavors will do anything but will try (tomorrow) and see if it fixes the problem. I’d appreciate if anyone with knowledge in this space could chime in with some help.

I would try SMB 3, but let me know if you’re still having problems here and we can look into this further.

How are you adding the NAS in Roon? IP address? Host name? Mapped drive?

I think I’ve resolved it.

AFP is very free and easy with file and folder naming, basically any character is acceptable. SMB is far more restrictive. If your files use a forbidden character then SMB converts to the old 8.3 MSDOS naming convention. To do this requires some very funky munging of the file name into a unique 8 character set.

Roon does its ingestion and apparently does not know what to do with these munged file/folder names. For munged folders Roon seems to still parse into the folder and grab any music files it can find. Apparently it is then relying on embedded meta data within the file to assign it to an artist and album. For munged file names Roon seems to give them a skip and does not attempt to ingest them. My guess is it doesn’t see them as a music file.

Here is a link with a pretty good description of the constraints on SMB file/folder naming:

http://www.portfoliofaq.com/pfaq/FAQ00352.htm

I’ve tested my hypothesis and so far it is ringing true. I’ve done some track renaming and some folder renaming, rescan the watch folder and bingo - album is ingested and even the art work is found.

Looks like I have a lot of tedious homework to do. Grateful Dead second sets run song into song into song. The typical CD sign is the “>” character. See note #1 in the above link about the right angle bracket character - strictly forbidden. I only have 240 albums of 3-4 CDs each of removing ‘>’ from ends of songs. :frowning:

PS: I did try SMB3 and that didn’t solve it. After googling and reading many articles on SMB I can see it would not do anything on the character restrictions.

Ok, I can’t make any commitments here, but we may be able to do better here. We’d probably want to get remote access to your system so we could extract the files exactly as they appear on your NAS, so we can reproduce in house and experiment.

If you’re interested in us looking into this further, drop me a PM and we’ll see what we can figure out. Thanks!

Found cause to the last broken part - munged file names. Any folder names that ended in a blank (i.e. 20 hex) caused their munging. Editing out the trailing blank characters and the folders now appear.

Like the file names, the folder names were all debugged using two Finder windows, one connecting via AFP and the other SMB2. Once cleaned up Roon was able to ingest everything. I still have some minor meta data issues but these look to be caused by actual meta data glitches in the target files.

Regarding taking a look at my source files I’ll email you as you suggest.

I have this problem too. Will try the same fix.

(Naming restrictions to folder names. What year is this…?)

Naming restrictions are painful and I would hope that trailing spaces might be better handled. It’s a good pickup by @rayski and I’ll drop a flag for @mike to let us know if that’s possible.

Other naming restrictions arise because of interoperability. One of the main principles of Roon design is that the various components (Core, Control, Output) should all work together regardless of what OS is running on each device. Roon does that by deprecating all special characters in all applicable operating systems.

1 Like

I understand. The issue is with the protocol, not Roon.

Since I sometimes mark folders with “special” characters, I need to manually go through my libary. Being a mac guy, that feels odd…

Fredrik - While investigating my problem I went through a lot of Goggling articles on the AFP/SMB subject. From what I read Apple is slowly moving away from AFP, adopting SMB as the standard. It seems with El Capitan Apple defaults the Finder to read other systems via SMB forcing you to go out of the way to not use it.

So although it is a painful exercise to visit your files and clean it up, it might be something you’re going to end up doing down the road for all your files if you run any kind of shared network. As a first step you should consider stop using all those funky characters in future files names.

Thanks rayski.
From a mac perspective – is there a difference between SMB 1, 2 and 3?
Weird naming – a quick look gave that my most common funky character does not cause trouble (appending “ƒ” to folders)

Thanks for flagging this up.

I’m still in the thinking about Roon phase and although largely a Mac user, had planned to try running a headless RoonServer on a HP Microserver with Windows 10.

I suspect i’d hit exactly the same file/ folder name issue - although I feel sure there must be a technical solution out there somewhere. Something like Dougs Applescripts maybe…

I chose Windows because it seemed most likely OS to allow Roon to get the best out of lower spec hardware, maybe I need to think about Linux or Hackintosh on the HP.

I’ll keep an eye on this thread to see what developments happen.
———————————
Had a look on Dougs Applescripts and nothing stood out as being a solution to this problem.

Also had a google around and didn’t find anything. I’m surprised - because it feels like a common enough problem, that someone would’ve pulled together an application, to review a set of files from Mac OS and apply Windows / SMB ‘rules’ to identify problem files.

To then to offer the opportunity to apply edit rules to groups of those files - seems relatively straightforward to my very much non software developer mindset.

SMB character limitations as I understand it:
Nether albums nor songs can contains any of the following characters:
:
?
;
,
*
"
/
Blank space as last character

Problematic characters:
&
#
|
<
>
%

/Rant mode on/

As a mac guy I find these restrictions pathetically stupid. I had cases of all of the above. How many album have “&” or “,” in its name? Classic music with “/” or “#”? How do you correct for the blank space as last character error?

Case in point: Aimee Mann’s 2008 album @#%&*! smilers. Not allowed. Oh no. Nope.

/Rant mode off/

Change the file names to delete deprecated characters or substitute them with z. You can do it as a batch process on a library using an editor. Provided the file tags have the correct names they will still show up correctly in Roon.

Yes, it is inconvenient, but the alternative is seeing different content under different OSs. Roon opted for a uniform user experience across platforms. This means less ranting across the user base.