Some new directories not found in watched folder

[Jan_Prummel]

Core Machine (Operating system/System info/Roon build number)

Mac mini (late 2012) macOS Sierra 10.12.6, 16 GB 1600MHz DDR3 RAM, Roon v1.7 build 610

Network Details (Including networking gear model/manufacturer and if on WiFi/Ethernet)

Audio files on QNAP TS351 NAS connected via ethernet

Audio Devices (Specify what device you’re using and its connection type - USB/HDMI/etc.)

Squeezebox, iOS devices and RME soundcard

Description Of Issue

First time user: Import of iTunes library went fine, even without using any of the iTunes otl files etc. The directory structure and filenames and metadata from around 9500 ALAC files were indexed correctly (as far as I know, I just finished checking the As and Bs).

Next, I added some Bandcamp purchases, also downloaded as ALAC, but otherwise not touched by iTunes. Upon forcing a rescan of my NAS storage location, Roon found three out of five. One of the successful ones is a pre-order with only a single file, while one of the unsuccessful ones is a full album. I read that too small collections could be of influence, but that doesn’t seem to be the issue.

EDIT: I just realized that the successful additions were by artists that were already in the database, so their new directories were added to an existing ‘artist name’ top directory. The unsuccessful ones were added as artist/album/song.m4a in the top level of the watched directory.

What is the best way to move forward? I’ve read about simple steps like “Clean Up Library” and “Restarting the Roon core” to very involved schemes like “move the unsuccessful dirs out, clean up, move them back, force rescan” etc.

Thanks for any hints,
Jan

[nuwriy]

Hello @Jan_Prummel, so we can get a better image of what you’re seeing, could you please share some screenshots? Thanks!

[Jan_Prummel]

Hi Nuwriy,
I’m not sure how I can make this clearer, but let me try.
My current Library contains 9477 songs, located on a NAS drive.

Among these, most are CD rips which were done by iTunes, but now I also add the occasional purchase from Bandcamp or from the bands themselves. An example is this album Axiom by Christian Scott. Note: It is a new album of an already known artist.

Another example of a Bandcamp purchase is the album Source by Nubya Garcia

Let’s now search my Library for Nubya.

Surprise, no solo album by Nubya, let’s verify if Christian has been found.

Yup, no problems there. Hypothesis: If you add a new album to an artist that is already in your Library, it is found, but if you add the first album of a new artist it is not found. Let’s test that hypothesis. I recently ordered the vinyl of the remixed and remastered version of Tel Your Friends by Snarky Puppy. It comes with an MP3 download. Here is the situation in Finder before adding the directory.

And here is the new directory added, just drag and dropped it here from the Downloads folder. Note there are 8 tracks.

OK, now force a rescan of my Library.

Yep, 8 additional songs were found. And they happen to be the new Snarky ones, still no Nubya.

Hope this helps clarify my description above.

Kind regards,
Jan

[nuwriy]

Hello @Jan_Prummel, could you please upload a copy of the Garcia album here? I’m not seeing this with my own content and I’d like to see what the results are if we’re both using the exact same album. Thanks! Please let me know when you’ve had a chance to upload the album.

[Jan_Prummel]

Hi Nuwriy,

Done. Please delete the data when no longer needed to resolve this issue.
Please note, I have at least one other example of the same type, a purchased album from Bandcamp that is not found by Roon, while the artist was new to my database, like Nubya.

KR,
Jan

[nuwriy]

Hello @Jan_Prummel,

I’ve downloaded the files you sent and added them to a watched folder on my C: then told Roon to force a scan, the number of tracks has increased. I have nothing from this artist prior to this.

I used the search from the same screen and was sure to use the “only show my library” option. Can you move the files onto your desktop in a new watched folder and see if you’re able to find it? The folder structure I tested this with is below.

[Jan_Prummel]

Hi Nuwriy,
That worked, immediately. I didn’t even have to force a rescan.
However, with another album (same issue) it took more than 24 hours before it was discovered.
Also, whenever I move either album back to the main location (NAS) they’re not discovered.
Do you think it will be possible to move them back eventually?

There was a similar issue with an artist (Marcelo Woloski) of which I (or iTunes) had misspelled the name (both artist and album artist). This had as consequence that the artist’s only solo album wasn’t found by Roon. Not even a (correctly spelled) track from the album, while he was recognized as collaborator on other artist’s albums. Until I moved the complete directory to the test location.

So now I end up with the uneasy feeling that some of my CDs are not correctly imported by Roon. Which may take forever to find out. I don’t feel that moving dodgy directories to an alternative location is a solution, but more a way to diagnose the issue.

Do you have a longer term solution in mind?

[grossmsj (Scott G)]

Have you considered that your file path might be too long? The file path length would be determined by the total number of characters in the whole file path plus the title. In some systems there is a limit.
If files aren’t found when they are at the bottom of a long file path, but are found when they are moved up near the front of the path, this could be your issue.

[Jan_Prummel]

That is an interesting point, Scott.

I checked the whole database and the longest path (by far) is this one:

“Volumes/Media/Music/SqueezeboxMusic/Music/Compilations/Rodrigo_ Concierto De Aranjuez, Fantasia Para Un Gentilhombre/05 Rodrigo_ Fantasia Para Un Gentilhombre - Espanoleta Y Fanfare De La Caballeria De Nápoles.m4a”

It is found by Roon and at 214 characters it is more than twice as long as the longest path of the Nubya Garcia album that wasn’t found (103). So this is probably not the limiting factor, but I appreciate the notion that it might be somehow OS, rather than Roon, related. Can you think of another possible cause?

[grossmsj (Scott G)]

The other major thing that comes to mind are ‘forbidden’ characters. Another user had a similar problem and it seemed they have a couple of characters that kept the file from being recognized.

Roon does not see new music in NAS

Thank you for this real example of a problematic path. Just in case you didn’t know: Not using characters documented as to avoid in file and folder names might help too. https://kb.roonlabs.com/Skipped_Files#Special_Characters https://kb.roonlabs.com/FAQ:_Some_of_my_files_aren’t_showing_up_in_Roon#Special_Characters You can still use them in the metadata of your files.

I noticed that the Nubya files are .m4a. I’ve seen that people will sometimes have problems with some .m4a files being recognized and that seems to get fixed by transcoding them to another (similar) format like FLAC. I sort of doubt this is the problem you are having. If that was the problem, Nuwriy would have had a problem reading it too.

[Jan_Prummel]

Thanks for the link to the previous thread and the article, Scott.
I have indeed found a few illegal characters in my database ($ and `), but surprisingly, all files have been found by Roon.

I’ve trouble (in Excel, where I’ve pasted all file paths and separated all sub directories by selecting ‘/’ as delimiter) to find ? and * (those are found in all cells) and ~ (Excel doesn’t respond with the popup that it can’t find it, yet remains in the current cell, even after clicking “Find Next”). I’ll check it in a text editor next.

[Jan_Prummel]

Hi @nuwriy, @grossmsj,
Just checked everything else (’~’, ‘?’, ‘*’). None of these are in my database. I can rename the files with a ‘$’ or a ‘`’, if that would make the whole database more robust, but since these files were found without issues in Roon, I don’t believe that this is the problem I’m facing.
Any other ideas?

Related to this: Is there a way to export all files that Roon has indexed from the Library into a text file (favorably of file paths or file names)? In that case I can try to automatically search for omissions.
KR,
Jan

[nuwriy]

Hello @Jan_Prummel, you can export to Excel here.

[Jan_Prummel]

Thanks @nuwriy,

That export function is very helpful.

It turned out there was only one more issue in the 9000+ song library: Roon misidentified Santana (the one with the lion’s head line-art) and Santana III as the same album. But that was easily resolved.
And magically, all albums that had to be stored in the alternative location are now known to be alternative versions, hence correctly found in the preferred location.

Although there was no solution that actively corrected the faulty situation, Roon ended up with the desired setup.

Thanks for your help,
Jan