Roon has trouble indexing a copy of my iTunes music library

[Quinten_Kroes]

Core Machine (Operating system/System info/Roon build number)

Windows 10/Intel i7/Roon 1.7 build 511

Network Details (Including networking gear model/manufacturer and if on WiFi/Ethernet)

wired connections, Roon core is connected over fiber to Netgear GC110 switch->ethernet to router->ethernet to QNAP NAS TS-251B

Audio Devices (Specify what device you’re using and its connection type - USB/HDMI/etc.)

Chord Electronics Hugo TT2 connected over USB directly to Core, various other networked devices

Description Of Issue

I’ve built up a library of around 9K tracks in iTunes over the last decade or so, including many CD-rips to .wav. These are indexed in the usual iTunes way [artist]/[albumname]/[tracknumber_trackname], in a directory called iTunes Music. Mostly these are complete albums, sometimes a selection. The iTunes library was stored on a NAS. When I first installed Roon a couple of years ago, I installed the core-software on my networked iMac and pointed Roon to the music directory on my NAS, which includes the iTunes library (music files plus iTunes library xml and itl files). I did not encounter any major indexing issues then.

Earlier this year I decided to upgrade to a dedicated core, so I built a Windows based audio PC from scratch, including two M.2 drives, one for Roon and one for my music library. Once the PC was up and running in my network, I installed the Roon core software on the first drive and made a full copy of the iTunes Music directory (only the music files, not the iTunes XML/ITL files) onto the second M.2 drive. When setting up this new PC in January, I pointed the new Roon core to the music directory on the second M.2 drive. I did not make it watch the original drive on my NAS, as I thought this would only result in duplicates (I wanted the NAS to only serve as a backup, and no longer as a streaming source). Having done that, I noticed recently that Roon was missing many albums, entirely or partially, even though they have been copied in full on the second M.2 drive.

To see if it made any difference, I added the original iTunes music folder on my NAS as a ‘watched folder’ on my new Core PC. This made some difference:

• my Core did seem to find some - but not all - albums on the M.2 drive of my Core-PC that it had not found initially;
• for the ones it still did not find, it did see some of these sitting on my NAS, even though an identical copy of the same album was also sitting right on Core-PC itself.

I had hoped to make a clean start by having Roon index my library anew, without relying on iTunes metadata, but so far this has resulted in a degradation of my Roon library.

I assume this problem has something to do with the iTunes way of indexing, and the fact that .wav files don’t include metadata, but unfortunately there is no consistency to what I’m seeing. Any help would be much appreciated.

Best regards,
Quinten

[dylan (Dylan Caudill)]

First, I’d like to point out some documentation we have that may be helpful here. In our KB we have an article about Skipped Files and an article about why some files might be missing from Roon .

If you go to Settings > Library > Skipped Files do you see any of your missing files there?

I’d like to recommend looking at a couple of specific examples of tracks that did not get imported into Roon. Are they all .wav or or there any other file types? Are they showing up in Skipped Files? If it looks like they should be importing, let me know! Screenshots of these files in their storage location would be helpful for discovering what may be happening.

[Quinten_Kroes]

Hi Dylan, thanks for your response.

My skipped files only contains a handful of songs, so that doesn’t explain it:

Here is an example that illustrates the problem.

Here are the identical .wav files for one album on both my new Roon core PC (called ‘q-cat’) and my old NAS (‘TS-251B’). Remember that my entire library of music files was copied from the NAS to the new PC:

For the album on the Roon Core PC (q-cat), it only found two songs which also happened to be part of a playlist:

For the album on the NAS (TS-251B), it found the full album:

If I search for a missing song by title (e.g. Sex on Fire), it lists the song as ‘03 sex on fire’ on an album called ‘Kings of Leon - Only By the Night’ by ‘Various Artists’:

Opening up that so-called album results in an enormous list (355 hours) of apparently orphaned songs (in .wav format) that all start with 01 in their title:

If I look at the path for the first song listed here (‘01 01 Free - Prelude’), this is what it shows:

Hope this helps in your analysis. Let me know if you need more info.

Kind regards,
Quinten

[dylan (Dylan Caudill)]

Thanks for the screenshots, @Quinten_Kroes. Definitely helpful.

Can you share a screenshot of Settings > Storage as well so I can see how your content was added?

Let’s take a look at one of the other tracks in that album — 01 01 Free - Prelude. Where is this stored? Can you share a screenshot via Finder?

Just to be clear — Are all of your albums in their own folders in iTunes? Can you share some screenshots of the structure in Finder?

[Quinten_Kroes]

Hi Dylan,
I took a drastic approach over night trying to make a fresh start with the music library on my PC, by deleting this in its entirety, and then transferring music library using iTunes (so instead of just copying the music files), after having disabled the drive for Roon. The procedure for using iTunes to move the library to another network drive that I followed is described here: https://www.pcmag.com/how-to/how-to-move-your-itunes-library.

That seems to have gone alright over night. But having now re-enabled the drive on q-cat for Roon, and Roon having completed its new indexation, I don’t see any major improvement. The examples I gave still persist.

Here is the screenshot you asked for from Settings>Storage:

Here is the screenshot for the 01 01 Free - Prelude file from Finder:

By the way, if I doubleclick on that file, it starts playing just fine in iTunes. If I search for Jill Scott (of whom I own quite a few albums) in Roon, it only finds The Light of the Sun, which is the only album in .aac format, all the other ones being CD-rips in .wav.

To answer your final question, yes, all albums reside in their own folders. Here is another screenshot:

And this screenshot from Roon shows the current directory path:

If I search for the second track on “Experience_Jill Scott +” in Roon, it finds the entire album as such (with the artist and album name lumped together in the album name) but under ‘unknown artist’:

Here is a screenshot of the path to the second track:

If I try to manually edit the metadata through Roon, it cannot identify the proper album, even after I add Jill Scott as the artist. But that could just be because this particular album is unknown in Roon?

I hope this helps to sort this out, because I really can’t think of any cause or solution…

[nugget (David McNett)]

I’m not sure if this is a factor, but it’s really weird that the track number prefixes on the iTunes media files are doubled up. It appears as if you named each track including the track number “02 Gotta Get Up” and then iTunes is of course adding it’s track number prefix as well (corresponding to the track number recorded in the iTunes database) resulting in the track numbers being doubled up.

I wonder if this is just Roon not being able to recognize the album because it doesn’t know anything about a song named “02 Gotta Get Up.” It’s expecting the track name to just be the track name “Gotta Get Up.” Roon can obviously cope with iTunes-styled filenames where the track number prefixes the track name, but perhaps not when it’s doubly so. I think maybe you’re seeing artifacts from Roon doing it’s best pulling artist and album information from the full path of the file, but ultimately not doing a great job because the track names aren’t what it expects.

It’s weird, in any event. I’d suggest renaming the tracks in iTunes to remove the extraneous track number from the name and seeing if that helps.

[Quinten_Kroes]

That makes sense, I’ll try that and report back.

Given the huge number of files which have the tracknumber in their name, can anyone recommend any editing tool or app that works more efficiently that me doing it manually in iTunes? Perhaps MediaMonkey or Yate?

[nugget (David McNett)]

I’d say try one or two albums then make Roon re-scan the library. See if it helps. No sense worrying about how to do it en masse until you know it will make a difference.

Since you’re on a Mac, minimally it wouldn’t be too tough to craft a shell script that can at least identify all the tracks that would need to be renamed. I expect that would take a lot of the work out of the process.

[Quinten_Kroes]

I’ve removed the track numbers from the Jill Scott album ‘Experience: Jill Scott +’, and from another album that had the same double numbering. Here is the info from Finder:

And here is the confirmation from iTunes:

However, the album still doesn’t show in Roon (if I force a re-scan, it just says that Roon is watching realtime). If I search in Roon for the song ‘free - prelude’, it still finds a track called ‘01 01 free - prelude’ by an unknown artist and gives the following file information:

Notice that there is no File Path listed anymore, and when I try to play it nothing happens.

But maybe we shouldn’t get too fixated on tracks with double track numbers. A far more typical example is the following:

Here is another example of a file path of a track that shows up in Roon as ‘unknown artist’, ‘unknown album’:

D:\Music\iTunes\iTunes Music\Music\Michael Jackson\Off The Wall\01 Don’t Stop 'Til You Get Enough.wav

If I search for this song, it finds an album “Michael Jackson - Off the wall” by an unknown artist and lists some but not all of its tracks:

[nugget (David McNett)]

Bummer, I was hoping that would be it. Sorry, I’m all out of ideas. Hopefully someone from Roon Labs with a better understanding of the underlying code will have more ideas.

[dylan (Dylan Caudill)]

Hi @Quinten_Kroes,

Using the Jill Scott example above, can you share this media with us so we can take a closer look? You can send it via a shared Dropbox link or any other file sharing service link in a Private Message.

[Quinten_Kroes]

Hi Dylan,
I sent you a PM with a Dropbox link.