Roon not showing thousands of classical files

[David_McAughtry]

@support Hi. While waiting for the new Roon Android support to enable playback on my V30 phone, I have another issue. I have an extensive and very well tagged collection of around 80k tracks, split between Classical, Jazz, and Modern categories. In the Classical collection there are 20k tracks, and I’ll use Bach as an example. There are 3k Bach tracks comprising 500 or so pieces, each arranged in a subfolder. I have turned off Roon tags and turned on file tags in settings. Roon has imported all the tracks as shown below (the folder in question is at the bottom of the list:- “2CFLAC”, i.e. Classical FLAC).

However, when looking by composer at ‘Bach’ it only shows 17 Bach pieces, one of which is (hilariously) a Dusty Springfield track (at the bottom).

The actual content of the Bach folder as shown in MediaMonkey shows all the title, composer and genre information is correct and in place for this folder.

Can you offer a suggestion as to what could be happening and how to fix this?. It goes without saying that my tagged collection can be seen and managed perfectly in other collection managers like MediaMonkey or Jriver, and the tags are 100% visible and correct in MP3Tag. I have probably another dozen issues with Roon in using it to actually play Classical music, but with this problem, the other issues are not worth trying to address.

A general comment:- Roon as a collection manager seems unusable to me - certainly for Classical music It is astonishingly lacking in categorization, search and playlist function, and completely opaque in it’s own choices. But as a music player system, Roon is astonishingly brilliant. It finds every item of audiophile equipment on my network, and simplifies and makes it all reliable. As an example, it not only makes the Devialet Air protocol usable on my Devialet amp, it provides a phone or tablet remote control for it - something that will probably never be provided by Devialet themselves. It also makes Chromecast reliable and usable. A superb piece of coding. Why is the collection manager so bad, when the playback system is so good? Here’s a suggestion. Split the two parts of the Roon program into separate products, and let users like me run grown-up collection managers (like MediaMonkey or JRiver) at the back-end with Roon’s brilliiant playback system at the front end. That would be the perfect solution. At the moment Roon is only usable for me as a very expensive bolt-on to Tidal. I’d really like to use it for my personal music collection as well.

[Geoff_Coupe]

You’re probably not going to like this, but I suspect that the way you have structured your folders are making things very difficult for Roon to make much sense of your content.

Roon seems to prefer a folder structure along the lines of /Artist/Album/ - probably because an “Album release” is one of the primary objects in Roon’s object graph.

I’m not sure how you can help Roon make better sense of your content without some restructuring, but perhaps someone else has some suggestions to make. Flagging @support for you.

One other observation, is there some reason why you have defined the storage paths to such a depth? It seems to me that you could replace the five Storage path definitions shown in your screenshot with just one: This PC > I: > Music.

[xxx (Banned - toxicity)]

@Geoff_Coupe

is there some reason why you have defined the storage paths to such a depth?

It’s that whole folder/subfolder thing.

[David_McAughtry]

Thanks to Geoff_Coupe and Slim_Fishguttz (!) for helpful tips. Much appreciated. There seem to be three suggestions here - Folder structure (Geoff), the level of nesting (both) and the number of top level folders (Geoff).

I am somewhat puzzled by all of these. I turned off Roon’s tagging and turned on file tagging for all key values, including ‘composer credits’. I also checked that Roon could handle multiple sub-folders (which it seems to do). And I expected that when Roon had been told to prefer the file tags, that it would indeed then use the file tags. Roon imported all 19k files, so it clearly saw every sub-folder. It also saw every composer. It just then failed to use the tags for composer that it had been told to use (for 90% of the Bach and other composers).

I have multiple subfolders in general because each folder contains a different format: e.g. FLAC, HD-FLAC, MP3 (for where there is no Lossless version). However, the entire classical collection is in only a single folder, 2CFLAC. The reason I added multiple subfolders instead of a single ‘music’ folder for my Roon import is that I have many more folders/formats in my ‘music’ folder than I am importing for this test. However I can rerun as a trial with only a single top-level folder. Non-classical files have imported fine by the way using exactly the same folder structure.

On nested storage paths, they are there for apps like Phones where indexing is even worse that Roon’s and it is easier to search by folder than by index. But they are not complex: this is the sub-folder structure for the 2CFLAC folder.

A folder structure of Artist/album is problematic for Classical Music, and further confirms to me that Roon is a long way from optimised for searching classical music collections. When searching a very large Classical collection (and mine is about double what is currently loaded on Roon -this is just a trial), I want to search for Composer and then Genre (or vice versa) and then Composition and then possibly Performer. For example I might want a specific set of the Bach Piano “Goldberg Variations” when my Bach collection alone has at least 4k tracks/1000 compositions to wade through, so I find Bach, then the Piano genre, and then easily locate the Goldberg variations (see above). Having found that piece, I might then choose the performer if there are multiple versions (which is not frequently the case). In the above structure, note how quickly I can get to the Goldberg variations, out of hundreds of compositions. On the other hand with an Artist/Album structure, searching for say I first have to remember who the pianist was for the Goldberg Variations, then look him up by artist among the 5000 or so artists in my collection, and then through all his albums for a specific Bach piano piece. No-one wants to do that.

Because my master collection is managed by MediaMonkey, I can easily do a test to see if a single top-level file works, or if less nesting works. But neither should be needed if Roon just picked up the tags, like it says it will.

Both of you mention the “folder/subfolder thing”. Is this documented by Roon anywhere?

[dylan (Dylan Caudill)]

Hey @David_McAughtry — Thanks for sharing your report with us, and apologies for the difficulties here.

First, we took a look at the Dusty Springfield track that is showing under Bach. This is some bad metadata from one of our sources and we have submitted a correction to them. We will be sure to update you when we receive a response from them.

For the trouble importing your library, Geoff’s comment above is correct. First, I would like to echo his suggestion that you use a more general storage path for your watched folder (i.e. This PC > I: > Music). Additionally, in order for everything to properly import into Roon you’ll need to rework the structure of your files. For best results, try structuring your files in the following manner:

.../Artist/Album/

What I’d like to recommend for starting this process is pick out an album or two that you’re having issues importing and place them into this new, single album folder structure and import it to Roon. If you’re still having troubles with importing them we can investigate further as to why those tracks are not imported, but you will likely see your issues resolved with this structure change.

We also have an article detailing some of the reasons that tracks may not be imported into Roon that I recommend checking out. Unsupported files, corrupt files, special characters, and blacklisted folders can all cause tracks to not be imported. You can check also find skipped files by going to Settings > Library > Skipped Files.

Thank you,
Dylan

[Geoff_Coupe]

@David_McAughtry

A folder structure of Artist/album is problematic for Classical Music, and further confirms to me that Roon is a long way from optimised for searching classical music collections.

I’m sorry, but this is just a wrong assumption on your part. The point about having the …/Artist/Album structure is that it is the easiest for Roon in recognising what you have in your collection during the import process.

Once that is done, then object entries corresponding to Albums, Composers, Releases, Artists, Compositions and more are created in the Roon database. You can browse the object graph in many different ways, not just by Artist or Album, and you can use the Focus tool to slice and dice your collection along many dimensions to create smart playlists.

I’ve said it before, having a folder structure of …/Album/Artist is the best way to oil the import process of Roon to ensure that everything gets ingested quickly and as fault-free as possible. The old joke about “what’s the best way to get to Tipperary?” applies here.

[David_McAughtry]

@support. OK, I have taken all the steps as suggested above. First I cleaned out the dtabase according to this advice: “Remove all the extra watched folders and leave the default one there (I actually removed every folder). Make sure there is nothing in the default watched location. Close and re-open Roon. Go To Settings, Go to Setup, at the bottom.click Library Maintenance. Check all that apply and then hit Clean Library. Should clean out all the old stuff. Also, make sure your Tidal account has been removed.”.

I then rewrote all 3000 Bach files to a single folder called “Roon test” with the artist/album structure, and imported only that folder into Roon. Here is the file structure in the folder:

And here is that folder as shown in the Roon/settings/storage page - note that every file has been imported:

All the Library import settings have been turned to ‘prefer file’ (including ones you can’t see here like genre):

And the result is this time only 2 compositions are shown. Amusingly one of them, I assume randomly, is Bach’s Goldberg variations, the example I used above:

There are no skipped files:

Every file in this directory plays in MediaMonkey, JRiver and iTunes/iPod - there are no corrupt files. I have been playing this collection for years with zero problems in many other music systems. All these files show up properly in MP3Tag and have correct ID3 tags as shown in MM and Jriver. There are no forbidden characters in the file names as shown in your link Dylan.

Even assuming anyone should have to use the artist/album structure when Roon should JUST BE READING THE TAGS, I find it hard to agree with the contention that it is “is the best way to oil the import process of Roon to ensure that everything gets ingested quickly and as fault-free as possible”. It manifestly isn’t here. Moreover it is patently clear that album/artist is unsuitable for classical music. Here is an example of a music file with that structure - it exceeds the Windows file name length limit. For simple music where ‘Bob Dylan’ is the artist, the album/artist structure is possibly fine. But not for this:

That’s what classical music is often like. Long artist names. An orchestra, conductor, soloists, a choir. Get the picture?

So none of the suggestions above work. Any thoughts?

[anon47919701]

Would you mind sharing a Roon screenshot of the actual album(s)? My guess is that the album is not recognized / identified and therefore Roon cannot match compositions. Composition identification only happens when there’s at least one identified album with a composition present.

[David_McAughtry]

Sure, here is a MediaMonkey listing of the albums.

These are by the way not albums as in whole CDs, but instead whole pieces, like “Mass in B minor”. Actual physical CDs have filler tracks often by different composers and frequently present entire pieces out of sequence to optimise available CD space. So my ‘Albums’ are whole pieces/compostions in numbered sequence where applicable, not the titles of commercially available CDs. This is exactly like Tidal or any other streaming/download service, where I select just the composition, not the album. I naively assumed that Tidal would use the tagged ‘album’ name, because I told it to do that. Does it actually ignore ID3 tags?

[dylan (Dylan Caudill)]

Hey @David_McAughtry,

I wish to offer my sincere apologies here — I believe I misunderstood your original post. To clarify, these tracks are in Roon, they’re just not showing as compositions under a specific composer (in this example, Bach), correct? You’re able to navigate to these tracks via the track or album browser but not via the composer?

-Dylan

[David_McAughtry]

@support. Dylan, that is correct. All the tracks are there, and with Albums and Artists correctly identified. BUt not by Composer. With 30-40k files, a simple track listing is not a viable method of finding a particular piece, let alone setting up a playlist.

[Geoff_Coupe]

I’m afraid that your screenshot shows that your “Albums” are not what Roon considers “Albums” - they are your composition names, taken from your ID3 tags. So Roon has not in fact been able to identify the majority of your collection, and relate it to its known database of albums, composers, artists and releases…

When Roon identifies an album, the Track Browser will show something like this:

And here’s the corresponding album for the Bach Partitas…

[WiWavelength (Andrew J Shepherd)]

Folks, forget it. This is a wild goose chase. David does not want a library of albums; he wants a library of artists and works of his curation that he can browse by folder. Roon does not do that. Like it or not, Roon is an album based library.

AJ

[anon47919701]

If the file tags would contain WORK and MOVEMENT tags Roon could at least show this. I don’t know if Mediamonkey supports those file tags and offers an easy way to mass copy this information into the respective tags.

[David_McAughtry]

Geoff: there is no commonly used ID3 ‘composition’ tag. There are ‘Work’ and ‘Movement’ tags via MP3Tag. None of these tags are mentioned anywhere in the Roon settings I have seen. The ‘Album’ tag and ‘Composer’ tags are by far the most commonly used. . My Album data uses the correct ID3 ‘Album’ tag. The data appears as ‘Albums’ in the Roon track listing above.These and the Composer tags are present and correct in the file metadata andare in absolutely standard ID3 tag format. They should be picked up by Roon because that it what is says it will do! Why does Roon not put the ID3 file tag 'Composer into the Roon 'Composer field, and the ID3 'Album tag into the Roon ‘Album’ field. How hard is that?

No other music libray system that I am aware of has a ‘Compostion’ tag. MP3Tag does not show this tag either. if what you are saying is that Roon takes the Album tag and ignores Composer if it cannot match the Album with a commercially available compete CD, then fine. Roon iwould be utterly unusable for my purposes. Maybe OK for yours. But I don’t need Roon to tell me that Bach weas a baroque composer, or the history of the Goldberg variations. I know this. And my tagging is better than Roons – I have invested man-years in it. I would have liked to use Roon for it’s exemplary player services to access my entire music collection, without having to retag 40k tracks. It’s clear that it can’t be used for this - and it’s only currently suitable as a very expensive Tidal add-on.

I think I am done. Nothing Roon can deliver is worth the effort of a) figuring out from their almost complete lack of usable documentation how to make it work and b) the enormous sacrifice of adapting to their eccentric and unweildy data model and retagging tens of thousands of tracks.

But I would like to thank you for your interest and helpfulness. The forum editors and support team, along with the outstanding music player system (but sadly not the data structure) are the genuinely excellent parts of the Roon world.

[Geoff_Coupe]

Yes, I’m afraid I said at the outset that Roon can’t make sense of your data model. You’re using the ID3 Album tag to hold the title of a work, i.e. the composition. And yes, to move from your model to that of Roon would be an enormous amount of effort on your part. I can totally understand why you do not want to do this.

[David_McAughtry]

Geoff - thanks for your help. Sadly it looks that way. I would say that many serious classical collectors use this data model. if you want to play a playlist of complete classical pieces, or see the complete sequential works of a composer, or use a central data store for several music playing systems you pretty much have to use a model like this. Roon’s inability to easily shuffle albums (or compositions) would have made the playlist job difficult anyway (was going to be my next hurdle).

Thanks once more more your patience.

[David_McAughtry]

AJ, FYI, your characterisation is not at all correct. If you have used other music libraries on large classical collections you would know exactly what I want. Which is a library searchable by Composer/Genre/Album(Work). Roon does all of that except it fails to associate the Composer in the ID3 composer tag with the album data in the ID3 album tag. This is despite Roon accepting that the Composer data is taken from the file. There may be a reason for that - but it eludes me.

In general I am almost completely uninterested in Artists or Folders. When you play a piece e.g Sibelius’ 3rd symphony, you don’t think, ‘what album by Simon Rattle will I play this evening?’ You search for Sibelius, Orchestral, and then Symphonies. Then you find the Rattle version. Much easier for Pop/Rock/Jazz, which is where I guess Roon is aimed.

Doing this properly is not difficult on any other music library. But Roon has an opaque and not easily discernable method for handling data. I am irritated because had it been made clear I would have not wasted so much of my time on it.

However, I give you one thing. It’s a wild goose chase. Roon, as so many others have found, is not a platform for serious classical music collectors, and it’s certainly not worth wasting your time and probably not mine either on this fools errand.

[WiWavelength (Andrew J Shepherd)]

David, your characterization of Roon organization as “eccentric” and “opaque” is off base.

How would you organize LPs, CDs, etc., on a shelf? You cannot slice and dice up physical media just to suit your organization dictates. You have to file by album. That is what Roon set out to do in a virtual sense and does quite well. As an added benefit, Roon navigation and search are leaps and bounds better than scouring shelves.

Anyway, “serious classical music collectors” are like the Gelgamek Catholics – an edge case. Roon is not likely to change its rational philosophy to suit an edge case. “South Park” gets it right: “Maybe we just need to forget about the Gelgameks for a second and focus…” on what makes sense for the vast majority.

AJ

[David_McAughtry]

AJ, Roon is a digital format and digital allows what you cannot do for physical CDs - to split out works from albums. Its been common practice in digital media libraries for a decade. Why in a new digital product, pretend that the only formats are physical CDs? The world has moved to streaming and digital downloads already. And you may be surpised to hear the benefits of a digital library system are well understood, and have been for a decade in numerous other, grown-up music library systems. Roon navigation is leaps and bounds behind most of these unfortunately. What Roon has is nice metadata, and a matchless playback system. Shame I can’t use it except for Tidal.

And I am amused by your ‘edge case’ Gelgamek Catholics analogy. Roon postions itself as an audiophile system and charges £450 for a lifetime license. I would call that pretty’edge’… It has full support for the max few hundred users of Devialet Air. Also fairly ‘Edge’. It is working on a fix for the handfull of LG v30 phone users (like me) who have raised the issue of support. Also quite Edgy… It supports Tidal users. Not exactly mainstreaming.

You might regard classical music listeners as weird, and ‘edge’ but they are as big a market as Jazz or any other mainstream genre. But no matter - It is clear Roon is for simple collections of simple music. So be it…