Large Collection (280,000 Tracks) Slow search

From ROON support to me while dealing with my issue:

The issue is more around how Roon treats file structure, in combination with how your library is built. In a simplified schematic of astandard music storage branching file structure, a directory will resemble Main Folder → Artist Folder → Album Folders. Thus, speaking generally, Roon will anticipate subdirectories to contain around 10-100 tracks. Core algorithms (analysis, importing, etc) are optimized for that range of tracks in a subdirectory.

Obviously, there’s tremendous flexibility built into this calculation. Subdirectories can have dozens, hundreds, and even thousands of tracks (think playlists, etc.) without breaking or slowing down Roon. But consider the average track count per subdirectory in your Watched Folders. When users have large libraries - particularly libraries that are stressing local storage or Core operating system capacities - then subdirectories with a huge average track count is the straw that breaks the camels back.

Any suggestions on the ideal hardware and operating system? I have HiFiBerry DACs and Amps in 5 zones following the Rock, and would want to control those as I have been doing with Roon for many years.

I have received word from Roon support, and it is crystal clear now that they DO NOT peruse the community discussions. In fact, when invited they WON’T look at these community forums. Here’s what I was just told by support:

@Ronald_Lyster, the thread you referenced is a #roon discussion thread. If you require assistance post in #support, and complete the template in the opening post.

For ease, I’ve pasted this below:

Please be sure to include details of your core, network, storage location of your media files, and any messages displayed by Roon. Thank you.

Roon Core Machine

<!‐-Include your operating system and machine info (Model, CPU, RAM)-‐>
<!‐-Write below this line-‐>

Networking Gear & Setup Details

<!‐-Your network gear (model of routers/switches) and if on Wi-Fi/Ethernet, are you using VPN-‐>
<!‐-Write below this line-‐>

Connected Audio Devices

<!‐-Specify what devices you’re using and their connection types, like USB/HDMI/Chromecast, etc.)-‐>
<!‐-Write below this line-‐>

Number of Tracks in Library

<!‐-Tell us how large your music library is, e.g. “30,000 tracks”-‐>
<!‐-Write below this line-‐>

Description of Issue

<!-‐Tell us about the problem you’re having in as much detail as possible. Screenshots are always appreciated!-‐>
<!‐-Write below this line-‐>

Including these details in your first post helps resolve your issues faster.

after reading community guidelines I was “axe grinding” instead of praising.
So as a the kids like to say…“My Bad”

@thomas_clark1
Just to clarify your recent posts and subsequent removal.
This was NOT Roon Labs doing, this was the community and the Roon Volunteer Moderator actions.
They were not really helpful to the ops present situation and deemed off topic.
Please consider your posts carefully especially when in someone else’s thread who is looking for assistance.
Let’s try and stay on topic and to the point here please.
Thank you for your patience and understanding.

I just wanted to point out that the person (Martin) that responded to your support thread is not part of the Roon support team, he is a community (volunteer) moderator. In fact the Roon team is taking the long weekend off, I wouldn’t expect to hear anything back from them today.

I would suggest that, rather than expecting them to read through this lengthy thread to dig out the nuggets of information that they would need, that you make it as clear as possible the issue that you are having and everything that has been done so far at the top of that thread. Concisely summarizing will undoubtedly help whoever responds get a clear understanding without having to read this comparatively long thread.

Best of luck to you.

1 Like

A support thread has been created so there should be no need or requirement for Roon to wade through this thread.

Sad. So much has been said here about what could be the problem, and you would think the Roon techs would be able to straighten everything out. This could be an educational process for everyone - including the Roon techs. Can they make their product better? Is anyone else having the same problems? What ideas do their users have that are correct or are wrong? Can the community possibly be helpful - thinking of things that may not immediately come to mind? Are their users losing faith in their product? Should they be? If not, why not point it out? Should we organize our files in a certain manner? Should we switch the core machine when our music collections get too big for Rock? Can Rock be improved? Should their users know of future plans? So many good reasons for the Roon techs to pay attention here.

I’ll wait for the support team’s suggestions before resetting my database. I haven’t been playing much music from my home audio system lately.

So thankful for all the discussion here. We WILL get to the bottom of this eventually.
Could I have bad import settings? Here’s what I have now:



I assume its not realistic to reverse all that de-duping at this stage but if hardware upgrades are not having much of an impact it may be worth looking at how your library is organised on disk. Roon is very album oriented. Your track to folder ratio of about 6 seems very low as another poster has commented. Mine is more than 3x that. In general I have not meddled with album structures so that I have relatively few unidentified.

What is the average track count of your various artist compilations? I usually break mine into 20 track sub-folders. For example I have a large 80’s folder of mp3’s built up over years which I subdivide this way. Roon seems to have a practical limit of abut 180 tracks in a subdirectory anyway after which it will just cut off the display. I know from my own experience that before breaking up large hand-built compilations I was having performance problems.

That’s for sure. You’ve got an average of 18 tracks per album? Seems pretty high.

I’ve disabled my “named artists” tracks. Let’s see what Roon reports as the number of albums left in “various artists.” (Be back soon.)

Here are the “various artists” only stats:

18.76.

That is the top end of that article but I think roon is calculating in terms of albums, rather than CD’s. I have a lot of double CDs and also larger box sets as half my library is Classical and that is how Classical is packaged and sold. Classical is essentially a “covers” genre so I have a lot of duplicates but I have left the album structures as is and rely on roon’s library organization features to hide the duplication.

That is even smaller at +4 tracks per album. You seem to have a lot of very small albums relative to tracks. I don’t know if that is causing a performance hit. Roon would need to comment. What I do know is I never de-duped tracks. I have comprehensive tags and very few unidentified albums, compositions, artists and composers. I also have a larger library than you, a much higher album to track ratio and very modest hardware. But I don’t experience your performance problems.

1 Like

Hit tracks taken from albums leaving an incomplete album. Maybe

Roon will still try to ID the album and fail as not all tracks complete

Is this the case ?, Roon will thrash away trying to ID , as you are experiencing. Roon works well on Albums but fails even iif a couple of track times don’t match

Just a thought but that drive is your Various Artists drive? You have mentioned several times that your folder structure drills down to CD1, CD2 . . . etc. The average track to album ratio on that drive is 4.35. But does it contain a lot of multi-disk albums where CD’s now have just one or two tracks or even entire CD’s deleted?

I wounder if your unidentified albums are concentrated on this disk?

I am curious. What does SongKong do with all the dups? Just delete them from disk?

That speed increase was only temporary. Nevertheless, these are good questions. I do have a folder with duplicate tracks (not enabled for Roon to search). Not all duplicates are there, as de-duplicating has been going on for literally decades - never caused a problem before, so many of the duplicates were simply deleted. I mentioned before that Roon sees my subdirectories for multi-disk albums as separate albums, so I’ve been forced to “merge” these. I’ve got many, many more that are still counted as separate albums (I got lazy, I suppose, and with things now slowing down the process takes a long time).
As for empty directories, I’ve also been in the habit of running “empty folder nuker” which finds and deletes those.

This must be a new behavior for Roon, since I’ve been using Roon for many, many years now and this has never been a problem before.