Large Collection (280,000 Tracks) Slow search

The suggestion was meant to be either/or. Not both. That is switch audio analysis off entirely whilst importing and then rely on audio analysis on demand whilst playing. This could be complimented by temporarily switching on audio analysis (12 core) overnight whilst sleeping or otherwise not listening to roon.

OK. Iā€™m not really listening to Roon much at all. Iā€™ve previously had things on ā€œthrottledā€ (not completely off) - would that cause this slownessā€¦? Iā€™ll turn the ā€œon demandā€ analysis ā€œoffā€ now (since Iā€™m not listening anyway), and leave background on 12 cores. Iā€™ll report back tomorrow.

The difference between one thread and 50 threads was night and day for me.

No. Thatā€™s the opposite of what you should be doing.

  1. Turn audio analysis off completely. No audio analysis will then be slowing down the importation process. Throttling is also good but you may as well just turn it off completely.
  2. You can keep audio analysis on demand switched on as it will have no effect until you are playing something after the importation process is completed.

Regardless, there must be something odd about your set-up that no one else is doing.

You say you have 70,000 various artist tracks. But from various posts you seem to be using thousands of tags to group them into compilation albums. If that is what you have done, roon would not be expecting that.

Do you have a lot of compilation albums of the following form?

My Various Artists Album (Disk 1)
My Various Artists Album (Disk 2)
My Various Artists Album (Disk 3)
etc.

What roon expects is that every track has an identical album title. That is ā€œMy Various Artists Albumā€. You then tag each disk number field as 1/3, 2/3, 3/3 (roon will interpret correctly many similar disk number conventions like 1of3, 2of3 . . . etc.) and put each CD in a separate sub-folder, CD1, CD2, CD3. Then roon knows it must keep the tracks together as a VA album. No other tags are necessary for roon to keep VA albums together, however large or complicated.

I had things pretty much as youā€™ve suggested here for several weeks. Apparently, there are two lines of thought here, and Iā€™m going to give the alternative a try for a day or two.
I have various artists in album/disk/track name format. Here are ā€œmy tagsā€:

How many albums do think you have ā€œmergedā€?

Hundreds. Roon seems not to recognize subdirectories that contain separate disks. There are hundreds more that have not been ā€œmergedā€ (and I merge them as I see them).

Wellā€¦ After I made this screenshot (about 15 minutes later - everything goes quite slowly here with Roon) My Tags filled up with box sets. There are 80 on a page. I didnā€™t go past the first page, so there may be many more of these. Should I delete them all?

Backup made, all tags deleted, server restarted. Letā€™s see how it goes from here.

Fingers crossed.

If roon is not recognising your CD sub-directories as parts of various artist compilation albums and box-sets then the most likely thing is that you are missing one or more necessary steps.

It shouldnā€™t be necessary to use these hundreds (thousands?) of ā€œcompilation albumā€ tags as a workaround for organizing your 70,000 various artist tracks. There is a standard procedure for ensuring that roon will correctly group various artist compilations and box sets with sub-directories into albums:

  1. All album titles in the individual tracks must be identical. It is common to use conventions like
    My Various Artists Album (Disk 1)
    My Various Artists Album (Disk 2)
    My Various Artists Album (Disk 3)
    etc. . .
    This doesnā€™t work in roon. All tracks must have identical album titles otherwise in this case roon will make 3 albums.
  2. Group the tracks into CD1, CD2 CD3 ā€¦ subdirectories.
  3. Tag the disk numbers 1/3, 2/3, 3/3 (or something similar like 1 of 3, 2 of 3, or 3 of 3 . . .)

It is not sufficient to just do step 2). Roon will not consistently auto-group tracks into various artist compilation albums and box-sets if that is all you have done.

I know from my own experience that if you do not follow the procedure above you will definitely experience fatal performance issues with large various artist directories of box sets and compilation albums. Chopping the elephant up into smaller alphabet sized chunks or adding Musicbrainz idā€™s from Songkong will help in the importation process but it is marginal until you address the above.

There is another complicating factor in your case in that in an attempt to organise your 70,000 tracks you appear to have resorted to a complex system of hundreds (thousands?) of box-set ā€œtitleā€ tags. You can see that it takes time for roon even to populate your tags page so there is a clear performance hit there. In all likelihood it is also a contributory factor to slow search. Over the years there have been several reports of complex systems of tags, particularly nested tags impacting search performance in much smaller libraries than yours.,

But as far as the slow importation is concerned I rather suspect it is a red-herring until you address the above disk-set organization procedure. Once you have addressed that, then by all means delete those tags as your VA albums and box-sets should then be grouping correctly and the tags would not then be serving any useful purpose.

If you do decide to delete those tags, just remember that unless you have restructured your 70,000 various artist tracks along similar lines to the above (or Songkong has already done something similar) then you will loose any organizational structure of your various artists albums within roon. So re-structuring your VA and box-set directory is really a pre-requisite.

2 Likes

Iā€™m confused here. I have zero tags (says Roon) because Iā€™ve deleted them all. There were no Roon tags created by my merging albums that needed to be merged. I checked that by finding a few more unmerged albums - and merging them - and there are still no tags showing in Roon. I think the only tags that were showing in Roon were inserted by Song Kongā€™s ā€œRoon Profileā€ for box sets, and all of those have been deleted - and the problem persists.
Another issue is that the ā€œold Roonā€ had zero problems with my file structure (except for failure to merge multi-disk sets, which I fixed with the ā€œmergeā€ function).
Another bit of confusion is with the numbers related to ā€œAdding Music to Libraryā€ (tracks / added / and identified change when you restart the Roon server), Background analysis speed (with the number of tracks being analyzed not synching with any other number that I can see), and the actual number of tracks in play (Iā€™ve not yet started the scans for various artists albums starting with U, V, W, X, Y, or Z). Here are the screenshots (as of a minute or two ago):
Adding


(This following one is a bit truncated, but you get the idea):

You deleted them in Roon, or, did you remove the RoonAlbumTag metadata that the SongKong software embeds in the actual audio tracks - As described in this post?

If that metadata is still there then Roon will try and recreate all those Tags again.

Just in Roon. Iā€™ll take them out using Song Kong too, but because this problem was there before the tags were I really doubt this will do anything at all.

I have removed all Roon Album Tag tags that were inserted by Song Kongā€™s Roon profile, using Song Kongā€™s Meta Grater. I then restarted the Roon server, just to give everything a fresh start. Sorry to say, that did nothing to speed things up.

Posted here way back in March:

Looks like I was too Optimistic! Still slogging away with no end in sight.