Sluggish Performance. Large Collection

You are quite right, I think I got confused with another customers support files.

Okay since all the metadata was removed and many of the songs matched MusicBrainz Song Only first time round, it would definitely be worth running Fix Songs again to see if it matches more albums second time round, would be interesting to even run it a third time to see what effect that had.

Actually you are not quite right, some of the songs still have metadata in, e.g D:\Backups\Music\Various_UV\Vietnam - Songs From The Divided House\Black is Black - (Los Bravos).flac

But most do not.

Not sure how that would have happened, since I did apply Jaikoz’s remove metadata to the entire collection… Anyway, I’ll run fix songs again. I’ve set it to “rematch everything.”

The Mozart Box set may be the biggest (with 200 CDs), but there are others, such as:

  • Various_S\Sony Music - The Classical Collection (30 CDs)
  • Various_T\The 100 Greatest Recordings of All Time (50 CDs)

Yes, but by getting it to work well with the mozart test case it should hopefully work with other boxsets. Can test these as well, but best to just work on one thing at a time.

I’m running SongKong now with “rematch everything.” I will, of course, send support files once that is done. I can then run it yet another time, per your suggestion. Should I keep it on “rematch everything” for that third run?

I would say not.

I’ve reset Roon’s preferences for “file based” information over Roon’s information. That reset has unfortunately added more things to be done with my database, and my bet is that it will delay completion even more - maybe 2 years rather than 1 year :woozy_face: to get back to normal (where I was back in February).

My thought was that I can rely on the Song Kong tags more than those on Roon, which seems to be struggling all the time.

Was this move a mistake?

Second pass on Mozart Boxset directory done, and support files now being created and uploaded.

Third pass on Mozart Boxset directory done - ignoring files that have already been matched. Support files on the way.

Hi,the 2nd/3rd pass idea was my suggestion for the whole Various folder, no point doing anything with Mozart yet since I havent done any more fixes since last time you run against it.

The original idea of removing metadata from T folder was that you remove the folder from roon, remove the metadata to let SongKong rematch without the previous bad matches influencing the matches, then readd to roon for it to reproces, has that finished or still processing?

If you have now removed all metadata from live roon instance then rematched with SongKong then really need to readd to roon and let roon reidentify using the better metadata,or have you just changed data on backup im confused about what you have done.

But you need to rerun 2nd/3rd pass with SongKong to improve things further before re-adding. I’m not quite sure why you suddenly decided to do this all of a sudden

The use file/use roon is a bit irrelevant. I would stick to using roon, but the point is you want the file metadata to be correct and as complete as possible to help roon identify albums more easily, and also so roon can use the file metadata in the cases where itcstill cannot identify album.

The metadata was removed from the backup files for the entire Various Artists directory (in the backup files) before I re-ran Song Kong. That was what was in the support files I uploaded quite a few hours ago. I misunderstood you when you once again mentioned the Mozart sub-directory. Thought that was all you were looking at, and it’s a lot quicker to run fix songs on just that sub-directory rather than the entire Various Artists directory.

Before shooting everything back on the live Roon server, shouldn’t I wait for your latest “fixes” and then run fix songs again?

In any event, I’ll run fix songs on the entire Various Artists directory (on the backup drive) now - I assume I should be ignoring songs that have already been matched.

By the way, nothing is “all of a sudden” with this. I’ve been working on this, following tips of users, installing a more powerful computer, purchasing a better and larger hard drive, etc. etc., since March. Roon has been slogging away all this time! I was a relatively early user of Roon, and all was well until the “big change.” Sure wish we could have that old Roon back. I would hate to make the new Roon start this process all over again. I would gladly give up fancy new features if only I could listen to music as I did back in February.

BTW, Paul, couldn’t I eventually just run a full “fix songs” scan of the live Roon directory (of everything) using “rematch”? Or do I have to overwrite everything with the “treated” files on my backup drive?

Roon has been running 6 days since the last restart. Here’s what’s been accomplished:


Apparently, Roon has “discovered” 20209 tracks (that number ever so slowly increases day by day). Here are the totals (less one sub-directory which contains one boxset with 200 CDs - the “Mozart Boxset”"):

Are you aware that Mozart 225 splits into 29 “Mini” volumes which are ID’ed automatically by Roon and then navigate easily , the biggest is around 20 CDs

This box was discussed in a thread on Search just after the transition of search to the cloud. It was identified as a very difficult set to search simply because of its size. I’ll see if I an find the thread if it helps.

I stopped trying very big boxes in Roon for this reason, this is how it looks in JRMC

Anything over say 50 CD’s is simply too big to be usable in Roon

Hi Ron, maybe miscommunicatrion problem. By “all of a sudden” I meant this statement.

I read it as meaning you had ran Fix Songs and Delete Duplicates on the live Roon drive, this would potentially affect all the songs in your Roon system, but did you just mean you ran on the backup?

If the backup then there there is nothing you need to do currently, just wait for me to fix the SongKong problems I have identified and then try the new version when I make it available.

Regarding Roon updates, it was my understanding you were working on each Various Artist letter one a time, waiting for it to complete before adding next letter. So I m assuming there was 20209 tracks in the T folder, and it has added 20039 of them, meaning only 170 to add, and then that bit should be complete, or does the identification continue, after all added. But looking at FixSongs 137 report ran on your backup there are 11,088 files which is approximately half, although interestingly it does say 3149 songs matched to MusicBrainz which is similar to the 3173 identified by Roon. So if you could give me some more information on this that would be useful.

So in our test on varius artists folder backup we found that if songs have already been added to incorrect albums then rematch is likely to rematch to the same wrong album because the existing metadata will match that new album. So we deleted existing metadata first, this allowed best chance to match to correct album, however the disadvantage is that removing existing metadata (which may be correct) may prevent a match because that good metadata not available to help find good match, and for songs that could not be matched we now have no metadata, whereas before they had some metadata (which may have been correct), although many songs were matched Song Only which means that song metadata was added just missing album metadata. So you will get different results if keep or delete existing metadata, since it is probably only the Various Artists albums that are badly matched it makes sense to focus on only those folders. Whether you do by copying over backup or working on live folders may not make that much difference, but since the metadata for these albums should be removed for best results first I think it is better to do on backup outside of Roon.

Full fix songs done again on Various Artists from the local backup, and support files are on the way to you now. Songs previously matched were ignored. Recall I had previously deleted all metadata except Acoustid and ran fix songs. Here’s the summary:

I had been enabling groups of tracks, as you mention, but the number of songs added by Roon grew to a number in excess of the number of files in the _S and _T directories. I opened up all of the directories except for the Mozart Boxset (which I have placed in its own separate directory) in the hope that at least some progress will be made.

Here are some of the existing storage folders:

My thought is to disable the _S and _T folders and then restart the Roon server. I can then share with you what files are being added immediately after the restart. If it is as has been in the past, several thousand files will be shown immediately as having been “added,” but identification will take considerably longer. The number of files “added” will then continue to creep up day by day. Before I got to the _S directory (leaving _T, _UV, _WX, and _WZ disabled) Roon would complete it’s analysis. Things got slower the greater the number of files added. (All of the named artist directory was added and identified within a day and a half, for example.)

I’ll re-run fix songs on the various artists backup directory.

BTW, “songs added” has crept up over the past few hours.

Why is that then ?

Don’t know… I’ll shoot you an update of the files added and identified after my Roon server restart (_S and _T now disabled.)