Large Collection (280,000 Tracks) Slow search

The collection has been gathered over many years, and there have been many duplicates from various albums. I have routinely used SongKong and Jaikoz and EF Duplicate file remover to remove these duplicates, undoubtedly leaving a number of albums short of tunes. These albums started when I used Squeezebox (many years ago). All are lossless (nearly all flac).

1 Like

Have gigabit FIOSā€¦

Well, maybe not on your but Roonā€˜s side.

Was afraid of that

It may well be that your library is of a size where you are pushing at the limit of what a NUC can do. I note that Roon Labs say this in their article on hardware recommendations:

Largest Libraries

If you have over 250K+ tracks in your library, consider us impressed! Youā€™re among the top .01% of Roon users, and you have a library most of us could only dream of.

With libraries this large, we expect the right hardware will work, but itā€™s definitely not something that we test with in-house. Your best bet will be to get a beefy Roon OS setup with a fast new CPU and plenty of RAM, but a very high-spec system running Linux, Windows, or macOS can work just as well.

We strongly recommend you engage with members of our user community when making any hardware purchase decisions for a very large library. There you will find a number of music collectors, like yourself, who have first-hand experience with getting the most out of Roon when the library track count is best represented in terms of fractions of a million tracks.

As an experiment, could you try halving the size of your library to see what effect that has on performance? If there is a substantial improvement, then I think it would point to the need for a high-end desktop system with a Core i9 CPU, rather than the laptop-class CPUs used in NUCs. If search performance is unchanged then we need to look elsewhere. Perhaps open a support request in the Support category of the forum and ask the support team to look at your logs.

Thereā€™s a report of slow performance on a system that is currently being investigated by Roon Labs. The library size is twice that of yours, so the cause may or may not be similar. You might want to keep an eye on this investigation as it develops:

2 Likes

I seem to have a similar problem with search and ARC, and similar library size. Is there a thread with recommended hardware? Since Roon is continuously optimizing, what and how much do you need? Are there folks who have figured this problem out for ARC and search? Could the info be compiled somewhere?

1 Like

Do you have a large number of Tracks for 1 artist/composer? Like 50k or higher Diana Krall tracks? Or a large number of Tags?

Oh, the horror! :rofl:

4 Likes

I do like Diana Krall, but I havenā€™t specialized in any one artist. I use Song Kong on all of my tracks; all of my tracks are lossless.

I can disable one of the external hard drives - perhaps the one with the most files - and see where that takes us.

As I have said, my collection is divided between two external hard drives - both WD and both relatively new. One drive (the smaller one) has albums with ā€œvarious artistsā€; the larger drive has albums with named artists. The files are organized into directories with the named artist (if not various artists) / album name / disk number (unless it is a single disk album, as are most). I just disabled the larger disk, leaving only 71123 tracks. I restarted the Rock and then tried opening and playing albums and searching for artists. Everything is still painfully slow - perhaps a bit better than before disabling the larger drive, but still unacceptibly slow. I looked at one of the various artists albums and found a tune by Harry Chapin. So I searched for ā€œHarry.ā€ I found quite a few artists with the first name Harry - but not Mr. Chapin. I then searched for ā€œChapinā€ - and found him. Something odd is going on here.

Iā€™m going to re-enable both drives and wait to see if anyone can provide a solution.

Upon re-enabling the second drive, I decided to see what happened if I ran just that larger drive and disabled the smaller one. Amazingly, the speed improved remarkably. When I added back the smaller drive (both drives enabled) things slowed down again.
I can try two things - replace the smaller drive, or try running a new, single giant drive - perhaps Raid 10?

1 Like

What is the format being used on the smaller drive and is it different to that being used on the larger drive? That might give a clue.

And personally, I wouldnā€™t bother about using any form of RAID. Having a backup strategy in place is far more important IMO.

1 Like

I wonder if a lot of non specific artists could be a key

When you say various artists are the individual tracks allocated to a named artist or just a generic ā€œVarious Artistā€

Roon will obviously try to match Arist to the search criteria?

The other issue is VERY big box sets , say 100 + these were causing real slowness a few updates back.

ā€œArtistsā€ are all individually named, but ā€œAlbum Artistā€ is set to ā€œVarious Artists.ā€ I do have a few big box sets. Would it help to make ā€œAlbum Artistā€ blank?

Same ExFat format on both drives.
Isnā€™t RAID faster? I already have a full backup of these drives on a Synology NAS.

Unless the drive is being faulty, speed is not the issue, imho. Could be the other USB port as well. Have you tried just the large drive plugged into the USB port where the 2nd drive would usually go?

Recall that I have replaced the NUC in an attempt to resolve this matter. Speedier, more memory, later version. No change.

1 Like

RAID is about maintaining availability of access to data in the event of disc hardware failure. Itā€™s not about speed of access.