Poor performance when filtering/searching in Tracks view

[HeavyMetalDrummer]

Roon Core Machine

Nucleus+ Rev A. Version 2.0 (build 1272) production
Recently upgraded to dual Kingston 16GB RAM chips and a Samsung 980 PRO SSD 500GB PCIe 4.0 NVMe (flashed ROCK and updated with Nucleus code).

Networking Gear & Setup Details

The Nucleus is ethernet connected to a 1G Frontier fiber connection. TPLink gigabit switches and router. Speed is excellent, at least 950mbps at the router and Nucleus and >500mbps from wifi devices.

Connected Audio Devices

UltraRendu and Brooklyn DAC+
Multiple Sonos Amps and Arcs

Number of Tracks in Library

94,000 total tracks, 5500 albums, 1000 artists
84,000 local tracks on a SSD drive in the Nucleus, the rest are on both Tidal and Qobuz

Description of Issue

Roon is performing very slowly in Tracks view. If I apply a Focus criteria or a Tag, or filter a field, it will take anywhere between 5 to 25 seconds to apply. This happens from both my Mac and my iPad. Searching and filtering in Album or Artist view is fine, very responsive. Tracks view is the problem.

This has been occurring for the past six months. I recently updated the SSD to a Samsung 980 Pro, flashed ROCK and had the Nucleus stuff applied and restored my backup. I would say that performance all around has increased, Album and Artist search are almost instantaeous, but Tracks view is still very slow: 5-25 seconds to perform filters and searches. This is better than before the upgrade when the same searches were taking up to 45-60 seconds.

I also notice that a Roon server reboot takes about a minute a a half from the time that the Nucleus reboots and Roon server starts, to the time that Roon is accessible on any of my remotes. It never used to be that slow until about six months ago. It is better now since the SSD upgrade, it was over two minutes before.

I also tried a fresh database, i.e. I changed the name of my database directory and built a brand new database and let it load the 84000 local tracks. I got almost the same performance on the kinds of searches I was running on my original database. And it took the same about of time to become accessible after a reboot. I since changed back to the original database.

So I have a very fast Nucleus with extremely fast internet. So why so slow on Tracks searches? Even on a fresh database?

[HeavyMetalDrummer]

I posted this 9 days ago, can someone from @support reply?

[benjamin]

Hey @HeavyMetalDrummer,

I’m sorry for the delay in getting to your thread! It would be good to review what might be going on through your core diagnostic which I’ll enable now.

If you could, please reproduce some of the poor performance behavior and share the specific date, time, and name of track playing. Thanks!

With that, are you able to provide more details around your local library folder structure? How are things typically grouped?

I’ll be on standby for your reply.

[HeavyMetalDrummer]

Thanks for the reply. I just uploaded a video to YouTube of several searches so you can see what I am experiencing in real time. I did this at 8:45am ET today: Roon Search Demo

You will notice it takes at least 10 seconds for a lot of the searches involving focus and tags and turning favorites on/off. And it took almost 30 seconds to search for “Tom Petty” in the AlbumArtist filter.

As for my folder structure, it is all in one storage folder location in Settings, which is the Nucleus+ Internal Storage folder. See the screen shot below for how its organized. There are multiple top level folders, each is organized by Artist and then by Album, and then the tracks are inside each album’s folder. I am very structured about how I organize my folders and follow all of your guidelines for box sets, etc,

[benjamin]

Hey @HeavyMetalDrummer,

Thanks for the breakdown! I’ll need to take this information to our developers for further diagnostic review. In the meantime, how many watched folders do you have active in your storage settings?

If you disable a few, does performance improve? If you’re able to test this a bit and let me the results, that would be helpful!

[HeavyMetalDrummer]

Here is what my storage settings looks like:

[HeavyMetalDrummer]

Any update on this? As per above - I only have one watched folder which has all the subfolders organized by artists and albums.

[benjamin]

Hey @HeavyMetalDrummer,

I do not have any updates for your at this time. We still need to investigate things further, as your core diagnostic doesn’t point to anything specific.

@HeavyMetalDrummer

I only have one watched folder which has all the subfolders organized by artists and albums.

I would consider splitting this up into a few subfolders and test out what I mentioned above.

If you go to Roon Settings >Library>Clean Up Library, what do you see? Could you share a screenshot?

[HeavyMetalDrummer]

To split up my folder, would I just go to Settings>Storage and delete the folder, an then add back new folders at the subfolder level? Would that cause my whole library to be rebuilt? I don’t want to loose of the edits and tags and playlists I’ve created.

Here’s my library clean up page.

[benjamin]

Hey @HeavyMetalDrummer,

Before anything else, I’d make a fresh backup of your current database - which will save everything as it is now.

From there, disable your primary internal storage folder, and add a handful of smaller sub-folders (you’re correct there) and let me know how things perform.

[HeavyMetalDrummer]

I went to Settings>Storage and I disabled the Nucleus Internal storage folder, and then selected “add a folder”, but it won’t let me select the Nucleus Internal Storage folder to add just selected subfolders. It seems like I can’t select subfolders from the Internal storage location.

[Geoff_Coupe]

That is true - for the internal drive, Roon just takes the top-level folder as the default. It’s only for external (USB) drives or Network Shares where you can navigate down the folder tree to add specific folders as Watched Folders.

[HeavyMetalDrummer]

Thanks, makes sense. I thought that perhaps one issue impacting performance could be that the folders containing my files are two levels deep under internal storage. See image below, diskstation1_1 and RoonLibrary are just mirroring the folder structure from my NAS which is where I sync my files from. I tried to just move all the folders from RoonLibrary to be directly under InternalStorage. I stopped Roon on the Nucleus, moved all the folders, and then restarted. However, when Roon restarted, it began importing all the files again from the new location, adding them as duplicates. So I didn’t want to wait to see what might happen, it would take a few hours to go through 88K tracks, so I stopped it, moved the files back and restored Roon from backup and all was good again. So I’m confused about how I would move my files under a new directory in internal storage without creating duplicates…

Any ideas?

[Geoff_Coupe]

Actually, having multiple levels of folders is not an issue for Roon. What is more troublesome is to have a folder with thousands of entries.

Given that the common arrangement is for a structure of the form:
/Artist/Album/Tracks

Then if you have thousands of artist folders at the top level, it would be better to restructure by creating alphabetic folders at the top level, and group the artist folders alphabetically into these folders.

Not sure why Roon was creating duplicates for you - this might in fact have been a temporary situation while Roon was analysing the new structure. I’ve just stopped my Roon Server on my ROCK/NUC and moved a few folders around, and Roon has not created duplicates of the album up restarting.

It did take a moment or two to update its database with the new locations of the files. First time I tried to play the album, Roon said it failed to load the file (because it was using the old location information). Within a minute it had updated its database with the new locations and all was well.

[HeavyMetalDrummer]

Thanks, that’s interesting. Maybe I should have just let it finish doing its thing and it would have figured it all out eventually. Perhaps moving the folders containing >88K tracks was a lot to do in one move. I could try moving just one subfolder at a time and see what happens.

Regarding the number of entries in a folder, I think that the most I have is one folder that has 830 artist folders. Probably not problematic, but I can try segmenting into several alphabetic subfolders.

[HeavyMetalDrummer]

A further report: I reorganized my folder structure, I eliminated those extra folders at the top level so that all the subfolders are now folders at the top level. There are 11 folders sitting directly under “Internal Storage” . Most are organized by artist/album/tracks. Some are just album/tracks. The largest of the folders had 840 artists in it, so I split it into two alphabetically. None of the folders has more than 400 subfolders. Most album folders have 10-20 tracks. A lot of the multi-CD boxes have subfolders for each CD, but some of them have all the tracks in one folder. So there are at lot that have 30-50 tracks, a few with 100 or so tracks in a single folder, and there is one with around 300 tracks.

After doing that there’s been no change in peformance. Searching and filtering in Tracks is still slow.

As an aside, re: the issue I noted above where Roon started adding duplicate tracks after I moved the folders around. It wasn’t really an issue. When restarting Roon after moving some of the very large folders, it would count up all the tracks it found that were moved, the tracks counter went up dramatically, and then when it was finished, the tracks counter went back down and all the newly added duplicates gradually disappeared and everything was back to normal. The whole process took about an hour.

[benjamin]

Hey @HeavyMetalDrummer,

Thank you for taking the time to thoroughly test the above! I’m also sorry that it didn’t change the sluggish performance. We were able to take another look at a fresh Nucleus diagnostic, but unfortunately aren’t seeing any traces relating to poor performance.

I did see some interesting errors around your Sonos connection. As another small test, could you test out disabling your Sonos endpoints, and see if that changes performance at all?

Outside of this, we’ll need to bring this thread to our development call next week to investigate further. I apologize for not having an immediate solution for you here!

[HeavyMetalDrummer]

I disabled all the Sonos endpoints and just used my MacBook. There was no different in performance.

Interesting about the errors regarding Sonos - I’ve had no issues using Sonos other than the occasional gaps while playing gapless tracks which has been an issue for a long time.

[benjamin]

Hey @HeavyMetalDrummer,

Thanks for the update.

@HeavyMetalDrummer

A lot of the multi-CD boxes have subfolders for each CD, but some of them have all the tracks in one folder.

I’m curious, for your multi-CD boxes, are they all labeled as Various Artists? Roughly how many tracks do you have under ‘Various Artists’?

If you could, please reproduce the issue and share the artist name you are searching for when you experience the decreased performance. Thanks!

[HeavyMetalDrummer]

At 5:55pm ET I went to Tracks view and did a filter of AlbumArtist for “Eric Clapton” and it took about 45 seconds to come back. I took the filter off and then did a Focus on Eric Clapton as the Performer (obviously will return a different result than the AlbumArtist filter) and it only took about 10 seconds. I then did the same on Bob Dylan and it similarly took 30 seconds for the filter search and 10 seconds for the Focus.

The multiple-CD boxes are typically single-artist, except for about a dozen various artist boxes that have between 3-4 CDs each. These are typically organized with all the files in a single subfolder rather than in separate subfolders for each CD. See an example below, this one has 73 tracks in one folder, with the files named as disk/track number/artist/track name.

I have 132 albums in my Various Artist folder. Most are one or two CDs, and about a dozen are 4 CDs. Roon identifies 123 albums as “Various Artist”. There are a total of 2973 tracks in those albums.