ROON painfully slow after update with large library sizes [Roon Investigating]

[Andreas_Philipp1]

@connor

issues that, combined, affect large databases.

I want to contribute an observation about performance issues with large databases, with a little insight gained from long-term monitoring of my Roon Core. My database is right now about 270k tracks, with abut 56k local tracks, and the rest contributed by Tidal and Qobuz. It is running on a dedicated Linux server using an i5-8600K and 16 GB RAM, OS and Roon on a Samsung 970 EVO Plus NVMe SSD.

With just reproducing music and using Remotes to search and browse the albums, the performance is very good. No problem at all with that. But, as soon as I intend to save another album from Tidal and/or Qobuz into the database, Roon server starts processing the Tidal and/or Qobuz storage library, a very heavy hitting process, which brings one of the processor cores to 100% and eats up lots of RAM. While this process runs, the performance of the Roon Remote is severely impacted. Interactions with the UI (button press, search, browsing albums, etc.) give no immediate feedback and can last up to several seconds. Music playback is not impacted by this.

The storage library processing time will increase with a growing database. And it will be triggered by near every intent to save new albums from streaming services into the storage database, or to delete albums from the library. It also will run ever longer with increasing running time of the Roon server. A restart of Roon server does speed up this server process, but as long as this server process is running, it will impact the performance of the Roon UI. This in my experience is the single most performance-impacting process running on the Roon server, and its impact is not really noticed until after the storage library exceeds a certain size. I wasn’t bothered by this, until my library passed 100k tracks, 56k of which corresponded to local tracks.

The log entries for the running process look like this:

03/28 17:13:53 Debug: [music/storage] [Qobuz Storage Library] processed full update for 202:0:143727497
03/28 17:13:53 Debug: [music/storage] [Qobuz Storage Library] processed full update for 202:0:143727498
03/28 17:13:53 Debug: [music/storage] [Qobuz Storage Library] processed full update for 202:0:143727499
03/28 17:13:53 Debug: [music/storage] [Qobuz Storage Library] processed full update for 202:0:143727500
03/28 17:13:53 Debug: [music/storage] [Qobuz Storage Library] processed full update for 202:0:143727501
03/28 17:13:53 Debug: [music/storage] [Qobuz Storage Library] processed full update for 202:0:143727502
03/28 17:13:53 Trace: [clumping] clumping 24 tracks and 0 auxfiles
03/28 17:13:53 Trace: [library] finished with 24 dirty tracks 1 dirty albums 3 dirty performers 6 dirty works 6 dirty performances 3 dirty genres 24 clumping tracks, 0 clumping auxfiles 0 compute tracks, 0 deleted tracks, 24 tracks to (re)load, 0 tracks to retain, 0 auxfiles to (re)load, 0 auxfiles to retain, and 43 changed objects
03/28 17:13:53 Trace: [music/searchindex] [search-index]  removed in 0ms: 1 albums, 24 tracks, 0 works, 0 performers, 0 labels, 0 genres
03/28 17:13:53 Trace: [music/searchindex] [search-index] added in 2ms: 1 albums, 24 tracks, 0 works, 0 performers, 0 labels, 0 genres
03/28 17:13:53 Trace: [dbperf] flush 106523 bytes, 98 ops in 19 ms (cumulative 22955306714 bytes, 20986507 ops in 3319639 ms)
03/28 17:13:53 Trace: [library] endmutation in 85ms
03/28 17:13:53 Debug: [music/storage] [Qobuz Storage Library] processed full update for 202:0:5039501
03/28 17:13:53 Debug: [music/storage] [Qobuz Storage Library] processed full update for 202:0:5039502
03/28 17:13:53 Debug: [music/storage] [Qobuz Storage Library] processed full update for 202:0:5039503
03/28 17:13:53 Debug: [music/storage] [Qobuz Storage Library] processed full update for 202:0:5039504
03/28 17:13:53 Debug: [music/storage] [Qobuz Storage Library] processed full update for 202:0:5039505
03/28 17:13:53 Debug: [music/storage] [Qobuz Storage Library] processed full update for 202:0:5039506
03/28 17:13:53 Debug: [music/storage] [Qobuz Storage Library] processed full update for 202:0:5039507

[Shih_Sheng_Yang]

Thank you Andreas, your findings meet exactly I was suspecting for couple weeks. My 12-core Rock server with 64G memory has been working hard for two weeks importing 16T music files and it is ruining very slow once the tracks passed 100k. Even with the latest update I am still waiting for library importing to no avail. Roon is particularly unplayable at any given time. My fingers are keeping crossed many times with roon. I think the best way is not to use Roon with large music database.

[sandmanct (Joe)]

I ran ROON without ROOEXTEND and it was still slow even after rebooting.

[sonic_fanatic]

I am having the same issue and it’s great to see all this corroboration.

Roon ARC not showing any data being used with albums on hard drive ARC

[Screenshot_20230327-083656] Description of issue The key reason i went lifetime is failing. Roon ARC is not even downloading data and it’s saying i have a poor connection. A 3.9 on the android store is kinda a fail @brian. You should have kept this in beta longer and got it working well. [Screenshot_20230327-082932] [Screenshot_20230327-082920] You’d see little data bubbles for the app if it was downloading data. Yes i restated, etc. No, don’t ask me to start from scratch considering I’m…

Large Collection. Slow search

I seem to have a similar problem with search and ARC, and similar library size. Is there a thread with recommended hardware? Since Roon is continuously optimizing, what and how much do you need? Are there folks who have figured this problem out for ARC and search? Could the info be compiled somewhere?

If you try and do more than one thing in Roon, it all breaks. Can we have a running known issues list?

A lot more say and discuss on the items raised here and I’d still encourage folks to go lifetime, as I don’t see anyone going after this niche in the way Roon is. [image] So I had to unplug on my computer laptop to take a call. I plugged the Dac back in after and when I tried to play it just hung with the time bar moving back and forth and not playing. I skipped to the next track, then stopped by clicking play, and then double-clicked play and it worked immediately.

[feimei_liu]

too much music。

[Bernd_Ruths1 (Bernd)]

After some weeks with all the posts still no solution. Behaviour of Roon and RoonServer with a very large library is still very sluggish. Although there are times when library check and importings of new albums are done, Roon does its job much quicker. But truely not as fast as before this update to Roon 2.0.11. So working in parallel, what means importing, playing music and doing the audioanalysis makes it thus sluggish. Before 2.0.11 this was better possible, although things slowed down remarkably then, too. But after importing was finished, Roon 2.0.10 worked flawlessly.
I had a proposal to the Roon team:
I have still a database with Roon 2.0.10, just before the update to 11, where somehow Roon had to overwork the database because of some new feature. If I had Roon 2.0.10 or 2.0.9 for Mac, I could check out if behaviour is still good in this version. The team then could have a special look at the changes done from 2.0.10 to 2.0.11, that Roon made sluggish. Maybe the mistake is hidden in the overworking of the database?
So Roon team, what do you say? Could you send me Roon 2.0.9 or 10 for Mac?

[Gert_Taeymans]

Hi all.
I started with Roon and see the same behavior, but already at a much lower level of track count: the slowdown happened at around 29000 tracks. It may now take 10-25 minutes to add a single entry.

Tracks are on a USB3-connected drive.

Hope this may provide an additional datapoint.
Gert

[sandmanct (Joe)]

Any word on a solution? This has been going on for some time now and ROON is essentially unusable.

[Elling_Jacobsen]

So when will this happen?

[Noah_Lopez]

working through a lot of threads with the same unusable Roon issues that I seem to be having. replying here because I only have 200K+ tracks, but with a heavy focus on FLAC and 320mp3 files and have the same issues. wondering if there’s a way to disconnect Arc for troubleshooting…

[Jacky_Le]

The same issue for me start from ROON ver 2.xxx. Not sure if ROON team have looking on it to help fix issue. Evertime take 20~30 sec to load one album is terrible experience. Before same Music size ~120K track, each time load play just < 1sec.

[sandmanct (Joe)]

Still waiting on a solution. So do we have a time frame to expect a resolution?

[Gert_Taeymans]

Eventually, this issue closed for me. I changed my HD.

I had my collection on 2 locations:

1. A networked NAS
2. An external USB-connected HD. Roon would go on and on adding and identifying.

Then I added an M.2 internal HD. I copied all data from the external USB drive over (took a few hours only).

• I disabled the external USB HD location in Roon
• I added the new M.2 HD as a location in Roon.
• Roon then proceeded to add all 46K tracks in a few days.
• Today it finished and Roon now performs perfectly.

Hope this helps those still struggling.

Gert

[sonic_fanatic]

I don’t think this is the issue as I assume the OP has an adequate internal HD that holds Roon and the tracks may be on an external HD. 500k and 48k of lossless tracks is 10x different. I may try ARC again soon, but I’m not hopeful.

[Robbert_N]

Any news? It’s getting worse and worse.

[Bernd_Ruths1 (Bernd)]

Dear Roon team:
It‘s nearly 4 months since I and some others with big libraries, maybe especially on a Mac can no more use Roon the way one wants: just listen to music. I have since then begun to listen to music very rarely because of the sluggish behaviour in Roon. See: when something is too hard, you avoid it and you begin to loose interest…
I would say I am patient. But after going the Roon way in fall 2020, my first 2 years with Roon on a Mac there was the missing native .net problem. No Mac user was really informed about that and you, the Roon team covered this problem. This wasn‘t fun either because Roon often or mostly skipped the tracks I wanted to play.
For my part, I only had real fun with Roon the time 2.0 was launched in October 2022 until February 14th 2023.
When will you launch a version where these problems are solved? I am really kind of fed up with that…

[HeavyMetalDrummer]

Just to pile on, I’ve been really disappointed with how sluggish that Roon has been since sometime around the launch of ARC. My library is 95K tracks on a Nucleus+ (upgraded to a 980 Pro SSD and 32GB RAM), most of the content is on an internal SSD, and some on Qobuz and Tidal. My main issue is searching and filtering on tracks (see support topic I submitted this week below). And I also have sluggishness when I select an album to play - it can take 30 seconds or more to start playing.

I’d love to see Roon focus more on performance and optimization rather than introducing more new features.

Poor performance when filtering/searching in Tracks view

[Norman_Spector]

Hi @danny suggested I clean up library which I’d never done and I found 5780 tracks. Could this have been a significant factor in slowing updates?

[Jacky_Le]

Hi team. Finally I figureout Solution is buy PC with strong performance. I try NUC12 core i7 16g ram and M2 pcie ssd with super high reading speed. For who have more than 120k tracks

[Kroon]

I have the same problem.

I reported my problem in this thread, and had no support.

Roon extremely slow with high CPU usage

Hi @Kroon, There are routine automatic diagnostics associated with your Roon Core that communicate performance issues to our server. They don’t contain any private or personal information, only a record of your Core’s requests to Roon’s servers. If local storage is failing to load, for instance, the Core will give a generic loading failure trace. We’re happy to answer any questions. In this case, diagnostic reports showed your Core did not have access to the list of partitioned drives necessar…

I confirmed that the newly-installed Roon with the new database works so smoothly on my machine. So, definitely, the problem lies in handling of a big database by Roon.