Build 1670 - massive GC problems

I’m seeing ballooning memory usage on B1670 that over time is bringing Roon and my server to its knees. I’ve had playback dropping out periodically, and this afternoon had the Roon remotes on iOS or macbook unable to find the server and unable to see any items in the queue. Server runs on an 8GB Mac M1 Mini (pretty much dedicated to Roon Server at this point given how it’s been going from bad to worse).

FWIW, restarting the Roon Server process dropped memory use from >11GB to <2GB.

Here’s Claude’s analysis:

RoonServer v2.70 (build 1670, macOS earlyaccess): sustained GC pressure, ~150-200x log volume increase, correlates with playback/connection drops

Setup: Mac mini (Apple Silicon, macOS 26.5.2), RoonServer running headless under launchd (not the GUI Roon.app), local library ~34,000 tracks, Qobuz-linked. Core has run continuously as this build since 07/02. Two persistent zones (a grouped RAAT pair, plus a MacBook system-output zone), a couple of always-connected API extensions.

Summary

Something changed between the last release I ran without issue (v2.67, build ~1662-1662, through early June) and the current earlyaccess build (v2.70, build 1670, installed ~07/02). The server now runs with:

  • Managed heap permanently in the 7-16GB range (was 90MB-536MB total, ever, under the old build)
  • GC consuming 30-65% of wall-clock time, sustained for hours at a stretch (was 0.04-0.1% at steady state, 2.5-6.6% only briefly at cold start)
  • Individual full-GC pauses of 6-13 seconds routinely, spiking to 21.5 seconds at the worst observed point
  • Log volume roughly 150-200x higher per hour than the same server produced in May/June
  • A log line category, Trace: [library] endmutation in Xms, that is now ~91% of all log output and did not exist at all in six weeks of logs (Apr 27 - Jun 9) from the prior build

This is not a “GC is a little worse” complaint — it’s a qualitative change in behavior, and it correlates directly with two real incidents where playback stopped and the Roon remote app showed “Waiting for your Roon Server.”

The hard numbers: before vs. now

I have full RoonServer logs from the prior GUI-based install (Roon.app, same machine, same library) spanning 04/27 through 06/09, and current headless RoonServer logs spanning the last ~16 hours before I restarted the process. Sampled across 8 files from the old install and one representative file from the current one:

Period Build Span Log rate (bytes/hr) Managed heap (start->end) GC % (start->end) [library] trace lines
04/27-05/01 2.65 (1654) 78.1 hr 107,983 90->364 MB 5.41%->0.08% 0
05/13-05/16 2.66/2.67 (1657/1659) 73.8 hr 113,411 91->536 MB 6.50%->0.10% 0
05/25-05/27 2.67 (1659) 35.1 hr 140,344 91->221 MB 5.63%->0.05% 0
06/05-06/06 2.67 (1662) 33.7 hr 124,611 56->267 MB 2.70%->0.06% 0
06/06-06/09 2.67 (1662) 62.3 hr 135,608 264->385 MB 0.06%->0.07% 0
07/10 (current) 2.70 (1670) 0.4 hr 20,995,928 8,348->7,607 MB 15.12%->15.23% 21,941 (91% of file)

Every one of the six old-build samples shows the same shape: GC% is elevated for a few minutes at cold start (the JIT/library warm-up), then settles under 0.1% and stays there. It never once climbed back up over a multi-day session. Managed heap tracked with library size and stayed under half a gigabyte the entire six weeks.

The current build’s “resting” state (a routine 24-minute sample, not an incident window) is already at 15% GC time and multi-gigabyte heap. That’s the baseline now, not a spike.

Is the log volume itself climbing, or just chronically higher?

Worth separating two different signals rather than conflating them:

Raw log line volume, hour by hour, across the full pre-restart window (from the actual combined log, not sampled):

07/09 22:00   29,884  (partial hour - window starts 22:08)
07/09 23:00   88,819
07/10 00:00   90,341
07/10 01:00   96,234
07/10 02:00   56,589
07/10 03:00   30,991
07/10 04:00   30,717   <- the first incident (below) falls in a LOW-volume hour
07/10 05:00   44,428
07/10 06:00   89,157
07/10 07:00   80,289
07/10 08:00   88,592
07/10 09:00  122,649   <- peak hour
07/10 10:00   57,347
07/10 11:00   37,732
07/10 12:00   34,468
07/10 13:00   47,872  (partial hour - restart at 13:47)
07/10 14:00    4,439  (partial, post-restart)

This is noisy and bursty (roughly 30K-123K lines/hour), not a clean upward trend. It doesn’t, by itself, prove an accelerating leak — it’s just as consistent with “this build’s baseline debug-trace verbosity is simply much higher than the old build’s,” a flat-but-elevated new normal rather than something actively worsening minute to minute.

Managed heap, however, does show real drift within this same window, independent of the log-line noise above: it sat in the 7.0-8.9GB range for the first ~11 hours (22:00 through roughly 09:00), then established a new floor of 11.8-13GB from 10:00-12:45, then climbed to 12.3-15.7GB in the final 45 minutes before I restarted the process. That’s a genuine, if gradual, upward drift in the heap’s resting floor over the observed window — each GC cycle appears to reclaim less than it should, leaving progressively more “stuck” memory behind. This is the more trustworthy signal for “is this getting worse over time,” separate from the flatter, burstier log-volume picture.

15-minute-bucketed heap/GC/error correlation, 07/09 22:00 - 07/10 13:45 (pre-restart)

This covers the full window I have logs for. Columns: peak managed heap (MB) and peak single GC pause (ms) in that 15-min bucket, average % of wall-clock time spent in GC, and counts of artwork-fetch failures / API timeouts / RAAT-or-API disconnects / critical exceptions.

window_start     max_heap_mb  max_gc_pause_ms  avg_gc_%   artwork_fail  timeout  disconnect  critical
07/09 22:00           7613          10521       51.16          0          0          0          0
07/09 23:00           7962           9758       48.32          0          0          0          0
07/10 00:00           8100          12655       51.10          0          0          1          0
07/10 01:00           8765           9952       52.28          0          0          0          0
07/10 02:00           7957          10748       51.11          0          0          0          0
07/10 02:30           7161           5953       31.81          0          0          0          0   <- brief dip
07/10 04:00           9893          17689       42.16          0          4          3          1   <- 1st incident
07/10 05:00           6955           6070       32.02          0          0          0          0
07/10 06:00           7754          11447       57.69          0          0          0          0
07/10 09:00           8223          12544       64.02          0          0          0          0   <- GC% climbing
07/10 09:15           8348          12567       65.74          0          0          0          0
07/10 09:30           8560          12459       66.09          0          1          0          0
07/10 10:00          11779          12393       56.19          0          3          0          0
07/10 10:15          12964          10545       41.49          0          0          1          0
07/10 13:15          12297          10588       36.74          0          0          0          0
07/10 13:30          15718          21542       51.70         14          9          9          1   <- 2nd incident (peak)
07/10 13:45          14060           9840        8.22          0          2          0          0   <- process killed/restarted at 13:47
07/10 14:00           1210            821        0.46          2          0          9          0   <- fresh process, healthy
07/10 14:15            926            126        0.31          0          0          0          0

(Full 66-row table and raw per-line extraction available on request; omitted here for length — every 15-min window across the full 15.75 hours shows max GC pause between 5,953ms and 21,542ms and avg GC% between 8% and 66%. There is no window in the entire dataset where GC pauses were in a “normal” sub-second range.)

Two things worth separating

1. The sustained condition itself (7-16GB heap, 30-65% GC time, 6-13 second routine pauses) is present across the entire window, including hours with zero warnings or errors. This is the chronic problem, and it’s what’s driving the ~150-200x log volume, via the [library] endmutation trace category — I count roughly 8-9 of these per second, sustained, for the whole 15+ hours. That volume of small library mutations did not exist under the prior build at all (0 occurrences across 6 weeks of old logs). Whatever’s causing this many per-item mutation commits is the actual bug; the extreme logging is just its visible side effect.

2. The two acute incidents (04:00 and 13:30) are where a single GC pause got long enough (17.7s, then 21.5s) to exceed whatever timeout the RAAT and remote-API layers tolerate, and things visibly broke. The 13:30 event is fully captured in the raw log:

13:40:53  Warn: failed to get artwork url: Object reference not set to an instance of an object.  (x6)
13:40:53  Error: [raat/tcpaudiosource] connect failed: Object reference not set to an instance of an object.
13:40:53  Trace: [raat] [Naim Audio Mu-so @ ...] => Disconnected
13:40:55  Trace: [raat] [Linkplay Technology Inc. WiiM Ultra @ ...] => Disconnected
13:42:26  Trace: [roonapi] [apiclient ...] CONNECTION TIMEOUT
13:44:36  Critical: scx: System.InvalidOperationException: An attempt was made to transition a task to a final state when it had already completed.
13:44:36  Trace: [raat_ll/client] [...] OnDisconnected: BeginRead read count is 0
13:44:42  Warn: [...] RAAT Session timed out during initialization

This is what surfaced as “Waiting for your Roon Server” in the remote app and playback stopping. Note the NullReferenceException on artwork-URL lookup appears right at the start of the cascade — worth noting for anyone debugging this, though I can’t confirm from logs alone whether it’s a cause or just an early casualty of the same GC stall.

What fixed it (temporarily)

launchctl kickstart -k to force-restart the RoonServer process. Immediately after: heap dropped to ~170MB (climbing normally as the library reloaded), GC% dropped to near-zero, all zones and the remote app reconnected cleanly within about a minute. Given the heap’s rising resting floor over the ~15.5-hour window I have data for (7-8.9GB → 11.8-13GB → 12.3-15.7GB), and that the process had been running continuously since 07/02 with no restart until I forced one, this looks like something that keeps drifting upward the longer the process runs rather than settling at a fixed (if elevated) new baseline — but I only have this one window of data, so I can’t rule out that it plateaus somewhere past 16GB.

Ask for other users seeing this

If you’re also seeing the “GC percentage climbing” and rising physical memory reports referenced in other threads: check whether your RoonServer_log.txt is dominated by Trace: [library] endmutation lines (a quick grep -c will tell you), and whether your build is also 2.70/1670. That would help confirm this is build-specific rather than library-size- or config-specific.

@Monty_Kosma you mention extensions - consider disabling them one at a time to see if it improves the situation.

I installed an extension on July 10 at ~17:30 and see a significant increase in the number of mutation events with many endmutation log lines per second as you describe. You can see the jump in logged events below.

There’s approximately an 8x increase in the number of DB queries after installing the extension.

I’m going to disable the extension this afternoon and see what changes tomorrow.

With 8 GB of RAM there’s not much wiggle room before your Mac Mini starts thrashing from garbage collection events. I have a similar issue but with 32 GB of RAM there’s more runway before experiencing the same symptoms you describe, but my system will start thrashing in day or two.

Thanks for feedback and observations. As I mentioned here we are working on our next EA which will be loaded with perf improvements. Your feedback will be incredibly useful once it will go out.

@Monty_Kosma

Like all users of this version, I also think I am a victim of it.
Unfortunately, I no longer have all the logs, but I observed this increase as well:

11/07/2026  12:23         8411078 RoonServer_log.01.txt
11/07/2026  09:23          877130 RoonServer_log.02.txt
10/07/2026  16:09         7521875 RoonServer_log.03.txt
10/07/2026  08:28          419958 RoonServer_log.04.txt
10/07/2026  01:31          638976 RoonServer_log.05.txt
10/07/2026  01:30         8425339 RoonServer_log.06.txt
10/07/2026  01:09         8031706 RoonServer_log.07.txt
09/07/2026  10:38          385977 RoonServer_log.08.txt
09/07/2026  04:16         3293184 RoonServer_log.09.txt
09/07/2026  04:11         8435255 RoonServer_log.10.txt
09/07/2026  04:00         8435554 RoonServer_log.11.txt
09/07/2026  03:47         8435948 RoonServer_log.12.txt
09/07/2026  03:33         8435789 RoonServer_log.13.txt
09/07/2026  03:19         8435801 RoonServer_log.14.txt
09/07/2026  03:04         8435281 RoonServer_log.15.txt
09/07/2026  02:50         8460536 RoonServer_log.16.txt
09/07/2026  00:35         8490677 RoonServer_log.17.txt
08/07/2026  20:37         8470714 RoonServer_log.18.txt
08/07/2026  18:26         8471432 RoonServer_log.19.txt
08/07/2026  15:13         8459944 RoonServer_log.20.txt
11/07/2026  13:34         1360839 RoonServer_log.txt

Which, in my case, caused 2 server crashes:

I had also observed this increase in “endmutation” in the logs.
I am available to provide further information; note that the latest log I have is:
09/07/2026 03:47 8435948 RoonServer_log.12.txt

New EA release is out EarlyAccess: Roon 2.71 Build 1674 and ARC 1.81 Build 422 are Live!
Please try this one out and let us know whether things have improved. Thanks!