EDIT 4/2018: this post was written in June of 2017, and since then, weâve made a number of optimazations to Roon. The number that was previously stated as 300k is probably closer to 550k now.
First, some context
Itâs important to clarify: libraries of this size are extremely exceptional, and will always require special care. The group of people with 300k+ track libraries represents a small handful of people amongst our user base.
Off the shelf solutions simply do not apply for people in this situation. For people with huge libraries, we have always recommended the same thing: Get the fastest consumer-grade i7 you can, at least 4 cores, 16GB of RAM, a 256GB+ NVMe SSD, a big fan to cool the thing, a nice closet to put it in, and a locally connected storage array for the content, and then run Windows 10.
For the rest of us who donât have extreme librariesâŚ
Consumer operating systems like Windows, OS X, and most popular Linux distributions do not create the best headless appliance experience. They have frequent software updatesâsome of which canât be skipped. They upgrade packages one at a timeâwhich creates a risk of unintended breakage. They have a HUGE footprint of unnecessary stuff consuming resources on disk, in RAM, and running in the background. This makes them good general purpose OSâs, but not the best for a media server appliance.
This is why we created ROCK. It doesnât contain extra junk unrelated to the task at hand. It is not general purpose. It doesnât have a package manager upgrading packages all the time, and it doesnât run a bunch of stuff in the backgroundâjust the minimum needed to fully experience Roon. When ROCK is updated, the whole OS is updated atomically and in a âbrick-proofâ manner, just as appliances should be updated.
ROCK is very different than what youâd get if you started with a full-sized operating system like Ubuntu, Debian, Fedora, OS X, or Windows Server and tried to trim it down as much as possible without breaking it. The total installed size of ROCK is under 70 megabytes, including the kernel, root filesystem, and drivers. This is a small percentage of the minimum size of an OS X, Windows Server, or Ubuntu install. I think 70MB is actually smaller than the Windows Kernel alone
By starting clean, and avoiding all of that extra stuff in the first place, we can create an appliance-style experience, make things smoother, snappier, more trouble-free, and easier to test and support.
ROCK, Roon, and mono
The Roon Coreâon all non-windows platformsâutilizes the mono runtime, just as it does on other Linux (and OS X) platforms. Mono is shipped as part of Roon, not as part of ROCK, and ROCK uses the same Roon packages as any other Linux user is using.
When it comes to library management at extreme library sizes, the limitations that people encounter primarily result from monoâs performance characteristics, not Roonâs (or ROCKâs).
The thread I wrote above goes into a lot more detail, but Iâll quote the most relevant part:
.NET was created by Microsoft, so it should not be a surprise that the best performing implementation of .NET is currently available on Windows.
For non-windows platforms, we use a community-developed .NET implementation called mono. This is a mature and stable project thatâs been around nearly as long as .NET has. For almost its entire life itâs been stewarded by various businesses associated with the projectâs founder, including Ximian, Novell, Xamarin, and as of recently Microsoft.
Yes, Microsoft. Microsoft acquired the company that stewards the mono project recently, with the expressed intent of leveraging monoâs technology in order to directly support .NET on all platforms. So the prospects for .NET on Mac, Linux, Android, and iOS are looking pretty bright.
Today, mono is good enough that virtually all of our users donât have to care about what platform they run on. Looking towards the future, there is almost certainly going to be convergence between Microsoft and Monoâs offering, because they are now being funded and developed under one roof.
Unfortunately, when a Roon library gets up into the several-hundred-thousand-tracks range, it pushes monoâs memory management infrastructure so hard that bad behavior starts to result. Symptoms might include laggy screen loading, slow focuses/searches, audio dropouts, or other stuff like that. When the framework gets messed up like this it is literally standing in the way of the code we need to run to get things done.
A catalyst for this problem is the richness of Roonâs data modelâ500,000 tracks means weâre also keeping track of 50,000 albums, 750,000 performers, 20 million credits, and so on. Compared to most other music apps, we manage at least 10x more dataâso we are starting from a more difficult position.
Both mono and Roon have gotten a bit faster since I wrote that, but mono is still not yet on par with Microsoftâs .NET runtime when it comes to the extreme memory management scenarios that arise when managing the largest libraries.
Be careful trying to generalize these ideasâŚ
Some of you, after reading that, are taking away the idea that âWindows performs betterâ. When it comes to managing extreme library sizes thereâs definitely situations where it does, but in other areas, the same reasoning doesnât hold.
We see better performance/stability out of the Linux scheduler than we do on Windows. Code related to DSP or communication with audio devices is generally a lot simpler and bit fasterâboth because the compilers we use on Linux seem to do a better job at optimizing our audio processing code, and because the audio subsystem on Linux is more streamlined and simpler. Unlike with library management, these areas of the system do not make such heavy use of .NET/mono, so they are not subject to monoâs performance characteristics.
On top of that, ROCK adds an extra layer of benefits. First and foremost, the benefit of having a very small root filesystem, RAM footprint, and set of background processes/components. This creates a very quiet operating environment, where Roon has no worries about outside interference, or competing for system resources with other stuffâItâs difficult to get near this point when starting with a general purpose OS, even after heavy trimming.
In short, ROCK remains our first choice for all but the most extreme library sizesâthe ones that are so large that they would require careful hardware/platform choices regardless of the circumstances.
In ClosingâŚ
Itâs unfortunate that this a handful of people with extreme library sizes arenât able to fully benefit from ROCK, but the experiences of these outliers should not necessarily inform the decision making of the rest of us. Their situations are unique enough that special handling is required regardless.
On all platforms, there is a point where increasing the library size to the extreme results in performance degradation. On mono-based platformsâOS X, ROCK, other Linuxesâthat tipping point comes a little bit earlier. The work to close this performance gap is happening as part of the mono projectânot as part of ROCK or Roon. As mono gets better over time, the gap will shrink, and the âtipping pointâ will continue to increase.
Butâif youâre on the right side of that tipping pointâand most people areâwe believe that ROCK is the better choice.