Hey guys, @eric just brought this to my attention, so I thought I’d come and explain where things are.
Disappearing Tracks
We’ve investigated this about half a dozen times over the past year. It’s actually more of a problem with OS X than with Roon (which is why Roon on other platforms does not suffer this symptom), and it is relatively rare, meaning we have many users using network storage on Mac without this symptom, and only a handful who see it. This suggests that it’s in some way correlated with setup details or habits of those users.
The script is basically like this:
Roon: Hey OS X, is the network folder mounted?
OS X: Yup. Mounted
Roon: Ok, tell me the directory listing for my watched folder
OS X: That folder is empty, buddy
Roon: Really? That seems wrong. Are you sure the network folder is still mounted?
OS X: Yup, mounted.
Roon: Ok, tell me the directory listing for my watched folder again
OS X: That folder is still empty, buddy
Roon: Double-check that it’s mounted for me, OK?
OS X: Yup, it’s still mounted. Jerk.
Roon: OK, I’m gonna trust you on this…
Anyways, it’s clear to us that OS X sometimes lies to us about the status of mounted network folders, the contents of their directories, or both.
The trouble is: if Roon thinks the network folder is mounted, but sees the directory as empty, it’s going to assume that those files are actually gone. And then the album count starts ticking down.
We have spent a lot of effort trying to provoke this situation in the interest of developing a workaround, and the chief barrier has been the difficulty we’ve had reproducing it in house, which makes the process of developing and testing a workaround a lot more murky.
We are working on some general improvements to folder watching infrastructure right now–part of that work includes our fifth or sixth attempt to work around this issue. This workaround is more crude than the past ones (which could be summarized as “making the above conversation even more annoying and pedantic and drawn out to try to be extra sure”):
Basically, we’re planning to write a hidden file into the root of your network folder, and if we ever list that folder and see the file missing, we’ll know not to trust what’s going on.
This breaks some fundamental assumptions of watched folders (mainly, that they are read-only), so we’ve avoided the it so far, but it’s fairly likely that it will work. Work on this is ongoing.
Performance of very large collections
@RobertS and @Janis, you have something in common: your library sizes are in the 99.9th percentile. Congratulations 
@RobertS’s observation about Windows is astute–extremely large libraries do work better on Windows.
This is a tricky thing to say out loud, because it isn’t really our intent to endorse one platform over another, and the vast majority of our users will truly never feel the difference–but in the interest of openness, I’m going to explain what’s going on here.
Roon is built on top of a framework called .NET. This framework is powerful, flexible and mature, and it’s suitable for building complex, demanding, long-lasting software on every major desktop, mobile, and server platform in use today.
Our choice to use .NET is what enables us to keep releases on five platforms in sync without having five copies of the software. It gives us access to a great ecosystem, and high quality tools to work with. There are not very many alternatives for a company/product like ours that tick the same boxes.
.NET was created by Microsoft, so it should not be a surprise that the best performing implementation of .NET is currently available on Windows.
For non-windows platforms, we use a community-developed .NET implementation called mono. This is a mature and stable project that’s been around nearly as long as .NET has. For almost its entire life it’s been stewarded by various businesses associated with the project’s founder, including Ximian, Novell, Xamarin, and as of recently Microsoft.
Yes, Microsoft. A few months ago, Microsoft acquired the company that stewards the mono project, with the expressed intent of leveraging mono’s technology in order to directly support .NET on all platforms. So the prospects for .NET on Mac, Linux, Android, and iOS are looking pretty bright.
Today, mono is good enough that virtually all of our users don’t have to care about what platform they run on. Looking towards the future, there is almost certainly going to be convergence between Microsoft and Mono’s offering, because they are now being funded and developed under one roof.
Unfortunately, when a Roon library gets up into the several-hundred-thousand-tracks range, it pushes mono’s memory management infrastructure so hard that bad behavior starts to result. Symptoms might include laggy screen loading, slow focuses/searches, audio dropouts, or other stuff like that. When the framework gets messed up like this it is literally standing in the way of the code we need to run to get things done.
A catalyst for this problem is the richness of Roon’s data model–500,000 tracks means we’re also keeping track of 50,000 albums, 750,000 performers, 20 million credits, and so on. Compared to most other music apps, we manage at least 10x more data–so we are starting from a more difficult position.
That said, there are definitely some areas where Roon could be optimized, and some specific performance oriented projects are planned. We are just wrapping up a round of optimization on RAAT–our next scheduled bit of performance work is focused on improving library related performance, with a focus on mac and linux, which should help here.
That said, I’m not totally certain whether it will be enough for libraries this large–the only way to find out will be to test once the work is done. Hopefully some improvements from Mono/Microsoft will appear in parallel with our work.
In Summary
Looking towards the future, it’s fairly clear that these problems will be behind us at some point, but that that point is not today.
In the immediate term, @RobertS’s solution–a powerful Windows machine–is probably the most viable workaround for supporting extremely large libraries like these with good performance.
Unfortunately, they are some of the more difficult and long-standing issues that we are doing battle with, and the solutions are not clear or crisp.
I hope this helped explain things, and I’m sorry that we don’t have an immediate remedy for these problems.