Roon on macOS not working with huge library

I found this happen also. Best way forward at the moment is to look at the logs. Search for [analysis] from the bottom of your log files for periods where you know the analysis stalled. I saw several files/tracks repeatedly appear here.

A bit of trial and error revealed one album that had files that Roon didn’t like. They weren’t flagged as corrupt but dbpa also couldn’t convert them. It was free download sampler album so I binned it.

Analysis completed after that.

Hope that helps

I’m super happy to report that after upgrade to Roonserver the 70k album library is up and running stable, fast, and without anydropout case for a week! :stuck_out_tongue_closed_eyes:
Thank you Roon team and community for help and guidance!

One more happy customer…
:grinning:

2 Likes

I’m with Steve - seventy thousand albums; I can’t get my head around that…

Excellent. That is really good info for others to reference as and when.

Now you can start getting to know the s/w and fixing any albums that didn’t get an auto-ID match. Winter is coming…lots of long dark nights :relaxed:

1 Like

This is a really valuable thread, and thanks particularly to @brian for your explanation regarding mono memory management.

The things that drove me here were iTunes performance hit with >20K albums, and the poor home sharing experience, with no faith they would change. It’s your transparency and clear dedication to getting this right and offering a great experience, constantly striving for better, that will have me happily stumping up the cash when my 14d trial runs out.

I’m also having performance issues, but so much more manageable knowing the path ahead.

Of course my own wishes for swifter more meaningful browsing using metadata will only test performance further, but I’m sure you guys are ready for the challenge! And if you want another 99% percentile library size, info/tag obsessive, on OS X, not to mention .NET SaaS experienced experience designer, for road testing and feedback, I’m in!

1 Like

2 posts were merged into an existing topic: Mac OS Sierra is out [Roon testing in progress]

I have been experiencing this problem and reporting it since the spring. I hadn’t seen this acknowledgement by Roon before today and am glad to see Roon is working on it.

Yes, I have a very large library on OSX and my files do the disappearing act from time to time. I noticed that when Roon is watching a folder it really does not like anything in the folder to change! Nothing added or removed. Or, heavens forbid, that a file which was put into iTunes’ “Automatically add to iTunes” folder gets moved into its proper Artist/Album folder when iTunes starts up and is set up to organize files.

There are two unpleasant workarounds. One is to quit Roon before adding/subtracting files, then restart Roon and let it go through the scanning process. The other is to Disable the Watched Folder while adding/subtracting, and then re-Enabling that folder. Again, long scanning process.

Roon people, any updates on progress on fixing this?

Thanks.

We have a few clients with very large libraries – 25k to 100k albums. What factors contribute most to fast performance? Can we rank them such that the ones with the most impact are at the top of the list?

Factors I can think of are as follows, but I don’t know which ones have the greatest impact:

  • CPU speed
  • Number of CPU cores
  • RAM
  • SSD vs. spinning hard drive
  • RAID configuration vs. single drives
  • Thunderbolt connection vs. USB3 vs. USB2
  • Size of library
  • Mac OS

Thank you in advance.

Assuming you read Brian’s post further up the chain?

It seems the basic needs for extremely large libraries would be:

Use a fast windows machine
Make sure OS and Roon database is on SSD
Sufficient RAM.

How all the intricacies rank beyond that, I’m not sure anyone’s benchmarked such things?

Thanks for this. I read that, but we are on Mac OS, and not planning to change just for Roon. I have been experimenting with tiny libraries. With a very small library of, say 100 albums, it is quite easy to tell what is going on. Searching the database appears to occur within the Roon app, on the drive where the core lives, using cached data. I can very quickly find the album or track I want to play. The only lag is waiting for a spinning hard drive to spin up, in the event that this drive is dormant (i.e. in energy saving mode).

Remaining questions:

  1. This large library problem only happens on Mac OS? What is the ETA for fixing this?
  2. Would storing music on SSDs improve search speed? Based on the above, I assume no.
  3. Is 16GB RAM sufficient for a very large library?
  4. Would processing speed help? If so, which helps more – RAM vs. CPU speed?

Thank you.

Windows was a suggestion for your clients with large libraries - I’m not sure OS X performance is so much ‘a fix’, more incremental improvements of Roon to handle huge libraries on that platform. Beyond that, I think you’d need one of the developers to talk about specifics. 100k albums is huge - I think if your clients wanted ultimate performance, they’d have to consider the most appropriate platform. Also, are they using RoonServer?

As far as I know search speed is database related rather than music media related.

For the most part the critical item here is CPU speed. Due to the way that the Roon database works adding more cores isn’t going to have any real impact on search performance. If the Roon Core is used for other tasks then having an extra CPU core or two available for those tasks will reduce contention.

Important only in the sense that you never want Roon to run out of available physical memory. When that happens the OS will swap some memory blocks to the system disk and that can really kill performance. 8GB is sufficient for the wast majority of libraries, but by all means have at it!

Next to CPU speed this is likely the most important factor but only for the database. Roon is constantly hitting its database and depending on the search query it may be hitting the disk vs. memory. These are tiny random transactions and they are most suited to SSD.

On OSX the database is stored in the user’s home directory ~/Library/Roon. One option is to swap out the system disk for an SSD, but depending on the Mac this can be a pain. It is possible to relocate the Roon folder to a different drive and use some of the underlying features of the OS to point Roon to the right place, but there are several ways in which this can break so I wouldn’t recommend it.

For the database… use SSD. For the media use whatever makes you happy, but it’s not going to make a difference in the interactive application performance. Depending on the number of drives a RAID5/10 configuration can improve the speed of moving the entire library into place, but it’s not going to make a bit of difference in terms of performance.

For reference 2 channel DXD has a transfer rate of about 16Mbit/sec (2 MByte/sec) if your drive can’t hit that then you have much bigger problems!

I would avoid USB2 as it can be painfully slow. For what Roon is doing either USB3 or Thunderbolt are more than adequate for media or the database. I don’t see much of a point in the added expense of Thunderbolt.

The bigger the library the larger the database and its associated indices. A large database on its own shouldn’t present a problem, but it is going to need more resources in terms of CPU speed, memory, and database IOPS.

Of the three platforms on which Roon is available OSX has the slowest overall performance. This is due to a few factors but the two most critical are the way that OSX prioritizes tasks (as well as the number of things running in the background) and the Mono (.NET) execution environment that Roon relies upon.

If I were spec’ing out a Core for a client with a large library I’d likely go for an I7 CPU running at a reasonably high speed, 16GB of ram, and the fastest SSD I could find. I would not spec any Mac hardware for this task (unless it was running linux or Windows). I prefer linux as the OS, but a good stripped-down installation of Windows works just as well.

Ultimately, with 1.3 and the release of the Roon Core Kit I doubt that I will spec anything but that to run the Roon Core.

  1. While Roon is constantly optimizing their code this issue is out of their hands as it’s more specific to the OS (and they didn’t write that part).

  2. No.

  3. Most likely, but people here with 500K+ track libraries would be able to tell you for certain.

  4. Processing speed would help, but it’s not the silver bullet. Sorry, but OSX just isn’t the right platform for the Roon Server when the track count goes up. For a LARGE library the best practice is to run it on a dedicated machine with the most appropriate OS. When customers have no trouble spending $1000s on tweaks they can easily afford to spend a few hundred to get their playback “transport” dialed in.

7 Likes

Wow…that is a great response. No stone left unturned there.

I fully support the notion that if you have 100k albums, then that represents a ton of time if not money to accumulate. A decent server or NUC is less than the price of the HDD to store (and backup) that quantity of media. The benefits that Roon brings to libraries of all sizes would make a spend on a server well worth it.

An powerful i7 server with Linux Roonserver or a NUC with dedicated Roon Core Kit will be the best bet here. No need to move off whatever platform you are already on. Just add the dedicated server and run Roon clients as you have today.

Seriously big libraries need to be taken very seriously with dedicated kit.

Thanks for all the replies. I wonder what architecture the big streaming services like Tidal, Spotify and others use to make their enormous databases searchable. Surely there is an industrial-strength approach … Perhaps what they do is proprietary and requires a lot more horsepower. Hard to say. For now, I’ve guided clients on Mac OS to use Roon and Tidal, and only add the artists they want to hear to a watched folder. By the time they get through everything, we’ll be in the year 2050. :slight_smile:

It’d hardly be viable setting up a server farm to run Roon at home, nor is it required. My library sits at the lower end of the album count you mentioned. Roon Server is running on an i5-4690S with 16GB DDR3/1600 (8GB would suffice) running Linux. Roon’s database is on a dedicated F2FS formatted SSD, music is stored locally on 5900RPM NAS BTRFS formatted drives without any RAID config. Performance is good. With a bit of luck you’ll see search speed enhancements with 1.3. At 100k albums I’d recommend your clients start listening and forget collecting. :grinning:

I’ve received that message running Roon on a Mac Mini to an endpoint of an ASRock BeeBox (an Atom NUC) and I only have 300+ albums. Media is on a NAS and is supplied via a GB network. It’s that POS Mini and its operating system.

Dell R710 server with dual Xeon processors, running 16 cores at 2.6 GHz and 64 meg of ram. Used on EBay for about, with shipping, $350.

1 Like

Highly recommended! We run them with Freenas on them.

300 albums is a small library. On my Mac, I have over 3000 and roon works flawlessly. My Mac mini is running OS X server plus minimserver along with roon, with 26TB of disk attached.

1 Like

Hi @Brian and @support, it’s been 4 1/2 years since your very informative post about the problem of disappearing libraries, and MacOS tricking Roon into thinking drives are mounted when they aren’t.

I’ve been a Roon user for a long time and continually suffer from this phenomenon. Yes, my library is massive, over 500,000 tracks. The core is on a Mac Mini M1 with 16Gb of RAM. Library is SMB-mounted on a Drobo 5N2. Laugh at me! I know I’m in an eddy of problematic hardware.

Still, the odd thing about it is, that when I see that my track count is doing a nosedive, I can still see that my Drobo really is still mounted. I can play the track in iTunes (now Apple Music). The drive is mounted, and Apple Music can see it. Roon can’t, though. I’ve wondered why my track count doesn’t suddenly drop to zero, as it would if I, say, powered down my NAS.

Mostly, though, I wonder whether in the past 4 1/2 years you have identified anything I can do short of replacing the core with a Windows box to stop this from happening.

I should mention that I am also a perpetual sufferer of the skipping tracks phenomenon, where I could start playing an album in my library, and Roon will go down skipping track by track, sometimes landing on a track late in the album and starting there, or just skipping all the way to the end without playing anything.

Thank you.