Roon keeps working perfectly for me if I disable the automatic backups. How can the database truly be corrupt?
Hey @Alan_Spool,
I went ahead and re-opened your previous thread which had closed due to inactivity. I wanted to touch upon Norisâs last response:
Were you able to get in touch with SonicTransporter support?
Thanks! 
Sorry on taking a while to get back to you. As it turns out, I was thinking of upgrading my server so I just bit the bullet and did that. My latest report is on a new server (on the old server the manual backup did not work either). I think that pretty clearly rules out a hardware problem, at least on the server. The NAS I am using is the same, so if there is any way that can be a problem, that is still in play.
Just looking in on Roon at the moment, I see under Library Maintanence 1813 deleted files which i just cleaned up. The problem is, there should be no deleted files. Roon keeps deleting and adding back files on my NAS. I cannot help but believe that this unnecessary churn is part of the problem. Everyone else who has reported a similar problem also has their music on a NAS. Since SSDs are so cheap, I have thought of just buying one and moving my music off the NAS (or else just using the NAS as a 2nd backup since I already have a backup), but I do think this is getting a bit silly. The xisting hardware should work.
Hey @Alan_Spool,
Thanks for the update! To confirm:
- Youâre still having issues with loading your database on a new NAS? Iâm not seeing any new devices within your account information. Could you provide me more details with this?
Would you be able to grab a specific date and time of when this issue occurs? That way, weâll be able to hone in on your core diagnostic to see what might going on.
I realize this issue has been long-standing for you, so Iâm hoping we can pin down the culprit soon! 
I do not have a new NAS. I have a new server. I get a corrupted database error message any time I do an automatic backup. I have disabled automatic backups so I donât have this problem currently. Last time I did a manual backup, that worked fine (but I have not done that in a while because I have not been adding new files or playlist entries, and entries from Qobuz are stored there).
I see lots of churn every time I scan the NAS for new files. I can do that and then give you a time.
I started a scan at 7:58 pm last night. Since then I see over 1700 file deletions, and in recently added, some old albums that were not recently added.
Hey @Alan_Spool,
Thanks for the update! Weâre continuing to investigate a handful of potential issues linked to a degradation in performance, and potentially the issues youâre having here, related to the breakdown of library structure. One of them being a high number of tracks located within a subdirectory.
Another follow-up question for you:
Do you have at least one artist in your library with over 1000 tracks?
Do you have at least one artist in your library with over 1000 albums?
The biggest album i have is a John Eliot Gardner Bach Cantata album consisting of 52 disks, not sure how many tracks per disk but likely at least 12 each. Roon has always had trouble with these as these were published separately and as the set, and if I search for this album, I see several, one with all 52 disks, but others with a subset of these disks. I also have other albums from this artist.
I have the 100 CD collection from Yo Yo Ma so that could easily be over 1000 tracks alone, and there are probably other albums in the collection including more recently added ones in Qobuz.
Hey @Alan_Spool,
Thanks for the heads up! I donât have any immediate next steps for you at this time as weâre still in the early stages of investigating performance issues.
One thing you could test out - removing a few of the artists you mention above (make a fresh backup first - then close Roon, and temporarily move your local files/artist into a separate location not monitored by Roon - then perform a library clean up to fully remove the tracks from your database.)
I will certainly keep you in the loop as things progress on our end
Hey @Alan_Spool,
If possible, could you send over a copy of the database youâre experiencing these issues on? Steps to follow below:
- Zip up your
RoonBackupsfolder (right-click it and select âCompressâŚâ): - Submit the .zip file to us through our Database Issues portal
Let me know when this is complete. Thanks!
Done! Have at it.
I think I had come to believe this issue was resolved as at some point I stopped having the problem of getting the need to reload the database message and had several happy months running Roon almost trouble free. I admit if I did come on the system when it was re-scanning the NAS the system would be really slow, but that was only an occasional issue. Then, a few days ago, i again got the message that my library had been corrupted. I went to load from backup, and found my most recent backup the system could find was from July 2. Since I have no new files and havenât done too much to my playlists since, this was not a terrible hardship, so i did reload the backup. Then upon reloading it, i tried to do a backup immediately, but that seemed to fail. By restarting Roon, i was able to use the system for several days, but then this morning I got the corrupt library message again. This time I restarted the server, and now it is running again (after i had to wait for the interminable scan. I listened to some music, then attempted another manual backup. After some time (while i was typing this) the Roon app on the server seems to have or at least lost connection. After coming back up on its own, it again announces an issue loading my library.
I generally do not add music to the NAS anymore, as i tend to use Qobuz for new music. One thing I would like is to turn off the scan of the NAS as it so slows the system down and has no value add to me. Is there a way to do this? Also, i have not yet moved the 2 artists with very large track listings out of the library as you once recommended. Should I still try that?
From the diagnostic file from around when i tried to do a backup:
01/06 18:55:19 Debug: [broker/filebrowser/volumeshare] Use volume, user: broker/backups/fixbackuplocation, id: \\argus\music
01/06 18:55:19 Debug: [broker/filebrowser/volumeshare] Use dispose, id: \\argus\music
01/06 18:55:19 Debug: [broker/filebrowser/volumeshare] Use volume, user: broker/backups/getrestoreitems, id: \\argus\music
01/06 18:55:19 Trace: [backup] getbackupdirs, rootdir: AttachedDir:/mnt/RoonStorage_a526b9c5b2ce766ca53d96522a593bf37b3265b3/Roon/RoonBackups, machineid:
01/06 18:55:19 Trace: [backup] [backup getrestoreitems, manifestdir: AttachedDir:/mnt/RoonStorage_a526b9c5b2ce766ca53d96522a593bf37b3265b3/Roon/RoonBackups/0c2c410f-75eb-095b-8937-b3f222dcb94e/xx
01/06 18:55:19 Trace: [backup] [backup getrestoreitems, manifestdir: AttachedDir:/mnt/RoonStorage_a526b9c5b2ce766ca53d96522a593bf37b3265b3/Roon/RoonBackups/b7d3476b-3a4d-6258-7336-349c2388adb2/xx
01/06 18:55:19 Info: [broker/backups] item id: 0c2c410f-75eb-095b-8937-b3f222dcb94e
01/06 18:55:19 Info: [broker/backups] item id: 0c2c410f-75eb-095b-8937-b3f222dcb94e
01/06 18:55:19 Info: [broker/backups] item id: 0c2c410f-75eb-095b-8937-b3f222dcb94e
01/06 18:55:19 Debug: [broker/backups] success listing restore location: Result[Status=Success] (3 items)
01/06 18:55:19 Debug: [broker/filebrowser/volumeshare] Use dispose, id: \\argus\music
01/06 18:55:29 Info: [stats] 9808mb Virtual, 2692mb Physical, 963mb Managed, 5963 Handles, 64 Threads
01/06 18:55:31 Debug: [easyhttp] [4303] POST to https://api.roonlabs.net/device-map/1/register returned after 169 ms, status code: 200, request body size: 3 KB
01/06 18:55:31 Trace: [devicemap] device map updated
01/06 18:55:44 Info: [stats] 9816mb Virtual, 2693mb Physical, 962mb Managed, 5961 Handles, 68 Threads
01/06 18:55:46 Debug: [broker/filebrowser/volumeshare] Use volume, user: broker/backups/fixbackuplocation, id: \\argus\music
01/06 18:55:46 Debug: [broker/filebrowser/volumeshare] Use dispose, id: \\argus\music
01/06 18:55:46 Trace: [broker/backups] doing backup now
01/06 18:55:46 Trace: [broker/backups] computing list of files to back up
01/06 18:55:46 Debug: [broker/backups] preparing sync on FileBrowser.Entry: \\argus, music : \Roon\RoonBackups
01/06 18:55:46 Debug: [broker/filebrowser/volumeshare] Use volume, user: broker/backups/startbackup, id: \\argus\music
01/06 18:55:46 Trace: [backup] preparing backup on dir AttachedDir:/mnt/RoonStorage_a526b9c5b2ce766ca53d96522a593bf37b3265b3/Roon/RoonBackups
01/06 18:55:47 Trace: [backup] getbackupdirs, rootdir: AttachedDir:/mnt/RoonStorage_a526b9c5b2ce766ca53d96522a593bf37b3265b3/Roon/RoonBackups, machineid: 0c2c410f-75eb-095b-8937-b3f222dcb94e
01/06 18:55:47 Debug: [broker/backups] successful sync prepare on FileBrowser.Entry: \\argus, music : \Roon\RoonBackups
01/06 18:55:47 Trace: [broker/backups] [54857058-bb91-4cc5-bc88-8421c97f7e26] paused media thread
01/06 18:55:47 Info: [loadstatus] IsBackupInProgress False => True
01/06 18:55:47 Trace: [mobile] [remoteconnectivity] Port Verification started due to: load status changed, not testing port opening because the broker is not loaded and ready. account status: LoggedIn, machine status: Licensed, load status: BackupInProgress
01/06 18:55:47 Trace: [zone Family Room] Suspend
01/06 18:55:47 Info: [zone Family Room] Canceling Pending Sleep
01/06 18:55:47 Trace: [zone iPad] Suspend
01/06 18:55:47 Info: [zone iPad] Canceling Pending Sleep
01/06 18:55:47 Trace: [leveldb] closing /var/roon/RoonServer/Database/Core/99b2acc0a8a74e2ea1fbc5fd79382874/transport/zone_1601ca31cbd7cebcc744b6f0783bcd2a056d.db temporarily
01/06 18:55:47 Trace: [leveldb] closing /var/roon/RoonServer/Database/Core/99b2acc0a8a74e2ea1fbc5fd79382874/transport/zone_160132514408edcd068f8fb099c133b4d888.db temporarily
01/06 18:55:47 Trace: [broker/backups] [54857058-bb91-4cc5-bc88-8421c97f7e26] paused transport thread
01/06 18:55:47 Trace: [leveldb] closing /var/roon/RoonServer/Database/Core/99b2acc0a8a74e2ea1fbc5fd79382874/broker_4.db temporarily
01/06 18:55:47 Info: [broker/database/vacuum] ==================================================================================================
01/06 18:55:47 Info: [broker/database/vacuum] ===[ Validating Database ]========================================================================
01/06 18:55:47 Info: [broker/database/vacuum] ==================================================================================================
01/06 18:55:47 Info: [broker/database/vacuum] Validating /var/roon/RoonServer/Database/Core/99b2acc0a8a74e2ea1fbc5fd79382874/broker_4.db
01/06 18:55:48 Info: [broker/database/vacuum] processed 21 playlist entries
01/06 18:55:48 Info: [broker/database/vacuum] processed 30000 profile entries (skipped 0 entries)
01/06 18:55:48 Info: [broker/database/vacuum] processed 1676 radio entries
01/06 18:55:59 Info: [stats] 7919mb Virtual, 1750mb Physical, 946mb Managed, 354 Handles, 69 Threads
01/06 18:56:14 Info: [stats] 8099mb Virtual, 1876mb Physical, 946mb Managed, 342 Handles, 66 Threads
Hi @Alan_Spool,
Thanks for following up and sharing your status. Sorry to hear youâve run into additional database issues!
Reviewing a new diagnostic report from your sonic transporter, your recent corruption is due to lack of memory:
Error: [broker/database] corruption detected: Exception of type 'System.OutOfMemoryException' was thrown.
Do you have the option to increase the RAM on your server?
Hi @benjamin
This is a relatively new sonicTransporter i7 CDR (Gen 3) with 16 Gb of memory. Should this not be enough? What is the memory requirement for ROON? Does this not instead suggest some kind of memory leak issue or something?
@agillis
BTW the application hangs when trying to backup. For now I have disabled the automatic backup and so far it is staying up. I also removed stuff associated with an apple Music (was itunes) library which was previously in the scanned directory and I worry is doing something bad. I tried another backup after and the backup failed but that time the app did not crash and i did not get the library corrupted message. I am next going to remove John Eliot Gardner from the library who has >2K files listed (largely that 54 disk Bach Cantata set) and see if that helps.
Odd that the system can run out of memory - unless there is not enough backing store on the boot disk for virtual memory to page physical memory in and outâŚ
Hi @Alan_Spool,
If you can, I would suggest next setting up a different temporary server device, perhaps on your MacBook, and see if you run into database-related issues there as well.
Here is more info on the specs needed for Roon:
16GB should be fine. Weâre not able to review OS system logs to check for hardware-related issues, unfortunately.
@benjamin its not the hardware. This is a Roon certified server, it is showing no other problems, and it is relatively new, an upgrade from the device on which I initially started having problems. Also, if you look back in the thread, I did move the server to my macbook last summer and still had problems.
It is possible that there are hiccups in the connection to my NAS, although I have done everything I can think of to check its health. When I have time, I will continue pulling artists with very long track lists from the scanned directory. I could start over with a fresh database if there was a way to backup my playlists and reload them when done. Is that a possibility?
@benjamin I will look at that. Currently I am exploring possible issues with the Sinology NAS which is where I store my music, where the server scans, and in a separate directory where my backups are stored. It says it is healthy, but when I tried to copy one artist out of the library, macOS wouldnât let me (told me some files werenât found) and I did find some bogus redundant listings on there. I am concerned that there is some corruption and that I may need to do something about that. Maybe that is a problem that I need to solve first. I will keep you posted.
@benjamin I have now determined that my NAS is healthy (I am not sure what is going on with issues around ghost files seen by Finder on my MacBook, but looking straight at the NAS, the File server is behaving normally and extended scans show no problem). I removed folders with artists having very large numbers of tracks from the scanned folder. I cleaned up the library after a scan, tried to do a backup, and once again the app on the server crashed without completing the backup. I have offloaded all my playlists so I can reproduce them. In an attempt at a fresh start, I switched servers, using my MacBook instead, but pointedly did not restore a backup. The MacBook clearly had a database from last summer so it did not start from scratch, but it did finish scanning a couple of times and with a library clean up I then did a successful backup. I then switched back to my Sonictransporter, and restored that database from the recent backup. I then rescanned and cleaned the library and did a backup. That worked. Then I got cocky and moved the 3 artists with large numbers of tracks back into the scanned folder, rescanned, cleaned the library and tried to do a backup. i got the corrupted library message again.