Well, the re-scanning of 43Tb of data just takes time. Off my NAS, I had about 278,000 tracks. The scan off the backup hard drives is up to 167,000 tracks at this point, so still a little over 100,000 tracks to go. Roon is working fine at the moment — maybe a bit slow because of all the scanning going on — but not the 10 minutes for a search or to play a track. We will see over the next few days as the scanning approaches the prior 278,000 level.
If this holds up, it will demonstrate that the issue is not that the NAS is not functioning — it works for JRMC — but that there is an interaction problem between Roon and the NAS, which @support will need to address.
I’m just not clear at what point @support, as compared to the helpful community support so far, will review my logs and provide their input. JCR
For those following this thread, I’m up to 237K tracks so far. Thus, it appears Roon is loading about 23K of tracks per day. So, I guess it will take yet another couple of days to finish. Only then will I be able to conduct a fair test of the difference between using the NAS vs hard drives directly attached to my Roon core server. JCR
Ok, Roon seems to have finished identifying tracks in my library. Notably, I’m at 248K, 30,000 fewer tracks than with the same data on my NAS. I obviously can’t say what tracks overall are missing, but for example, I recently added “Kinn,” an album by Marc van Roon in Dolby Atmos 5.1.2 format (i.e., 8 channel multichannel FLAC), and it doesn’t appear. I have manually forced a rescan of the hard drives attached, but get no further tracks.
This said, Roon is still apparently trying to add music to my library. The spinning semicircle says of 67,751 tracks, 37,468 have been identified and added. This has not changed in the past 24 hours. I don’t know what this means and what resources are still being utilized to do this, nor why it doesn’t simply get the job done. Seems there must be some way to get this to quit.
Using directly attached hard drives instead of my NAS, search and startup of playback are as instantaneous as they’ve ever been. So, I am now prepared to definitively say the problem with using my library of tracks on my NAS is an interaction issue between Roon 2.0 and my NAS, which in my view should not be an issue since JRMC has no problem accessing tracks from the NAS for instant playback.
I say it is now time for @noris, @benjamin and the Roon team to pay attention to this thread. I previously uploaded my logs for review when using (or, should I say, trying to use) the NAS, as noted earlier on this thread. The issues here seem similar to those in the support threads started by @Roman_Petru and the support thread “Memory Leak in QNAP/Synology NAS.”
I have some further insight for consideration. As noted above, Roon stopped adding new tracks from the direct attached hard drives. At that point, Roon worked flawlessly – quick to search and start playing.
However, as also noted, I appeared to be 30,000 tracks shy of where I was using the NAS as my database. I rebooted my Roon core and Roon again started identifying and adding tracks (albeit extraordinarily slow – 700 of 30,000 tracks in the past 24 hours).
Once Roon starts the track identification process again, search and starting play slows again to a crawl, even using the direct attached hard drives rather than the NAS. I have uploaded a new set of logs to you now that hopefully will show this to the team.
It seems Roon server starts to scale very poorly beyond some number of tracks, maybe around 250K tracks. If I had the time I would prune my library to verify this hypothesis.
Martin kindly offered that my Synology NAS was the culprit struggling with I/O relating to music analysis. I took the NAS out of the equation and replaced it with three 16TB drives with the same music files, now directly attached to the Roon core server.
As noted, Roon stopped importing new tracks about 30,000 shy of my NAS total. When it stopped – as in the “Adding Music to Library” spinning semicircle was not spinning – Roon functioned wonderfully. I rebooted my Roon core server, thinking it might start up scanning for the 30,000 tracks apparently not yet imported from the directly attached hard drives.
Well, Roon then did indeed start the Adding Music to Library again, but that has caused an almost complete blockage of using Roon whilst that importing process is underway, just the same as if I were using my NAS. I just had a guest over at the house who wanted to hear a specific track off an album in TIDAL. It took 10 minutes for the album to populate and he had to leave another 10 minutes later before the track started to play.
I shut off the background and on-demand audio analysis in the Library tab, but that doesn’t shut down the Adding Music to Library function. And at the rate the Adding Music to Library function is actually adding tracks, it will take another month or so to complete, during which time I cannot use Roon.
Surely, there is something that can be done about this team? Thanks. JCR
@Mike_O_Neill , thanks for the suggestion. However, if I had a dime for every time I’ve rebooted my core trying to get this adding to music library to stop, or speed up, I could now retire.
Last reboot was 14 hours ago. As you can see in my last post, Roon was then working to add 29,577 tracks and had added 537 of them.
I recall having a similar issue a while back. I think it was due to corrupt files. I figured out which ones by reading through the logs. It is a bit hard to understand at first but you can see about every step in there.
Sometimes helps to restart, so you have a baseline. Keep refreshing the log file and you’ll see the progress.
There is a good post on finding and reading logs on here somewhere.
You’ve probably already looked at this but be sure to check Settings/Library/Skipped Files. Early on I had a lot of files in there. Both file types that roon doesn’t handle like Tif, and corrupt files.
Hello, @birban59. So yes, I do have skipped files which include ones identified as corrupt, I/O didn’t work or unsupported image file. But, these are skipped, right? So, how would they impact the still 29,850 tracks still being added to library by Roon, presumably over the next 9 weeks at the current rate? Are you suggesting that if I remove the skipped tracks from my library, everything will suddenly work?
The impact is just terrible. Our grandkids are over and wanted to have a dance party, starting with YMCA by the Village People. It took ten minutes for that track to start playing after a long time to load up. The dance party was a bust.
Roon team, we don’t want busted dance parties, now do we? JCR
I think it helps to get rid of corrupt files in particular. If you look at the logs the time stamps can even show that roon spends some time on some files, even getting stuck as some have mentioned. So if you have say 10 cores working but they’re each stalled on a bad file, there isn’t any progress.
Not saying it is the best design that it tries so long before moving on.
There was a post a while back about organizing you music so roon can more effectively categorize and curate it. That could be an issue I guess too.
Roon really does grab a lot of stuff.
I didn’t get why it is that you can’t play the library that is imported while waiting for the rest to finish? I seem to recall that I could play stuff as it gets added.
Really sorry you’re having this much trouble.
I just wish that I could shut off the importing manually so resources aren’t hogged and I could listen to music. I could then turn the importing, slow as it is, back on overnight. It’s not like I have an underpowered Roon server — it’s an i9-10900K processor and 32Gb of RAM. The CPU is hardly taxed.
But it seems that this simple functionality doesn’t exist in Roon. Bummer.
I have a separate support thread into Roon on this issue but the Roon team itself has not weighed in at all for two weeks now. As a lifetime member, a somewhat frequent poster and one who has referred many others into using Roon, I’m rather disappointed that the team cannot find the time to review the situation. JCR
I think Roon @support deals with things in order as quickly as they can with their limited resources. They don’t give preferential treatment to lifetime members, nor should they.
How about back to basics . Your Windows core , copy a few , say 1000 tracks to an internal hard drive.
Close the Roon GUI, Rename your main library to Roon_Current it will be in …/uses/…/ Local/Roon/Database. You can revert later by renaming again
Then open the Roon GUI , that will create a new clean db, now set the 1000 tracks as the Watched Folder, allow the library to build then reboot
Disconnect/Shut down any strange software on the PC , eg optimisers. HQ player etc
That “should” remove any Windows , NAS etc involvement
Also why 3 x 16Tb drives , I assume USB external, that’s massive , my 200k tracks is 4Tb . Several external USB drives may well confuse windows. How many tracks do you have to need this space 1 tb = 3000 albums @44.1/16 so 140 k albums of space ??? Sorry just seen 278k so around 5tb @ 44.1
If this plays normally to the System Output ie PC speakers then it’s a hint as to where to look next
PS what file formats are these I see ref to 5.1 surround are they ISO files, JRMC handles these natively , Roon will see them as one file , maybe a VERY big file , which could be what is stalling ?
This is a bug pure and simple but it’s affecting small number of users it has been going on for weeks for some of them. Somethings changed that’s causing Roon to be stuck in a loop indexing and it all started after an update. There might be some underlying issue in the users library that’s causing it that previously went undetected as it didn’t happen before. They did change something to do with db corruption detecting so maybe it’s somehow related to this?
Hello, @Mike_O_Neill and @CrystalGipsy. Mike, the full story for me is spread over a couple of support threads. Important to understand that after I had backed up my NAS files to separate 16Tb drives, disconnected the NAS from Roon and started the process of adding tracks to my library from scratch, I did reach a point at about 130K tracks when the “adding music to library” actually stopped. At that juncture, Roon operated flawlessly — quick searches and startup of playback.
Since I knew I was then still short by many tracks from what I had when the NAS was connected, I thought I needed to get Roon to restart the track additions. So, I rebooted my Roon core server and that got the track addition process back underway. But it then made it virtually impossible to use Roon again. Sigh.
My point is that I don’t think there’s any good purpose to trying to rebuild the Roon database a third time now, at least until Roon’s tech team looks at the log files I have sent along to them over the past couple of weeks.
Crystal Gypsy is correct in that I, too, have seen other reports herein by others with similar issues. It really does feel like a bug, unfortunately.
To your question about the size of data on my hard drives, I do have about 1,400 multichannel and Dolby Atmos 5.1.2 files on there. Those are very big — often 6-8 Gb per album. I also have the legacy iso files on the drives, which do take up space, although all DSD files are converted to separate DSF stereo and multichannel files for Roon playback. Further, the vast majority of other tracks are 24/96 at a minimum, up to 24/384.
I should mention that Roon ARC is essentially non-functional through this as well. I constantly get the error message about a poor connection out of home, which I know is not the case from my early access use of ARC before this whole database thing went sideways. JCR
1 Like
James_I
(The truth is out there but not necessarily here)
45
