Roon saves its scanning until I want to use the stereo

Yes, I like the idea of having hours in which scanning is ok, scheduling feature.

I have to say that after a r crappy week putting up with all sorts of scanning-related issues, the last time I sat down a couple of days ago…it was just perfect. So, I’m not sure what was going on last week.

Consistently, my Roon installation starts scanning as soon as it is finished scanning. It is always scanning regardless of whether any changes have been made to my watched folder.

Is this what 1.3 is supposed to do? It’s quite disconcerting.

@support

No. What kind of scanning are you referring to? Is this something reflected in Roon’s progress indicators, or something else you’re noticing on your system? If so, how are you noticing it?

The spinning progress indicator at the upper right hand corner is always turning. clicking on it reveals that Roon is
Connected
Scanning Now
281,401 Tracks Imported
(ever-increasing number of) Files Scanned

When the number of Files Scanned reaches the end, and all files are scanned, Roon just starts up scanning again… same numer of Tracks Imported, but number of Files Scanned restarts at zero and takes a good 12 hours to finish, only to start again.

Happens over and over.

Thanks for responding.

This is a bug, but it’s not yet clear whether it is ours or something in lower-level infrastructure going wrong. I think we’ve heard about this before–it seems to be some sort of resource issue. Linux specific I think. It’s been difficult to reproduce (I’ve tried a few times with no luck). How large is your library? Which linux distribution are you using?

@vova / @support, take note.

Not Linux, but the ever confounding OSX (mac mini) with library on a Drobo 5N, connected via ethernet.

I’m one of the people who has from the beginning (well, coming up on end of my 12th month), and continuing through build 200, had disappearing libraries. I had hoped this was fixed for me in 1.3, but it is not. Fortunately, the new backup approach has meant that the four or five times I have seen my tracks dwindle rapidly, I could restore from a backup…still, is this a good thing?

(There is much to like about 1.3…many of my wishlist items are there and work beautifully. But the dwindling library, combined with this ever-scanning, ever-spinning thing, is a killer.)

Library is about 281,000 tracks.

I should add that, though the dwindling tracks problem can happen spontaneously at any time, it will always happen any time I add a file to my watched folder.

The only way I can increase my library with new tracks is to quit Roon or disable my watched folder, then add files, then restart or re-enable. True in 1.2 and 1.3. Is this really the expected workflow?

Within Roon Storage Settings, how many Watched Folders have you set up??..could you post a screenshot of your storage settings??

Just one watched folder…

@brian and/or @support, certainly not Linux specific, I have it too on my Mac Mini i7 quadcore (14.5k albums). I upgraded upon release of 1.3 and finally, three days ago, rescanning and audio analysis have finished. But since then, the spinning wheel keeps spinning and the scanning is exactly like @Dan_Levy describes.

1 Like

@brian, @vova, @support

It’s worthwhile noting the similarities and differences here between these two cases

  1. Both Dan and Koen have large enough libraries [>250,000 tracks]

  2. Dan uses a Drobo NAS via Ethernet cable to Mac Core…while Koen uses a USB 8Tb Enclosure storage to Mac core…[please correct if I’m wrong here guys]

  3. Both use a Single Watched folder for their large library…Dan definitely does as indicated above…while I seem to remember that Koen also only has one watched folder {@Koen to confirm this point or otherwise]

So while the first point may be to think of SMB mounts etc with Dan’s issues…the USB storage used by Koen may counter that…summary for information only, just to try and circumvent some of the follow up questions

Thanks, Ronnie… I have been messing with SMB configurations for months and am willing to believe that there is something flawed in Apple’s or Drobo’s implementations, so it is surely interesting to see that in @koen’s case there is no Ethernet or Drobo involved at all.

Hi Dan
You may have already included this information in this thread…but just for the Roon guys, could you list the Mac specs…its OSX version [as that can have an effect on version of SMB used]…any Switches between Core and Drobo and whether the Switches are Smart or Dumb…as well as breakdown of the Storage and Disk setup within the Drobo

One last thing…have you ever tried splitting up your Music folder into e.g. Artists A-D; Artists E to G etc., etc to see if that made any difference??..i.e. have say 5 folders within Music…and then pointing Roon at each of those folders…i.e. having 5 watched folders

Thanks

OSX 10.12.3, mac mini late 2012, 16gb ram

I organize my music folder using iTunes, so I am not going to attempt your suggestion of dividing the music files among separate watched folders…thanks for thinking of it, though.

I use two Cisco SG200-08 8-port Gigabit Smart Switches (SLM2008T-NA) in between the Drobo upstairs and mac mini downstairs. It never occurred to me to monkey with their configurations but if anyone has ideas I’m all ears.

So, after a good long while (12 hours?) Roon scanned once again through my 281,000 tracks…and what did it do immediately after that? Started scanning again…here’s what it looked like just after the new scan:

Guys, I just came home from work and noticed there was a new bug fix, so I installed Build 200.

And… Apparently, it seems like in my case, it was a bug that has now been fixed. The spinning wheel has gone and at last, I can play with upsampling without dropouts, skipping or other problems. I’m halfway through an album now, upsampling redbook quality to DSD128 over ethernet, still no issues so really hope it’s stable now.

In reply to your answers, @Ronnie, indeed I did use one 8TB WD My Book, but I’ve recently (though before upgrading to 1.3) added a 4TB WD My Book Studio. So from one to two watched folders, both connected directly to my core via usb.

So thanks for Build 200 guys! And @Dan_Levy, hope your problems will be solved soon too.

Hey @Dan_Levy – can you zip up your Logs folder for me? You can find it your database, as described here.

Once you have the logs zipped up, if you can upload it to Dropbox and drop a link here, we will will investigate. If you don’t have Dropbox, let us know and we’ll give you some instructions about uploading them directly to us.

I would also be interested to see if anything changes with these switches removed. Is there anyway you can run for a day or two with these two devices plugged directly into a standard router?

@koen, we can absolutely look at your logs too, just zip them up and we’ll investigate.

@mike, please do! RoonServer logs are zipped (I assume you don’t need RAATServer logs?), how do I get these to you or @support?

This made it all make sense to me. I know what’s happening.

Roon knows that it can’t trust NAS’s to report on filesystem events in all cases–and there’s no way for us to detect whether events are coming or not. NAS’s just silently report a subset of events, or none at all. So “real time watching” isn’t real time all the time.

We did some research and found that just about every piece of software that attempts to do “real time watching” on NAS’s eventually runs into this limitation, and supplements it with a periodic scan of the storage folder. Plex does it, JRiver does it, so do others, and so does Roon.

This is not new with 1.3, by the way–it was added somewhere between 1.1 and 1.2.

So in addition to the “real time watching” Roon kicks off a scan of your storage locations once every 4 hours. This is a simple recursive directory listing–we are not actually poking around inside the files.

In the vast majority of cases it does not take hours. For example, on a spinning disk on my 5 year old Windows machine, it lists 100,000 files in about 10 seconds. Over an SMB connection to the same disk, the scan takes about 30 seconds. The same files on my QNAP TS-453 take 50 seconds. You could extrapolate this to much larger libraries at these rates and never see something take more than 5-10 minutes.

Before I used the QNAP, or this Windows machine, I had a Drobo for about 5 years. It seemed great at first, but over time, as it filled up, it started ruining things for me. Everything worked slow/worse/not at all. Eventually, I got down to troubleshooting the thing and figured out that each filesystem operation took about 100ms, not matter what it was or how idle the Drobo was. Open a file? 100ms. Seek 1024 bytes into the file? 100ms. Read 4096 bytes from the file? 100ms. List a directory? 100ms per file to get the file size/modification time. Even simple operations on that thing were excruciatingly slow. The Drobo was so bad that it nearly caused me to give up on NAS’s entirely.

The reason why it is scanning constantly is because an operation that should take a few minutes is taking 12 hours, and by that point, it’s time for the next “once per 4 hours” scan.

I’ll say one good thing about Drobo: it went through a couple of drive failures without losing any data. But it did take me almost 2 weeks to extract the data from its clutches 100ms at a a time. Drobo. Never again.

Not sure about @koen’s situation. The interesting question there is whether or not the scan is taking 4+hrs or not, since that’s the threshold for “constant scanning”.

(@ben @mike fyi)

1 Like

This is fascinating, though disconcerting! I appreciate your taking the time to write this up.

As I have been investigating, I noticed that the (optional) 64gb mSATA SSD I installed in the Drobo was no longer showing up in Drobo Dashboard. Why? I think it has failed, at least temporarily, because of a longstanding problem Drobos have with not sitting still long enough for the mSATAs to do garbage collection. Eventually they konk out. (I knew nothing of this til I read this post:

I wonder if, however maddening Drobos are anyway, the performance would improve once again if I installed a new mSATA card…they are cheap enough to try at $37 so I will report back on any improvements.

Thanks again, @brian!