10,000+ Albums Subtracted One by One from Roon

I don’t believe my eyes! After reading your reply, it happened again. I deleted Cosmo’s Factory from CCR and it started deleting again. Here are the important lines in the log:
Right after deleting the last track from CCR:

10/21 18:57:54 Trace: [directory] Deleting file: /Users/rvmac/Library/RoonMounts/RoonStorage_dbe43a27c47218f8cdf829fea45b6f8eb30fea82/Rock/Creedence Clearwater Revival - Cosmo’s Factory (K2HD)/Creedence Clearwater Revival - 11 - Long as I Can See the Light.m4a
10/21 18:57:54 Trace: [clumping] clumping 98 tracks

Then this happens:

10/21 18:58:04 Debug: [music/storage] queueing 9 removes for 8b4cd896-b3ce-5e04-0305-461c28c4bb34 (Server)
10/21 18:58:04 Debug: [music/storage] queueing 14 removes for 8b4cd896-b3ce-5e04-0305-461c28c4bb34 (Server)
10/21 18:58:04 Debug: [music/storage] queueing 12 removes for 8b4cd896-b3ce-5e04-0305-461c28c4bb34 (Server)
10/21 18:58:04 Debug: [music/storage] queueing 10 removes for 8b4cd896-b3ce-5e04-0305-461c28c4bb34 (Server)

Then this happens and the counter goed down real fast; I see the albums disappearing from my album overview.

10/21 18:58:04 Debug: [music/storage] processing simple delete for 67:0:Rock/ELP - Trilogy/ELP - 06 - Hoedown.m4a from Server
10/21 18:58:04 Debug: [music/storage] LibraryTrack[12985138, Emerson, Lake & Palmer - Trilogy - Hoedown (taken from Rodeo) IsAvailable=True IsDeleted=False ]
10/21 18:58:04 Debug: [music/storage] processing simple delete for 67:0:Rock/ELP - Trilogy/ELP - 07 - Trilogy.m4a from Server
10/21 18:58:04 Debug: [music/storage] LibraryTrack[12985394, Emerson, Lake & Palmer - Trilogy - Trilogy IsAvailable=True IsDeleted=False ]

Etc. etc. My files are still there.

I appreciate some help. Where can I send the logfile to?

I have seen threads like this, a search threw up this which may be helpful.

https://community.roonlabs.com/t/roon-on-macos-os-x-not-working-with-large-library/13478/21?u=sloop_john_b

SJB

1 Like

Generally, this has to do with Roon getting confusing messages from your operating system about what’s happening on your NAS. @brian detailed some of the underlying causes (and our ongoing efforts to work around this behavior behavior) here, so that’s definitely worth a read.

This does not mean music is being deleted, but we know it’s an extremely annoying bug, and we are hoping to deploy the improvements Brian described soon. For now, some more details about your setup may help @support get things more stable for you.

I had not found Brian Luczkiewicz’s post referenced above, so thank you for directing that my way. Clearly this is an issue you must be looking into as I am experiencing all of the defects described there: very slow Roon load times, very slow Roon search times, audio drop outs (although I was able to make these less frequent by turning off all audio analysis), disappearing tracks, etc. Given the speed of search engines these days, the slowness in search clearly speaks to architectural issues that should be resolved as soon as possible (iTunes had similar search speed issues years ago they had to re-architect to resolve). I can understand how joining all that metadata into one search query - especially as you probably can not flatten it all at insertion so that you can always include the most recent library updates - could be very slow and taxing on cpu. Perhaps you might want to implement a simple search that just brings up the artist’s albums, or something like that that makes sense using your domain expertise. This could be turned on for collections > x tracks, or be a settings toggle. Then the user could click more info. I understand that metadata has become one of your prime marketing drivers, but this would then only pull metadata for that one artist after the user already has a track list and ability to play. Rather be listening to music than waiting to find it.

Because of all this, I’m not sure that Roon is the right product for me personally, or others with large libraries, at this point in your product development path. Which is a shame.

A workaround question for now: Is it possible to get do a one time import from a main folder into Roon instead of making it a watched folder? Then use a much smaller folder for watched files? Would that solve this issue for now if the Roon server was not looking for changes in the main folder?

We have thousands of users running Roon with extremely large libraries and with NAS devices, and in most cases things are quite stable and snappy. The 99% percentile libraries discussed in that thread are on the order of 50,000 albums, so we’re talking 500k tracks or more.

So while most users are not experiencing these symptoms, we are aware of situations where confusing information from the operating system can result in Roon displaying these symptoms.

We absolutely plan to improve this behavior, but in the mean time we’d be happy to look into your setup to get some more clarity about what’s triggering these issues for you, and see if we can improve the situation.

But we need the details of your setup to do that:

Great news. Thank you again.

Roon 1.2.161
Mac Mini, OS 10.11.6, 8GB RAM
Synology 2414
USB to wired network on both ends
~200,000 tracks
Outputting to Benchmark DAC
NETGEAR ProSAFE 24-Port Gigabit PoE Smart Managed Switch
Synology has WFS enabled, SMB3, per another post’s suggestion

1 Like

Hi @AD_Music ---- Thank you for the follow up and the feedback. Both are appreciated! May I kindly ask you for the following:

  1. Can you please provide me with a description of your network’s topology and any networking hardware you maybe implementing. I want to get an exact understanding of how your devices are communicating and what gear you are using to make that possible (routers, switches, repeaters, extenders, powerline adaptors etc). The more specific and detailed you can be, the better :sunglasses:

  2. Can you please provide some insight on how you are accessing your NAS on your OS vs in roon. Let me expand a bit further on this thought. The NAS has to be mounted somehow. It can either be mounted in the operating system, thus browsing to it like a local drive, OR it could be mounted in roon, using our network storage infrastructure (typing the NAS’s address, user name, and pw etc).

Thanks!
-Eric

Hi Eric,

Thank you for your help. Here you go. Let me know what else you might need.

Basic star topology.
The mac mini is cabled into the NETGEAR ProSAFE
The synology is likewise cabled into the NETGEAR ProSAFE
They sit a few feet apart.
The Benchmark DAC is wired directly to the mac, which is wired to the amplification, as neither of these need the network.
Roon core server sits on the Mini.
Roon app sits on another mac and ipad that connects wirelessly over a UniFi wireless network.
The connection of core server to app is made wirelessly through the network.
All UniFi endpoints are usb wired from the central controller.
There are no repeaters or extenders in the audio part of this system as they are not needed for audio here.
There is powerline conditioning, separate circuits, power outlets etc.

Mounted through Roon.
Provided Roon a network path: smb://…to main music repo, username and password in settings.

1 Like

I have experienced the situation with the “subtraction”, too.
And I’m not a Mac user…
I stopped the Roonserver immediately, restarted it and the files were re-added into Roon.

My Setup:
ASUS Router AC 68U
Synology DS 212 (ethernet to router)
Sonic Transporter i5 (ethernet to router) with Roon core
PC Win7 (wireless), currently endpoint, waiting for the Blusound integration
1400 albums, 22000 tracks

The situation probably happened, when the NAS was under heavy load (re-scanning the audio files)

My setup:

Mac Mini 2012, i7, SSD, 16Gb memory, running Roon core.
Wired gigabit ethernet to QNAP 871 NAS 12 Tb RAID5, encrypted and cached with 1Tb SSD, single NIC used, 40% free disk space
Asus 66AC router, Netgear unmanaged switches.
My Roon library is about 6k albums, 80k tracks.
Output through USB to Mutec 3+ USB to Chord 2Qute.

The disappearing of tracks issue occurred when accessing the core from a Roon endpoint on my iMac, wired ethernet.

That is interesting. My core was also busy with scanning a newly added album in DSD64 for volume, etc when the track countdown issue happened.

Can you tell us what version of OSX you’re running? Thanks!

I am running El Capitan 10.11.6

Hi. I have not uploaded many albums in the last year. I have added files recently to roon and the same old problem happens over and over: the albums are subtracted one by one and it makes roon useless for the day or two. The album count that comes back is also variable, so that I never know if our albums are all there. This issue has clearly never been resolved. What is the status of this issue and are there any remedies? I am on 10.13.4 OS, but this has happened in all OS releases going back for the year or more. My gut tells me this is sparked by albums roon does not recognize, but that’s a gut response. I certainly hope you have more analytic data to respond. Thank you.

Hi @AD_Music ----- Thank you for touching base with us here. It’s good to hear from you, and I hope all has been well in 2018 thus far!

In regard to your most recent post. We have made some changes in this area of the application to improve upon our handling of changes that occur on network based storage devices and as such have not been seeing many reports of this lately. We can certainly try and do some additional testing to see what we can come up with, because as far as we know, this “trickling up/down” behavior with album counts should not be occurring.

In order to continue forward here, may I kindly ask you to please verify that your setup is still the same as above (#1 / #2) and furthermore I know you had listed how the NAS is being mounted in Roon in one of your responses above, but what would be great is if you could please provide a screenshot of your storage tab.

Additionally, can you please share another screenshot highlighting the SMB settings being used on the Synology 2414.

Thanks!
-Eric

Hi Eric,

Thank you for the kind reply. Everything is well here, and I hope there.

Yes, the setup is the same aside from the Roon version, which is updated to the latest version.

I have attached a screenshot of the storage overview from Synology: Is that what you mean by storage tab? If not, please let me know and I’ll provide.

I have also attached two screenshots of the Synology SMB settings.

Hi Eric. This is depressingly happening virtually every time I add albums. Hope you see something that might help.

Hi @AD_Music ----- Thank you for touching base with me and providing the requested feedback. Both are very appreciated!

Continuing froward, I am going to be opening a ticket in our system so we can track this behavior you are experiencing to ensure that it stays on our radar. Secondly, I am going to enable diagnostics on your account so we can have a closer look into into issue.

What this action will do is the next time that Roon is active on your core machine a diagnostics report containing a set of your Roon logs will automatically be generated/uploaded to our servers. I will keep an eye out for the upload and will touch base again once it has come in so you know we have it.

-Eric

Hi Eric,
I added some albums 30-40 minutes ago and it’s been counting back since then. Not sure if diagnostics is already enabled: but if so, the timestamp on this message indicates when this is happening.

Thanks for reaching out to me @AD_Music, the additional feedback is appreciated!

Continuing forward, I just checked our servers and it looks like the mentioned diagnostics report has not been received yet. I have re-enabled this feature on your account to try and refresh the request.

However, in the interest of time I would kindly like to ask you to please send us over a set of your Roon logs from the device hosting your core in a PM addressed to me using the instructions found here.

-Eric