Migrated Core, NAS Storage, Playlists Now (Mostly) Empty

Thanks @noris, I can definitely do that. I’ll kick off a new migration this weekend and will track it for the next few weeks. I’ll follow up in a few weeks on status.

Thanks!

1 Like

Just a quick update on this one: The 880 build has completely horqued running Server on the MacBookPro so I’m back on the NAS. Once we have a build that’s stable (with the new DB format) on Mac I’ll restart the testing.

1 Like

@noris The playlists have failed again and have reverted to the mostly empty state. I just caught it today but hadn’t checked in a few days so not sure exactly when they reverted. I captured logs after a clean re-install on 12/26 and then today after noticing that the playlists had failed again. I uploaded both tar.gz’s to the workdrive link previously provided by @beka.

Please let me know if you need anything else or would like me to test anything. Thanks!

1 Like

@noris Just checking back on my post from Jan 10th. Any progress? My playlists are still unusable. I’m happy to send any additional data you need to get past this issue with Mac core.

Thanks!

Hi @rcrawley ,

Thanks for sending those logs over, let me get these over to the QA team for closer inspection :microscope:

Can you please confirm a few of the playlist impacted by this, are the ones from your earlier post impacted? E.g:

Also, some additional info regarding the issue would be appreciated:

How are you adding these tracks to the playlists? Are you adding them in iTunes and then generating a new library.xml file?

Does this issue impact only iTunes playlists? If you have a streaming service, can you try to:

  • Go to an album
  • Don’t add the album to your library, instead add some tracks to a playlist
  • Go to that playlist and check to see if tracks remain stable

Thanks, @noris! Yes, exact same impacted playlists and almost the same numbers after the pruning (two in this example are the same numbers, two aren’t but close). Seems odd but true. Below is a screenshot from just now, it matches the one I provided when it happened last time.

  • How are you adding tracks?

I haven’t added any new tracks since the pruning error cut them down, but I normally add tracks using the “***” next to the track in album view and select “Add to playlist”.

  • Impact only iTunes playlists?

No, all local playlists are impacted (ie Tidal playlists are still fine). Some playlists which I created net-new in Roon are now showing zero files. So basically every local playlists has been incorrectly pruned.

  • Adding new tracks/albums

I did just test adding new (local and Tidal) tracks to an existing playlist (one that was created in Roon, not iTunes, which now has zero tracks after the loss) and adding single tracks and adding a complete album to that playlist worked fine and as expected. I’ll keep an eye on them.

Let me know if you need anything else. Thanks!

Screen Shot 2022-01-19 at 10.18.33 AM

1 Like

Hi @rcrawley ,

Thanks for those additional details, I have forwarded your remarks to the QA team. Thanks in advance for your patience while they review your case once more :hourglass_flowing_sand: .

Hi @noris, just checking back on any progress. This process also made me wonder if there’s another way to export my playlists from the still-working original NAS Roon server in a format that I can pass into the new Mac Roon Server as an import. It looks like my only options for exporting are either text files or exporting with the music files themselves, correct? No way to export just the playlists from one server and import them into another? I thought there may be a way since the storage locations are the same.

I am at a point where I think I’m going to have to manually rebuild them, but after exporting CSVs for all my playlists on the working Roon server, I’m looking at manually dealing with 14k tracks in those playlists, and clicking through 14k times to re-add manually doesn’t sound fun. :slight_smile:

Thanks!

Hi @rcrawley ,

I checked with QA on this and they have been unable to reproduce the issue in the lab, the next step would be to take a look at the database itself.

Can you please exit out of Roon and then zip up the contents of your Core RoonServer (or Roon folder if you’re not running headless anymore) and upload it here?

https://workdrive.zohoexternal.com/external/2e650d4f25a099594d247323ba7a7d4761679cdcb0e6fa4399f03bf3a9c3a2c9

Please let me know once uploaded.

Do you by any chance have Soundiiz and streaming service? That might help while we look into the database.

Thanks @noris. I’ll work on the DB part. re: Soundiiz, yes, I do use it and also have Tidal. But I don’t see Roon as option on Soundiiz. Am I missing it?

It is not a side-bar option, but rather you can import the Roon playlist contents:

Note: CSV Export is now supported too (suggested method).

Thanks @noris. I was able to import my playlists to Soundiiz from the good Roon Core but I’m not seeing an option to export them back out to the Roon Core with the busted playlists? My goal is to migrate the playlists from the NAS to the MBP, and that doesn’t appear to be an option as best I can tell (unless I’m missing something), natively or via Soundiiz.

@rcrawley how many of your folder names or files contain question marks, quote marks, slashes, or contain a terminating period? Some of this reminds me of issues I had with items showing up in playlists before I fixed those problems.

Hey @rcrawley ,

You won’t need to export them, after you import them to TIDAL/Qobuz, they should automatically appear after your next TIDAL/Qobuz sync. This way as long as you sign in with the same TIDAL/Qobuz account, they’ll automatically be downloaded from the TIDAL/Qobuz cloud.

13 folders contain quotation marks, none contain question marks, none contain slashes (in either direction), and none contain terminating periods (my auto-backup software on the NAS doesn’t like trailing periods). Good thought, but none of those dir names have changed since original import.

Resurrecting this thread because I’ve now been able to reliably reproduce the issue at will. As part of this thread – RoonServer Stuck Initializing on new Mac Installation, Logs Showing Critical Exception [Ticket In] - #67 by Matt_Armstrong – I just made a fresh backup of my DB from Roon running on my NAS and immediately imported into “full” Roon running on a freshly wiped MacBookPro. As soon as I imported the fresh DB backup I immediately checked my Playlists and saw that they’re back to the “only 5” issue that I’ve been having since originally reporting the issue. I removed Roon from the MBP, removed the Library dir, and tested the process again, same result. Everything else in the DB restore seems to be correct. Rinse and repeat over and over.

The new MBP also has a complete local copy of my music files; I’m no longer mounting the NAS from the MBP and storing all music locally to remove that as an issue. I also tried both restoring the DB first and then changing my storage locations to point to the local drive as well as adding the local storage to Roon first and then restoring the DB, both with the same result.

To compare, here’s a cap of my correct playlists on the NAS core compared to the incorrect playlists on the MPB, taken at the same time. It’s literally the same DB and same set of local music; the only difference is one core is on a Synology and one core is on a MBP.

Good playlists on NAS:

Bad playlists on MBP:

Hey @rcrawley ,

Thanks for the additional information here, and for letting us know that you are able to reproduce reliably now.

Have you had a chance to upload this database for review? I’m wondering what the playlists look like when loaded up on our end.

How does the current path of the music files look like, can you share any screenshots?

Let me know when you have a chance, thanks!

Hi @noris. I’ve been doing a lot of testing over the past two weeks, with many NAS DB backups and restores on the MBP, and here’s where I am at the moment:

  • Per a question from the original conversation, I renamed all of the playlists on the original NAS by removing the [Roon] prefix just to rule it out. I then took a fresh DB backup and have been using that as my source for restoring.

  • Without the [Roon] prefix, the playlists were correct for a few days after a restore but eventually fell back to the recurring “5 songs” problems. It’s probably a red herring but with this test I noticed that the playlists didn’t get screwed up until after I enabled Tidal integration (where I also have a bunch of playlists). In that test I wasn’t connected to Tidal for the first few days, and then not long after enabling Tidal in Roon I noticed the playlists went back to the problem state.

  • Strangely, without completely removing Roon no amount of DB restores from the clean one I made from the NAS made any difference with the playlist issue. I had to completely remove my Roon core (including everything in ~/Library) and re-install, then re-restore the DB to get back to a truly clean state. That was a really weird discovery; I assumed a DB restore would wipe the existing DB first but that doesn’t look to be the case.

  • Right now I’ve been running on a clean DB restore for a few days and still in the good playlist state, but I also haven’t enabled Tidal integration. I’m going to let this one sit for a few weeks w/o Tidal to see if that really makes a difference.

I haven’t had a chance to do that yet when I’m in the failed “5 songs” state. The next time I get back to it I’ll zip up the Roon contents and send them your way.

@noris The playlists have again revered to the “Only 5” issue sometime today. I looked through logs and nothing jumped out, and nothing changed today on my end (still no Tidal, I didn’t add any music or change any of the DB today), I just saw it had reverted sometime between Noon at 9:00 PM my time. I just tar’d & gz’d the entire RoonServer dir and the resulting tar.gz is 3.4GB. Can I upload a file that big? Thanks!

1 Like

Hey @rcrawley ,

Yes, you can upload the database to a folder I created for you here and let me know afterward:
https://workdrive.zohoexternal.com/external/ae29c9d1ccb4453bf9e8c1d2361b5d6dfc96f3e7c4f54421c69d25a8eb62b542

Though please note that it may take some time for QA to look this over.