Database Update Issues

Krzysztof_Grygorowic · December 6, 2019, 1:19pm

When i had roon 1.6 all worked good - but after update to Roon 1.7 on 24 November I have similar problem. It seem that Roon have trouble with conversion database - after 1 week of still lasting update process I see most of albums as tidal album even though I have my private copy. Some my private album (with check show only library album) is displayed but with status file unavailable - this status is display in history too and don’t concern a tidal files but played from my private library (physically this file are good and I can play this album with another program).

before and after updating Roon configuration is the same.

This update process last too long for me and it seem that Roon 1.7 is not final version (to many bugs as I read community site). Could you Help me quickly resolve this problem or can I remove this version and still stay with Roon 1.6 (I have backup copy of database)

noris · December 6, 2019, 8:15pm

Hi @Krzysztof_Grygorowic,

So we can better assist you, please provide a brief description of your current setup using this link as a guide.

Make sure to describe your network configuration/topology, including any networking hardware currently in use, so we can have a clear understanding of how your devices are communicating.

Downgrading to previous versions of Roon is not supported due to the reasons listed in that article, but we can certainly take a look to see what’s happening on the current version.

Can you share a screenshot of your Roon Settings -> Storage tab? Do you see your watched storage locations visible there?

Ado you see any “Activity Spinner” in the top right-hand corner indicating that there is activity going on? If so, can you share a screenshot of that as well?

Krzysztof_Grygorowic · December 6, 2019, 9:34pm

configuration - qnap nas TS-453bt3 16 GB ram 9Tb RAID 5 Connected to edimax BR-6478AC home wi-fi router by ethernet cable, edimax is by ethernet cable connected by ISP ROUTER Technicolor CGA2121 to internet
on qnap nas i have INSTALED QTS 4.4.1.1117, vpn and other network Settings
for roon port forwarding is udp 9003 and tcp 9100-9200
I have on Qnap ca 35k tracks. Now I see in roon ca 25k tracks and ca 9k tracks in setting library

Activity spinner - very slowly something change but i seem that spinenr is freeze few days.

In history tracks played properly before upgrading to roon 1.7 now are visible as unavailable - the album is show in this way too

At the and - I see output and can play tracks from tidal.

noris · December 10, 2019, 5:35am

HI @Krzysztof_Grygorowic,

Thanks for sharing those screenshots. Something doesn’t look quite right here, it’s almost as if Roon can’t see your watched storage locations properly. I wonder if logs would show any further information regarding this issue, can you please use the instructions listed here (https://roononnas.org/en/qnap/) and upload them to Dropbox / Google Drive and post a link here? Thanks!

Krzysztof_Grygorowic · December 10, 2019, 9:58am

Roon loogs : https://www.dropbox.com/transfer/AAAAAHrOopPdIUjm04fvZo5tEkCTPXqbRILrHBPUS8Bgx2kS-KLihNE

noris · December 12, 2019, 8:08pm

Hi @Krzysztof_Grygorowic,

Thank you again for your patience while our technical team has been looking into this issue for you.

I spoke with our team today regarding their evaluation of the logs from your install. Sometimes, errors like the one you saw can be spurious, but unfortunately in this case they are seeing signs of low level corruption in your database.

This type of error is extremely rare for us and not one that we see often. Our database infrastructure is designed specifically to prevent this type of corruption, and we don’t take this class of issue lightly.

We’ve traced a few reports like this in the past to hard drive integrity issues but generally speaking, this means that Roon is reading information from your hard drive that is different from what was originally written, and the database is now unable to load properly.

What’s Next?

If you’ve been making regular backups, my advice would be to install Roon fresh on your Core machine, and roll back to one of your backups. If the database loads properly, your edits, playlists, tags, etc should be intact, and we can confirm everything is performing properly once it’s been restored.

To set the old database aside on QNAP, you would have to specify a different location for the database via the QNAP interface and remove the old one. Once you have done so, you can restore your backup.

If you do not have any backups, unfortunately you will need to start with a fresh database.

Again, this class of issue is extremely rare for us and you have our apologies for the trouble here.

Krzysztof_Grygorowic · December 17, 2019, 8:42pm

i made some trails:

I made fresh database - and all works well - but without history etc.
I restore few backups (even from November 2018 - made with Roon 1.6 416) - I have access to history etc.

When I disabled tidal - number of local album is the same. When I enable tidal I observed Difference in case 2 - because spinner wheel display that: Adding music to library: of 5014 tracks, 5014 added, 0 identified.

Despite regularly made backups Problem is the same what means that damage in database are not related with QNAP but rather with conversion database from 1.6 416 stable to 1.7 505 stable.
It’s not acceptable option for me to start with new fresh database in case when are trouble despite having many backups.

Your backup method as I see is not perfect because each time I had successful status of last backup when backup was made. On the future it seem that during backup process self testing of integrity of database and proper warnings should be advantage for user data security.

Is possible to prepare Simple tool to extract history, play counts, favourite from old database ( this data are good) and add this information to new database or copy some files to new database. (All folder, setting and files with music are the same). I know that this a bit complicated but it could be useful in similar cases and in the future.

Do You agree that method of creation new database in each case when problem occur despite many backups have no sense? In such as my case better would be restored e.g 90% od data than 0%. Finally user should decide which database will be next database (old with addition or even new).

At the and. Before upgrade to 1.7 (24.nov.2019) all works well. Problem occurs exactly during upgrade from 1.6 416 to 1.7 505 and not from restored backup but from main database folder with working database 1.6 416.

noris · December 18, 2019, 10:12pm

Hello @Krzysztof_Grygorowic,

There are a number of reasons that a Roon database can become corrupted, and this can include failing hard drives, as well as other environmental factors, like power loss, other hardware issues or sync programs tampering with the contents of the database. I did also notice that your Core is under spec’d to run Roon as well, so this may have played a factor here:

From the QNAP Spec sheet for this model:

Intel® Celeron® J3455 quad-core 1.5 GHz processor (burst up to 2.3 GHz)

Our Minimum Requirements:

Intel Core i3, Ivy Bridge+

It’s extremely hard to know what might be at play here, but what this means is that when Roon reads the database back, the files are different from what was originally written to the drive, and the changes are significant enough that Roon cannot load the database properly.

Sometimes, database corruption can be “latent”, meaning that part of the database is corrupted, but the database can still load. Often this latent corruption is revealed when Roon updates to a new database format, as this process requires that every record in the database is accessed.

This is likely the case here — The backup that you used to restore was made after the latent corruption existed and was only discovered during the upgrade process.

This type of error is extremely rare for us, and generally the solution is to try using an even older backup if you have one in order to use a backup from before the corruption occurred, can you try using one of your older backups to confirm if the issue is the same?

There currently is no way to extract just pieces of the database, it is an “all or nothing” process in which Roon attempts to load up the previous database as it was configured when the snapshot of the database was created.

While I agree that error-checking could be a good idea, this is currently not a tool available at this time in Roon. If this is of interest to you, I would suggest making your opinion known on the feature request section of this site, as our product team keeps a close eye on this category and it’s a great place to get feedback from the Community regarding a specific request.

You have our apologies again for the inconvenience here. If there’s anything else we can do for you here, or if you have any questions about the above, please let me know and we’ll do everything we can to get this resolved for you.