Note: This was split from another topic as @John_Steer1 is experiencing a different issue. Customer is reporting that images are failing to load in the Roon remote. Logs show timeout errors when contacting Roon’s cloud services
@AMP Hi Andrew, I have the same error continually on my ROCK server. I reboot it and it works fine for a day or so then the artwork disappears again and the log is full of the GET errors until I reboot again. Is this change you have done on the Nucleus available for the ROCK?
I made a settings change on your core just before I sent that message. The first restart forces your core to download updated settings. The second applies the change.
I have the same problem now and then. It seems to be network related.
Restart your router, that solves the problem - at least in my case.
Maybe it will help you as well.
I have tried rebooting the router but the only temporary fix seems to be restarting the actual ROCK Nuc which is connected via cat6 cable directly to the router.
Its almost as if something in the ROCK has stopped responding to the GET requests because I can run them manually from any other pc on the network and retrieve the images in a browser with no problems. Even tried a complete reinstall of the ROCK and restore from backup. Still keeps happening. It used to work great until a couple of releases ago. Hopefully a bug will turn up and get fixed.
@John_Steer1, I made a calculated guess without looking at your logs. Indeed you are seeing a different issue that others who have image loading problems. Our team is looking into it.
We have reviewed your logs further and it looks like the issues happen in batches, when there is an image error, it looks like network requests overall start to have issues.
Can you please provide some more details as to your network setup? What is the model/manufacturer of all your networking gear? How is your ROCK connected to the network?
Do you live in an area prone to network issues, in a rural area for example? Is your router up to date on firmware versions?
Hi @noris Thanks for looking.
I have a Ubiquity Amplifi router. The ROCK intel NUC is connected directly to the router via Cat6 cable. The firmware is all up to date. The internet connection is fairly solid. I’m in a metro area and internet is via a fixed wireless connection direct to my ISP at at 120/30MB with very few dropouts.
Recently whilst trying to diagnose the problem I even reloaded the ROCK and restored the Roon backup. I rebooted it yesterday and it was fine all day but the problem is back this morning. Doesnt seem to last much longer than 24hours before the problem reappears. The issue only seemed to present around a couple of 2 or 3 releases ago around the time you thought you had solved a lot of the extraneous DNS calls?? I have tried using my ISP DNS on the ROCK server and also Google DNS and currently 1.1.1.1 all seem to roughly the same.
Interestingly just restarting the Roon software on the ROCK doesnt necessarily solve the problem but a reboot of the ROCK does. It appears that something in the ROCK OS is going astray… maybe… just my general feeling. The Get calls that are failing can still be run manually on other pc’s on the network in a browser and still work fine.
Thank you for those additional details and your patience here, I’ve consulted with the team regarding your case.
It is interesting that you have to do a full reboot. Looking over your Roon logs it looks like just the images are impacted, right after the image fails, your Core is able to contact our other servers, so this looks like it’s impacting only the images. We have a few troubleshooting steps we suggest next:
Try a different cable or port on the router if you haven’t yet, it would be very strange if this was the issue, but good to check nonetheless.
Try to verify if temporarily hosting the Roon Core on another PC also has the same issue:
Open Roon on the other PC you wish to try as the Core
Roon Settings → General
Disconnect
On the “Choose your Core” screen, press “Use this PC”
If asked to Unauthorize, you can go ahead and do so. You are limited to one active Roon Core at a time but you are free to switch between them as often as you’d like.
Verify if the same behavior occurs on the different PC
Do you run these GET requests at the same time as ROCK is failing? If you try the GET requests after a while that ROCK fails, it would be better to test them side-by-side, in case there was a temporary network issue but then GET requests work later in the day.
If you are able to let us know when the issue happens and can leave ROCK in that state for a while, QA may be able to gather more info surrounding the issue via remote diagnostics.
Thanks for the suggestions. I’ll work through them one by one and see how we go. I have noticed that apart from the images also the streaming from Qobuz also stops working. It can see the album but when you hit play on a track it just sits there trying and the log file gets these error messages:
5/16 23:16:58 Debug: [easyhttp] [3107] GET to https://streaming-qobuz-std.akamaized.net/file?uid=1677402&eid=148979851&fmt=7&profile=raw&app_id=188245549&cid=1192784&etsp=1652744641&hmac=4blxITceLyOLluqQg-ameqgDIfw canceled after 5043 ms
05/16 23:16:58 Debug: FTMSI-B-OE qo/4C3227A6 interrupted req 393; missing block 0
05/16 23:16:58 Debug: FTMSI-B-OE qo/4C3227A6 rid:393 request ended – first block: 0 blocks read: 0 download speed: -256kbps response time: -1ms
05/16 23:16:58 Debug: FTMSI-B-OE qo/4C3227A6 created new req 394 for block 0 p 0; active requests 1
05/16 23:17:00 Warn: FTMSI-B qo/B2986E2C: block downloader too much time locked
05/16 23:17:00 Warn: FTMSI-B qo/2446B7EA: block downloader too much time locked
05/16 23:17:03 Warn: FTMSI-B qo/AB6DB8DD: block downloader too much time locked
If I drop the GET statement into a browser on another pc on the network it streams ok. Same as the image Get statements. Interestingly Tidal continues to stream ok.
First off though I have just changed the cable and the port on the router. So its up and going at the moment so 24 hours should tell if its going to fail again. I’ll let you know how it goes.
Hosting on another PC is a bit of an issue due to the size of my local library, it takes forever to get up and running on my laptop with 122,000 local tracks.
One step at a time
Regards
John.
Hi @noris ,
Just a quick update. Problem is back this morning after running 23 hours.
Yesterday I changed the cat6 cable and also changed the port on the router and rebooted the router. I have also updated to build 943.
@Rugby also queried the specs of the NUC which havent been included in this thread so just for your interest its an Intel NUC Gen 8 i5 with 8GB RAM. Purchased on 17/7/20 and has been running ROCK seamlessly for the last 18 months.
I’ll leave it in its current state for the moment so the guys can try doing some remote diagnostics.
Thanks for the update here, we were able to take a look over the system today and discovered an additional data point.
Even thought your ROCK logs are showing signs of the image timeouts, it looks like RoonOS (the operating system itself) is still able to access the image ok, the issue appears to be localized to when RoonServer is making that request.
We are going to discuss this further internally, but in the meantime it would be good to rule out if the database is related. Can you please temporarily try to use a fresh Roon database and a small subset of your usual content for however long it takes to see this issue? You can set up a fresh database with these instructions:
Hi @noris ,
Cool. Good to know that its maybe not hardware, network related. I have just set up a new database and maybe a quarter of the library and its currently importing tracks. I have added both Qobuz and Tidal as well. I’ll leave it running and see how it goes.
I’ll get back with more info as it comes to hand.
Hi @noris ,
Well that didn’t take long. All seemed to be going fine whilst the tracks were importing and being analyzed. Once that finished though I noticed that on the Home page the artwork for Qobuz and Tidal where missing. Trying to play Qobuz it just hangs again but Tidal is still playing just missing artwork as before. The artwork for the local tracks hasn’t disappeared as yet. There were a couple of albums that didn’t pick up embedded album artwork, but I added them as a cover.jpg in the album directory and it picked them up ok.
There is currently 14,791 tracks so it appears that the problem isn’t the database or the size of the library. Good to know at least.
I’ll leave it in its current state so you can check the logs if you wish.
Thanks,
John Steer.
Thanks for the additional feedback here and for trying the fresh database. I have been discussing this behavior with the team, and it does look like you are getting the image errors, even while playing TIDAL content. We are next hoping to answer the following two questions:
Has there been any change in behavior on the latest Roon release? It’s unlikely that anything changed, but it would be good to check this.
If you try to temporarily host the Roon Core on another PC, do you see the same behavior there or is it limited only to your ROCK? You can switch Cores by using these instructions:
Open Roon on the other PC you wish to try as the Core
Roon Settings → General
Disconnect
On the “Choose your Core” screen, press “Use this PC”
If asked to Unauthorize, you can go ahead and do so. You are limited to one active Roon Core at a time but you are free to switch between them as often as you’d like.
Verify if the same behavior occurs on the different PC
Hi @noris ,
I Loaded build 952 and it didn’t change anything.
I have just restored my Roon Core backup onto my Windows 11 laptop so I’ll see how it goes over the next day or so and let you know.
Regards
John.
Restarting now. Will the change be lost with subsequent upgrades?