503 errors from Roon endpoints (ref#49R335)

Support
1

Tell us what's going on

· Seeing some 503s from Roon endpoints this morning, causing discography to fail to load

Tell us about your home network

· 1Gbps, Opnsense

05/22 07:45:07 Debug: [easyhttp] [4632] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0000960854/tracks?grouping=performance&ordering=popularity&ascending=false&genresMode=and&rolesMode=and&composersMode=and&conductorsMode=and&producersMode=and&ensemblesMode=and&performersMode=and&worksMode=and&labelsMode=and&periodsMode=and&formsMode=and&instrumentationsMode=and&c=tidal-gb&contentPreferences=avoidMqa&languages=en,%3Een&tidal=max returned after 5305 ms, status code: 503
05/22 07:45:07 Debug: [easyhttp] [4631] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0000960854/albums?discographyVersion=2&grouping=auto&ordering=originalReleaseDate&ascending=false&genresMode=and&rolesMode=and&composersMode=and&conductorsMode=and&producersMode=and&ensemblesMode=and&performersMode=and&labelsMode=and&c=tidal-gb&contentPreferences=avoidMqa&languages=en,%3Een&tidal=max returned after 5398 ms, status code: 503
05/22 07:45:08 Debug: [easyhttp] [4633] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0000960854/works?grouping=none&ordering=popularity&ascending=false&periodsMode=and&formsMode=and&instrumentationsMode=and&c=tidal-gb&tidal=max returned after 5997 ms, status code: 503
05/22 07:45:26 Debug: [easyhttp] [4704] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0004327428/tracks?grouping=performance&ordering=popularity&ascending=false&genresMode=and&rolesMode=and&composersMode=and&conductorsMode=and&producersMode=and&ensemblesMode=and&performersMode=and&worksMode=and&labelsMode=and&periodsMode=and&formsMode=and&instrumentationsMode=and&c=tidal-gb&contentPreferences=avoidMqa&languages=en,%3Een&tidal=max returned after 5288 ms, status code: 503
05/22 07:45:26 Debug: [easyhttp] [4705] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0004327428/works?grouping=none&ordering=popularity&ascending=false&periodsMode=and&formsMode=and&instrumentationsMode=and&c=tidal-gb&tidal=max returned after 5269 ms, status code: 503
05/22 07:45:26 Debug: [easyhttp] [4703] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0004327428/albums?discographyVersion=2&grouping=auto&ordering=originalReleaseDate&ascending=false&genresMode=and&rolesMode=and&composersMode=and&conductorsMode=and&producersMode=and&ensemblesMode=and&performersMode=and&labelsMode=and&c=tidal-gb&contentPreferences=avoidMqa&languages=en,%3Een&tidal=max returned after 5530 ms, status code: 503
05/22 07:45:36 Debug: [easyhttp] [4759] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0004327428/tracks?grouping=performance&ordering=popularity&ascending=false&genresMode=and&rolesMode=and&composersMode=and&conductorsMode=and&producersMode=and&ensemblesMode=and&performersMode=and&worksMode=and&labelsMode=and&periodsMode=and&formsMode=and&instrumentationsMode=and&c=tidal-gb&contentPreferences=avoidMqa&languages=en,%3Een&tidal=max returned after 5264 ms, status code: 503
05/22 07:45:36 Debug: [easyhttp] [4760] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0004327428/works?grouping=none&ordering=popularity&ascending=false&periodsMode=and&formsMode=and&instrumentationsMode=and&c=tidal-gb&tidal=max returned after 5376 ms, status code: 503
05/22 07:45:36 Debug: [easyhttp] [4758] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0004327428/albums?discographyVersion=2&grouping=auto&ordering=originalReleaseDate&ascending=false&genresMode=and&rolesMode=and&composersMode=and&conductorsMode=and&producersMode=and&ensemblesMode=and&performersMode=and&labelsMode=and&c=tidal-gb&contentPreferences=avoidMqa&languages=en,%3Een&tidal=max returned after 5499 ms, status code: 503
05/22 07:45:45 Debug: [easyhttp] [4781] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0000619424/works?grouping=none&ordering=popularity&ascending=false&periodsMode=and&formsMode=and&instrumentationsMode=and&c=tidal-gb&tidal=max returned after 5331 ms, status code: 503
05/22 07:45:45 Debug: [easyhttp] [4779] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0000619424/albums?discographyVersion=2&grouping=auto&ordering=originalReleaseDate&ascending=false&genresMode=and&rolesMode=and&composersMode=and&conductorsMode=and&producersMode=and&ensemblesMode=and&performersMode=and&labelsMode=and&c=tidal-gb&contentPreferences=avoidMqa&languages=en,%3Een&tidal=max returned after 5499 ms, status code: 503
05/22 07:45:45 Debug: [easyhttp] [4780] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0000619424/tracks?grouping=performance&ordering=popularity&ascending=false&genresMode=and&rolesMode=and&composersMode=and&conductorsMode=and&producersMode=and&ensemblesMode=and&performersMode=and&worksMode=and&labelsMode=and&periodsMode=and&formsMode=and&instrumentationsMode=and&c=tidal-gb&contentPreferences=avoidMqa&languages=en,%3Een&tidal=max returned after 5396 ms, status code: 503
05/22 07:45:53 Debug: [easyhttp] [4836] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0004327428/tracks?grouping=performance&ordering=popularity&ascending=false&genresMode=and&rolesMode=and&composersMode=and&conductorsMode=and&producersMode=and&ensemblesMode=and&performersMode=and&worksMode=and&labelsMode=and&periodsMode=and&formsMode=and&instrumentationsMode=and&c=tidal-gb&contentPreferences=avoidMqa&languages=en,%3Een&tidal=max returned after 5151 ms, status code: 503
05/22 07:45:53 Debug: [easyhttp] [4835] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0004327428/albums?discographyVersion=2&grouping=auto&ordering=originalReleaseDate&ascending=false&genresMode=and&rolesMode=and&composersMode=and&conductorsMode=and&producersMode=and&ensemblesMode=and&performersMode=and&labelsMode=and&c=tidal-gb&contentPreferences=avoidMqa&languages=en,%3Een&tidal=max returned after 5368 ms, status code: 503
05/22 07:45:54 Debug: [easyhttp] [4837] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0004327428/works?grouping=none&ordering=popularity&ascending=false&periodsMode=and&formsMode=and&instrumentationsMode=and&c=tidal-gb&tidal=max returned after 6125 ms, status code: 503
05/22 07:46:03 Debug: [easyhttp] [4859] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0004327428/albums?discographyVersion=2&grouping=auto&ordering=originalReleaseDate&ascending=false&genresMode=and&rolesMode=and&composersMode=and&conductorsMode=and&producersMode=and&ensemblesMode=and&performersMode=and&labelsMode=and&c=tidal-gb&contentPreferences=avoidMqa&languages=en,%3Een&tidal=max returned after 5777 ms, status code: 503
05/22 07:46:03 Debug: [easyhttp] [4860] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0004327428/tracks?grouping=performance&ordering=popularity&ascending=false&genresMode=and&rolesMode=and&composersMode=and&conductorsMode=and&producersMode=and&ensemblesMode=and&performersMode=and&worksMode=and&labelsMode=and&periodsMode=and&formsMode=and&instrumentationsMode=and&c=tidal-gb&contentPreferences=avoidMqa&languages=en,%3Een&tidal=max returned after 5846 ms, status code: 503
05/22 07:46:03 Debug: [easyhttp] [4861] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0004327428/works?grouping=none&ordering=popularity&ascending=false&periodsMode=and&formsMode=and&instrumentationsMode=and&c=tidal-gb&tidal=max returned after 5827 ms, status code: 503
05/22 07:46:21 Debug: [easyhttp] [4906] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0004327428/tracks?grouping=performance&ordering=popularity&ascending=false&genresMode=and&rolesMode=and&composersMode=and&conductorsMode=and&producersMode=and&ensemblesMode=and&performersMode=and&worksMode=and&labelsMode=and&periodsMode=and&formsMode=and&instrumentationsMode=and&c=tidal-gb&contentPreferences=avoidMqa&languages=en,%3Een&tidal=max returned after 5266 ms, status code: 503
05/22 07:46:21 Debug: [easyhttp] [4905] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0004327428/albums?discographyVersion=2&grouping=auto&ordering=originalReleaseDate&ascending=false&genresMode=and&rolesMode=and&composersMode=and&conductorsMode=and&producersMode=and&ensemblesMode=and&performersMode=and&labelsMode=and&c=tidal-gb&contentPreferences=avoidMqa&languages=en,%3Een&tidal=max returned after 5375 ms, status code: 503
05/22 07:46:21 Debug: [easyhttp] [4907] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0004327428/works?grouping=none&ordering=popularity&ascending=false&periodsMode=and&formsMode=and&instrumentationsMode=and&c=tidal-gb&tidal=max returned after 5346 ms, status code: 503
05/22 07:48:32 Debug: [easyhttp] [4931] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0004327428/tracks?grouping=performance&ordering=popularity&ascending=false&genresMode=and&rolesMode=and&composersMode=and&conductorsMode=and&producersMode=and&ensemblesMode=and&performersMode=and&worksMode=and&labelsMode=and&periodsMode=and&formsMode=and&instrumentationsMode=and&c=tidal-gb&contentPreferences=avoidMqa&languages=en,%3Een&tidal=max returned after 5210 ms, status code: 503
05/22 07:48:32 Debug: [easyhttp] [4932] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0004327428/works?grouping=none&ordering=popularity&ascending=false&periodsMode=and&formsMode=and&instrumentationsMode=and&c=tidal-gb&tidal=max returned after 5320 ms, status code: 503
05/22 07:48:32 Debug: [easyhttp] [4930] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0004327428/albums?discographyVersion=2&grouping=auto&ordering=originalReleaseDate&ascending=false&genresMode=and&rolesMode=and&composersMode=and&conductorsMode=and&producersMode=and&ensemblesMode=and&performersMode=and&labelsMode=and&c=tidal-gb&contentPreferences=avoidMqa&languages=en,%3Een&tidal=max returned after 5446 ms, status code: 503
05/22 07:48:43 Debug: [easyhttp] [4949] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0000133719/albums?discographyVersion=2&grouping=auto&ordering=originalReleaseDate&ascending=false&genresMode=and&rolesMode=and&composersMode=and&conductorsMode=and&producersMode=and&ensemblesMode=and&performersMode=and&labelsMode=and&c=tidal-gb&contentPreferences=avoidMqa&languages=en,%3Een&tidal=max returned after 5744 ms, status code: 503
05/22 07:48:43 Debug: [easyhttp] [4951] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0000133719/works?grouping=none&ordering=popularity&ascending=false&periodsMode=and&formsMode=and&instrumentationsMode=and&c=tidal-gb&tidal=max returned after 5621 ms, status code: 503
05/22 07:48:43 Debug: [easyhttp] [4950] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0000133719/tracks?grouping=performance&ordering=popularity&ascending=false&genresMode=and&rolesMode=and&composersMode=and&conductorsMode=and&producersMode=and&ensemblesMode=and&performersMode=and&worksMode=and&labelsMode=and&periodsMode=and&formsMode=and&instrumentationsMode=and&c=tidal-gb&contentPreferences=avoidMqa&languages=en,%3Een&tidal=max returned after 5673 ms, status code: 503
05/22 07:48:57 Debug: [easyhttp] [4982] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0003741917/works?grouping=none&ordering=popularity&ascending=false&periodsMode=and&formsMode=and&instrumentationsMode=and&c=tidal-gb&tidal=max returned after 5498 ms, status code: 503
05/22 07:48:57 Debug: [easyhttp] [4980] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0003741917/albums?discographyVersion=2&grouping=auto&ordering=originalReleaseDate&ascending=false&genresMode=and&rolesMode=and&composersMode=and&conductorsMode=and&producersMode=and&ensemblesMode=and&performersMode=and&labelsMode=and&c=tidal-gb&contentPreferences=avoidMqa&languages=en,%3Een&tidal=max returned after 5654 ms, status code: 503
05/22 07:48:57 Debug: [easyhttp] [4981] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0003741917/tracks?grouping=performance&ordering=popularity&ascending=false&genresMode=and&rolesMode=and&composersMode=and&conductorsMode=and&producersMode=and&ensemblesMode=and&performersMode=and&worksMode=and&labelsMode=and&periodsMode=and&formsMode=and&instrumentationsMode=and&c=tidal-gb&contentPreferences=avoidMqa&languages=en,%3Een&tidal=max returned after 5656 ms, status code: 503
2

This means that Roon’s cloud services were unavailable to you. Since no widespread issues were reported, it’s likely a localised network issue.

Try restating your Roon server machine, and if this doesn’t help, restart your home network.

3

Hello @Henry_Buckle,

I have taken another look at your diagnostic logs, and I can confirm that the connectivity issues have resolved themselves. All of your recent API requests are now returning a successful status code: 200 and are completing in just a few milliseconds.

05/22 08:32:06 Debug: [easyhttp] [5379] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0000133719/albums?discographyVersion=2&grouping=auto&ordering=originalReleaseDate&ascending=false&genresMode=and&rolesMode=and&composersMode=and&conductorsMode=and&producersMode=and&ensemblesMode=and&performersMode=and&labelsMode=and&c=tidal-gb&contentPreferences=avoidMqa&languages=en,%3Een&tidal=max returned after 189 ms, status code: 200, request body size: 1 KB
05/22 08:32:06 Debug: [easyhttp] [5372] GET to https://api.roonlabs.net/discover/1/performers/122:0:MN0000133719/albums/new?profile=ddd8e72d-860b-4f82-b0a6-7744ecc469ff&c=tidal-gb&contentPreferences=avoidMqa&tidal=max returned after 502 ms, status code: 200, request body size: 0 B
05/22 08:32:06 Debug: [easyhttp] [5381] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0000133719/works?grouping=none&ordering=popularity&ascending=false&periodsMode=and&formsMode=and&instrumentationsMode=and&c=tidal-gb&tidal=max returned after 142 ms, status code: 200, request body size: 2 KB
05/22 08:32:06 Debug: [easyhttp] [5380] POST to https://api.roonlabs.net/browse/1/performers/122:0:MN0000133719/tracks?grouping=performance&ordering=popularity&ascending=false&genresMode=and&rolesMode=and&composersMode=and&conductorsMode=and&producersMode=and&ensemblesMode=and&performersMode=and&worksMode=and&labelsMode=and&periodsMode=and&formsMode=and&instrumentationsMode=and&c=tidal-gb&contentPreferences=avoidMqa&languages=en,%3Een&tidal=max returned after 315 ms, status code: 200, request body size: 11 KB
05/22 08:32:06 Trace: [performer/tracks] Browser load from service took 412ms

Since this issue cleared up on its own, it strongly suggests that the 503 errors were caused by a temporary routing or DNS resolution issue between your network and our cloud API. If this happens again, I highly recommend checking your OPNsense DNS settings (ensure you are using a reliable provider like Cloudflare 1.1.1.1 or Google 8.8.8.8) and verifying that no traffic inspection services are intermittently throttling those specific API calls.

Since everything appears to be back to normal on your end, is the service performing as expected now, or are you still encountering any issues with your library loading?

4

Yeah it resolved without me doing anything - I was mainly just reporting pre-emptively in case it turned out to be a wider issue.

I would challenge that it is an issue with my network though - the response code indicates the request is reaching Roon, but there is an issue with the server, hence returning 503 Service Unavailable.

5

The 503 status code may have come from your ISPs transparent proxy rather the remote (Roon) server. This is one reason why using different DNS servers is recommend.

Restarting local services, i.e., your network, is a sensible first diagnostic step, and changing DNS is highly recommended.

6

I saw the same issue (ie discography unavailable; I didn’t check the logs) at (?almost) exactly the same time this morning as @Henry_Buckle.

Resolved itself a few minutes later.

I’m using Google DNS with a BT Openreach fibre connection.