Possible API issue


#5

Same here. Once a zone gets powered down while grouped, the core starts spamming that unhandled exception until the core is restarted. @support Any progress addressing this?


(Harry ten Berge) #6

Yeah an update would be great @support


#7

For the record - I run into this too yesterday. Roon complained once or twice every second:

02/01 11:04:51 Critical: scx: in OnExit: System.NullReferenceException: Object reference not set to an instance of an object
  at Sooloos.Broker.RoonApi.TransportService_2.EndpointToJson (Sooloos.Broker.Api.Endpoint ep) [0x00028] in <dcbcf4f91fec401e800a83ba373c6044>:0 
  at Sooloos.Broker.RoonApi.TransportService_2.<ZoneToJson>b__37_2 (Sooloos.Broker.Api.Endpoint x) [0x00000] in <dcbcf4f91fec401e800a83ba373c6044>:0 
  at System.Linq.Enumerable+SelectIListIterator`2[TSource,TResult].MoveNext () [0x00048] in <ae162b7061064bfaa55021254699ac67>:0 
  at System.Collections.Generic.List`1[T]..ctor (System.Collections.Generic.IEnumerable`1[T] collection) [0x00077] in <370a0c27f4b74d1a81431037df6d75bf>:0 
  at Base.JList..ctor (System.Collections.Generic.IEnumerable`1[T] l) [0x00000] in <1e9187d858a94158b2a2c81586b0ef84>:0 
  at Sooloos.Broker.RoonApi.TransportService_2.ZoneToJson (Sooloos.Broker.Api.Zone zone, System.Boolean for_diff) [0x000e6] in <dcbcf4f91fec401e800a83ba373c6044>:0 
  at Sooloos.Broker.RoonApi.TransportService_2._ActuallyChanged (Sooloos.Broker.Api.Zone zone) [0x00000] in <dcbcf4f91fec401e800a83ba373c6044>:0 
  at Sooloos.Broker.RoonApi.TransportService_2.<_update_subscriptions>b__25_0 (Sooloos.Broker.Api.Zone x) [0x00000] in <dcbcf4f91fec401e800a83ba373c6044>:0 
  at System.Linq.Enumerable+WhereEnumerableIterator`1[TSource].ToList () [0x0001b] in <ae162b7061064bfaa55021254699ac67>:0 
  at System.Linq.Enumerable.ToList[TSource] (System.Collections.Generic.IEnumerable`1[T] source) [0x0001f] in <ae162b7061064bfaa55021254699ac67>:0 
  at Sooloos.Broker.RoonApi.TransportService_2._update_subscriptions () [0x000d1] in <dcbcf4f91fec401e800a83ba373c6044>:0 
  at Sooloos.Broker.RoonApi.TransportService_2.OnThreadExit () [0x00034] in <dcbcf4f91fec401e800a83ba373c6044>:0 
  at Sooloos.Broker.RoonApi.Module.ev_exit () [0x0000b] in <dcbcf4f91fec401e800a83ba373c6044>:0 
  at Sooloos.SynchronizationContextThread.OnExit () [0x0000a] in <7f0a74b68d2a4a0ba3084b62b8028591>:0

I cannot say when it started because Roon had been running out of log files already but I think it was the same root cause: powering down and starting up again selected or all devices from grouped zones in between.

First I thought to post into this thread:

… but it looks here things are more recent. :wink:


(Dylan Caudill) #8

Hi all,

Apologies for the delay here. I checked in with the team on this and they’re still actively investigating the cause of what you’re seeing. We appreciate your patience while the investigation continues. I’ll be sure to update you ASAP when I have more information from the team.


(Nathan Wilkes) #9

Thanks @dylan. I can corroborate that a reboot of ROCK is the only way to consistently fix the problem when it occurs, at least in my setup (4 RoPieee endpoints, 3 RoPieee displays, ROCK core).

Only for context, I have gone to significant lengths to remove any possibility of networking issues (everything now wired, everything around 1ms or less).


(Dylan Caudill) #10

Hi everyone,

I wanted to reach out and let you know that we resolved an issue here in our latest update, Build 401. You can read more about the update here:

Please give it a try and let us know how it goes!


#11

@dylan - seems like it didn’t go away. Had this spamming the logs again two or three days ago:

03/25 00:06:57 Critical: scx: in OnExit: System.NullReferenceException: Object reference not set to an instance of an object
  at Sooloos.Broker.RoonApi.TransportService_2.EndpointToJson (Sooloos.Broker.Api.Endpoint ep) [0x00028] in <75d52ccbda9342358cc2e0d9eacb8d79>:0 
  at Sooloos.Broker.RoonApi.TransportService_2.<ZoneToJson>b__37_3 (Sooloos.Broker.Api.Endpoint x) [0x00000] in <75d52ccbda9342358cc2e0d9eacb8d79>:0 
  at System.Linq.Enumerable+SelectListIterator`2[TSource,TResult].MoveNext () [0x00048] in <ae162b7061064bfaa55021254699ac67>:0 
  at System.Collections.Generic.List`1[T]..ctor (System.Collections.Generic.IEnumerable`1[T] collection) [0x00077] in <370a0c27f4b74d1a81431037df6d75bf>:0 
  at Base.JList..ctor (System.Collections.Generic.IEnumerable`1[T] l) [0x00000] in <021a9476948a45d9ad15e766804a7ee7>:0 
  at Sooloos.Broker.RoonApi.TransportService_2.ZoneToJson (Sooloos.Broker.Api.Zone zone, System.Boolean for_diff) [0x00102] in <75d52ccbda9342358cc2e0d9eacb8d79>:0 
  at Sooloos.Broker.RoonApi.TransportService_2._ActuallyChanged (Sooloos.Broker.Api.Zone zone) [0x00000] in <75d52ccbda9342358cc2e0d9eacb8d79>:0 
  at Sooloos.Broker.RoonApi.TransportService_2.<_update_subscriptions>b__25_0 (Sooloos.Broker.Api.Zone x) [0x00000] in <75d52ccbda9342358cc2e0d9eacb8d79>:0 
  at System.Linq.Enumerable+WhereEnumerableIterator`1[TSource].ToList () [0x0001b] in <ae162b7061064bfaa55021254699ac67>:0 
  at System.Linq.Enumerable.ToList[TSource] (System.Collections.Generic.IEnumerable`1[T] source) [0x0001f] in <ae162b7061064bfaa55021254699ac67>:0 
  at Sooloos.Broker.RoonApi.TransportService_2._update_subscriptions () [0x000d1] in <75d52ccbda9342358cc2e0d9eacb8d79>:0 
  at Sooloos.Broker.RoonApi.TransportService_2.OnThreadExit () [0x00034] in <75d52ccbda9342358cc2e0d9eacb8d79>:0 
  at Sooloos.Broker.RoonApi.Module.ev_exit () [0x0000b] in <75d52ccbda9342358cc2e0d9eacb8d79>:0 
  at Sooloos.SynchronizationContextThread.OnExit () [0x0000a] in <cd1db2e4c22841c29dacd496dc8d6afd>:0

Since I had rebooted the ROCK core in between / had it offline I may still have the logfile showing what happened before the exception spamming started. Seems like using an nvidia ShieldTV as an aditional zone in between triggered this.

BTW I was wondering if you have anything in ROCK which checks for stuff like this in the logs and eventually triggers a core restart if the exceptions don’t disappear themselves? Kind of a “hard self-healing”?


(Harry ten Berge) #12

Yeah I’ve got more users complaining the issue still exists…


(Dylan Caudill) #13

Hi @u_gee and @spockfish,

Thanks for the info update here. We’d like to gather some more info about what you’re experiencing so I can pass it along to the team:

  1. Can you verify that you (or affected users) are running the latest version of Roon?
  2. How exactly are things failing now? Is it the same as before? Is information not reaching extensions or is something else occurring?
  3. Are there any patterns that you’ve notice that tend to lead to Roon getting in this state?

Thanks!


(Nathan Wilkes) #14

Yes, always latest Roon, RoPieee versions.

The display screen will freeze (not update). Or, not start displaying anything when music starts for the first time. This is the same behaviour as before, starting with the Roon update late last summer (if I remember correctly).

It seems to happen when no music has been playing for awhile, it seems. It only happens with grouped zones, at least for me. And, the fix (always) is a reboot of ROCK (no change to RoPieee displays or end points required, or indeed helpful).

Harry (@spockfish) instituted some functionality a few RoPieee versions ago to restart the extension under certain conditions, and in my environment the normally grouped two zones always display a lower number of minutes since the last restart than a zone that is normally single (again, in my environment).

Does this help?


(Harry ten Berge) #15

The pattern is as follows:

People use RoPieee in combination with a screen. The screen uses an extension (and thus the API) for gathering information to display. As long as this is on a single zone it’s fine.

It goes wrong as soon as people group zones together. Then, after a certain amount of time, the extension fails to retrieve information.


#16

The latest one available - currently 1.6 - 401. :slight_smile:

Except from the log spamming I did not yet notice any impact. But then my ROCK NUC regularly gets shut down and I do only check log files now and then. Also, I’m not using any extensions and am posting here just because the issue gets tracked here.

Something with grouped zones together with ungrouped zones is playing a role, so it seems. I’ve saved two core logs from the last time I noticed the log spamming to happen - let me know if you want those.


(Nathan Wilkes) #17

One more thing: restarting the RoPieee extension usually reloads a snapshot of the album art / title etc, but this is frozen (no updates while music plays). Restarting core is the only solution.

Occasionally we experience the problem with an ungrouped zone, as happened tonight. Fortunately it is easy to initiate a restart of core via a URL.

Thanks for investigating.


(Dylan Caudill) #18

Thanks for the info here, everyone.

I spoke with the team, and we’d like to take a look at some individual reports here so we can enable diagnostics and take a look at what is occurring.

In Build 401 of Roon, we made some changes that resolved an issue causing a similar type of behavior. The issue was centered around endpoints becoming unavailable due to networking issues. There may be other things causing the same behavior, so we are hoping to gather a bit more data.

The next time this occurs please make a note of the time in which you experience this behavior. We can then enable diagnostics on your account so the team can take a look. Please also include some information about your setup and the endpoints that are in use here.

Thanks!


(Nathan Wilkes) #19

@dylan

Arrived home after a long weekend. Problem immediately occurred initiating play on grouped zone (Kitchen Pi and Living Room Pi), 18:50 Pacific, March 31. Music played, but display did not work. Restart of core fixed the issue (my normal course of action).

What further information do you require? 4 audio zones, 3 RoPieee displays, all connected via Ethernet. Ping times captured via rrdtool indicate no issues (between .5 and 1ms with almost no variation or abnormalities). ROCK core.

Also use the great Alarm extension, but not on the same zone as the normally grouped zones. Occasionally the alarm fails to stop the alarm (the second action), but never the first (start playing).

Music on Synology NAS, on own RAID1.

Let me know what else would be helpful, with thanks.


(Dylan Caudill) #20

Thanks, @Nathan_Wilkes.

I’ve enabled diagnostics on your account so the team can take a look. Can you provide some additional information about your current networking setup? What networking hardware is in use?


(Nathan Wilkes) #21

Hello @dylan . Here is a simple network diagram and a sample of ping times (times are typical – “Lemon Pi” is wireless and used for non-Roon things. Let me know if anything is unclear.


(Dylan Caudill) #22

Thanks, @Nathan_Wilkes.

I’ve passed this along to the team and they’re investigating this issue. I appreciate the information!


(Nathan Wilkes) #23

@dylan, another occurrence just now at 19:28 or so Pacific. I presume this gives you enough to work on, but let me know if any further information would be helpful.

Thanks.


(Dylan Caudill) #24

Thanks @Nathan_Wilkes. I checked in with the team and they’re currently investigating. At this time we don’t need any further information but I’ll be sure to keep you updated here when I receive additional feedback from the team.