RoonBridge armv8 memory leak

A “not” missing?

1 Like

Yes, sorry, @ged_hickman1. Thank you for the correction.


Ivan

1 Like

Hello @ivan,

Thank you for the follow up. I have shared the zip file here. It looks like logs contain and endless loop of connection events, I don’t know if this is expected. Please let me know if you need anything else.

Best regards

Thanks, @MusicPlayer_Silabs! I downloaded the logs package, will review it and then come back with further steps.


Ivan

Hi @jamie, @ivan,

Do you have any news about the memory leak issue? Can I help you in providing logs or debug info?

Best regards

Hi @jamie and @ivan,

Here are a few additional information. First, I have understood the reason for all those messages “network reachability changed, refreshing discovery” that you can observe in my RoonBridge_log.txt. They are in fact the consequence of DHCP IP renewal. Indeed my router (Mikrotik) had a DHCP lease time of 10 minutes, which leads to an IP renewal every 5 minutes.

So here is an advice to reproduce the memory leak issue: please reduce the DHCP lease time to a few minutes. This should help you to observe the problem on your side.

Now I have increased my DHCP lease time to 24 hours and here is the result on the log:

05/12 07:35:55 Trace: [roonbridgehelperserver] network reachability changed, refreshing discovery
05/12 19:35:54 Trace: [roonbridgehelperserver] network reachability changed, refreshing discovery
05/13 07:35:53 Trace: [roonbridgehelperserver] network reachability changed, refreshing discovery
05/13 19:35:53 Trace: [roonbridgehelperserver] network reachability changed, refreshing discovery
05/14 07:35:52 Trace: [roonbridgehelperserver] network reachability changed, refreshing discovery
05/14 19:35:52 Trace: [roonbridgehelperserver] network reachability changed, refreshing discovery
05/15 07:35:51 Trace: [roonbridgehelperserver] network reachability changed, refreshing discovery

And we can clearly see that the network reachability event now occurs every 12 hours. Doing this, I also notices that the memory leak seemed to reduce. So I have traced the RSS memory usage of the RoonBridgeHelper process over time. Here is the result:

We can see that the process has a constant memory leak, but there are some high steps that match exactly the DHCP renewal time, as circled in red.

So I guess a large part of the memory link issue is linked to the handling of DHCP IP renewal by RoonBridgeHelper. For me this means two possible fix suggestion:

  • A DHCP IP renewal should not result in a “network reachability changed”, which seems excessive (please note the IP address does not change)
  • A “network reachability changed” should not produce a memory leak

Can you please consider this additional info in you investigation and try to fix this memory link issue ? Thank you.

Best regards

PS: here is the pooling script I used to trace the memory leak, please use it to you will:

#!/bin/bash

p=$(pidof RoonBridgeHelper)
rm /mnt/mem.csv

while true
do
	d=$(date +%D)
	h=$(date +%X)
	m=$(ps -o rss= -q $p)
	echo "$d $h, $m" >> /mnt/mem.csv
	sleep 300
done

Hi, @MusicPlayer_Silabs, thank you for the deep analysis here. We will try to reproduce it locally. Actually, we had a similar problem with RoonServer on Linux some time ago. I will bring this up with our dev team and create a ticket. I do not know what will be the approximate timeframe for that fix.

Sorry for the trouble.


Ivan

2 Likes

Hi @ivan,

Did you had any chance to reproduce the issue on your side, by reducing the DHCP lease time?

BTW here is the memory graph on a longer period of time. During all this time the Roon Bridge is unused, only waiting in standby. The behavior is puzzling…

Best regards

1 Like

Hi @jamie, @ivan,

Did you had a chance to work on the memory leak issue ? Thanks.

Best regards

Hey @MusicPlayer_Silabs,

Unfortunately, we haven’t been able to reproduce the memory leak you’re seeing in-house. We compared your issue report to recent leaks that were discovered in Roon but the behavior is different. There were some indicators in your logs that appear to be IPv6 related, if you have that enabled on your router can you please deactivate it? Thanks!

1 Like

Hi @jamie,

Thank you for trying. I tried to disable IPv6 locally and I can still observe the same issue.
My router is a MikroTik hAP ac3 MikroTik Routers and Wireless - Products: hAP ac³, running on the latest firmware. In last ressort, can you please try this router in your lab? Maybe that will eventually be the cause of my issue here. Thank you.

Best regards

@MusicPlayer_Silabs,

In reviewing your past posts on this issue it appears that your problems occurred after installing a different image to your Pi. Have you tried reflashing the Raspberry Bullseye OS, or changing OS to see if that helps?

1 Like

Hi @jamie,

I have just checked and to my surprise, the memory leak issue is present in both RaspberryPi OS Bullseye 32 bits (armhf), and also in the older RaspberryPi OS Buster 32 bits (armhf). So I don’t understand why I did not noticed the issue before. Maybe you should try the MikroTik idea.

Best regards

Does the issue occur when Ropieee is used? Or ubuntu server with Bridge installed?

Hi @grizaudio,

Thank you for the suggestion, indeed testing the memory leak on Ropieee seems an interesting test. I installed it locally and it is running, unfortunately I cannot find a way to observe the memory leak. I need some kind of SSH access, but it seems it is disabled. Do you know how to enable SSH on Ropieee?

Best regards

I’ve been running RoPieee for weeks on my Pi4, without a problem at all. SSH access is not anymore supported on the new RoPieee, but on the older version I checked from time to time on RAM usage, and it always was stable and low. I never saw any evidence of a memory leak…

Me neither. Was just highlighting an option.

This I understood… was just convenient to use your question to offer as answer my experience. RoPieee just works…

1 Like

Absolutely.

This topic was automatically closed 45 days after the last reply. New replies are no longer allowed.