Audio zones lost after Roon update on MacOS (ref#T0R2N8)

What’s happening?

· Other

How can we help?

· None of the above

Other options

· Other

Describe the issue

I read the change log for 2.56 (Build 1582) and noticed that an issue where audio zones are lost during Roon’s update process on MacOS was resolved. I’m unfortunately still having the same issue, and have consistently for the year I’ve used my Mac as a Roon server. Following an update of Roon, the server comes back online but Roon remotes only display the Mac’s own output. I have to shutdown Roon server via the Menu Bar icon at the top of the display on the system running the Roon server, then relaunch the service.

Describe your network setup

Mac -> (very close to) TP-Link Omada EAP650 wireless AP -> TP-Link Omada managed Switch -> Firewalla Purple router -> ISP cable modem. Local wireless network sustains 700Mbps up/down where the Mac is located. Wireless APs are connected to the switch via Ethernet.

Network endpoints are Apple TV and Cambridge Audio AXN10 streamer, both connected wirelessly.

Blackhole is an audio loop back device driver installed on the Mac running Roon server.

Hello @soks

Thank you for the update.

The next step here is to gather more detailed diagnostics so our technical team can take a closer look. Before we enable diagnostic mode on your account, we’d like to make sure we capture the correct event.

Please reproduce the issue once more and note the exact time (including date and timezone) when it occurs. Then reply back here with that timestamp — this will help us correlate the correct entries in the logs.

In addition, please also send us a copy of your logs:

  1. Use the directions found here to locate your logs:
    https://kb.roonlabs.com/Logs
  2. Upload the logs to our secure file uploader:
    https://workdrive.zohoexternal.com/collection/8i5239cc05950ac07456889838d9319545a82/external

Once the logs are uploaded, just let us know here.
As soon as we have the timestamp + logs, we’ll move forward with the diagnostic review.

Thanks!

Sounds good! Happy to have logging turned on. It looks like updates go out roughly monthly, so is there a way to rollback the current update to create a reproducer sooner than the next official release?

Hi @soks,

This is not possible, unfortunately. I’d also test out using Roon without any virtual audio drivers and see if you run into the same issue.

Does this happen every time you reboot the machine?

We’ll be monitoring for your reply - thank you!

Hi @soks,

We see dropouts between RoonServer and local/upstream addresses throughout diagnostics logging associated with your account. These aren’t limited to device discovery traffic between RoonServer and endpoints/remotes.

Have you ensured that Roon’s processes are safelisted in the Firewalla router security settings? Please also ensure that multicast forwarding is enabled in any managed network components.

Where are your networked endpoints and remotes connected relative to RoonServer? Do they share an access point or are there hops in between?

Ooh, interesting. I don’t see anything specifically Roon-related getting blocked, but I also haven’t specifically whitelisted any behavior. I just did a quick observation while streaming and everything looked okay, but there may be random attempts getting blocked? I’d love to try and correlate the Roon and Firewalla logs to see if there’s something that needs to be allowed through.

In terms of placement, the machine running Roon and an Apple TV are definitely serviced by the same AP. There’s another streamer (AXN10) on another AP. But both APs are on the same VLAN and devices can communicate without any issues. I’ve tried to keep the individual VLANs as close to a standard network as possible, devices just can’t communicate across VLANs. Also, both APs are wired to the same switch.

In terms of reproducing when restarting the machine, I’ll try and see what happens! If I can read my logs, I can also try to correlate the failures you mentioned.

Good day @soks !

Would that be possible for you to temporarily disable your firewall and check if the dropout issue keeps happening ?

It will allow us to narrow down the scope of the problem.

After the experiment feel free to turn on the firewall back.

Looking forward to your reply!

Regards.

Monitoring is currently disabled for the system running Roon Server and the AXN10, which should be the equivalent of shutting off the firewall specifically for these two devices.

Well, I think I reproduced the behavior? If I shut Roon down before restarting my system, it’s fine and ends gracefully and finds all my endpoints when the machine is online.

I restarted my system with Roon still running, which caused Roon to unexpectedly shutdown (~16:38 timestamp). When it came back online, Roon couldn’t see any network devices. It eventually found AirPlay devices, and only found the AXN10 when I opened the Roon Remote on my phone around 17:00.

Hi @soks,

Did you also temporarily disable the firewall on the issue mac(s) themselves?

Yes indeed. The Roon Server machine is the problem Mac. I also see the firewall exclusion was for 12 hours, so I just made it indefinite.

Hello @soks ,

Thanks for the update, do let us know if there is any change in behavior with the indefinite firewall exception!

Hi @soks,

If you wanted to check Firewalla logs to see if device discovery traffic was intercepted, check for content related to RAAT on port 9003 or 9100-9200. This range covers most of RAAT’s mDNS device discovery and control traffic.

Roon relies on dynamic port assignment for Cast, Airplay, UPnP, and other integral protocols. It’s tricky to try and create rules that cover the full range; if you can instead whitelist Roon and RAAT in any stateful security it might help.

The fact that opening Roon Remote prompted the server to discover your AXN10 in the last report is possibly good news. When the Remote shook hands with the server, it likely initiated the server to initiate network device discovery cycle, which discovered the AXN10 without issue.

This smells suspiciously like MacOS not allocating RAATServer socket access after waking from sleep or when the server has been backgrounded. The team hsa a ticket in the pipeline to make Zone retention more robust on MacOS.

However, in most cases, all devices (including coreaudio-managed devices like Blackhole or System Output) would disappear, which is why we needed to troubleshoot the stateful security.

We’d need to investigate another device discovery failure event to be sure. Let us know if you’ve had any issues in the last few days. Thanks!

I can only filter Firewalla logs for the last 24 hours of blocked network flows, but that port range didn’t come up. I’ve added a rule that will “allow” all traffic within the VLAN to communicate over those ports so I can monitor to see how many times that rule is hit. That will at least provide a sense as to whether Firewalla is even detecting this inter-network traffic. I think it should, given that I can see it block Apple-to-Apple device discovery from one VLAN to a different VLAN, which is intentional. But all traffic within a VLAN should be fair game.

I might be able to observe the network discovery of things like the AXN10 get blocked if I restarted the Mac running Roon, since I don’t see any discovery-related traffic with Roon currently streaming to the AXN10. It’s just “working,” so to speak, now that the endpoints are visible. But I do recall reviewing the Roon logs to see what was happening when it couldn’t find the AXN10 right after I launched Roon last week. It failed because there was “no route to host,” which I found odd because the Mac running Roon could AirPlay to the AXN10 at that moment.

As a test, I did just try using a pkill -9 Roon command to try and get it to fail, but it was graceful and didn’t reproduce the discover issue, so the mechanism the OS uses when I try to restart must be more aggressive…

Hey @soks,

Thanks for the update - that sounds good. Let us know the results.

In addition to that, here are some other considerations:

Your symptom - endpoints are invisible until something restarts is often tied to IGMP snooping tables not updating immediately after a Roon update.

TP-Link Omada EAPs/Ethernet switches have several features that can disrupt Roon discovery:

Disable temporarily for testing:

  • IGMP Snooping on the switch
  • Multicast Enhancement on the EAP650
  • Broadcast Filter / Multicast Filter (sometimes called “Layer 2 Isolation” variants)
Test:
  1. Disable these for the single SSID your endpoints sit on.
  2. Perform a Mac reboot
  3. Do endpoints appear immediately?
If yes, the switch isn’t populating IGMP groups fast enough after the update.

We’ll be monitoring for your reply, thank you! :folded_hands:

Hello @soks ,

Your thread is scheduled for auto-closure shortly, we wanted to check in with you to see if you still need assistance? If so, please let us know the information that @benjamin requested above, thank you!

We haven’t seen a response to this thread in some time, and we’re going to allow it to auto-close under the assumption that the issue was resolved offline. If you require further troubleshooting, you can reactivate the conversation by submitting a new technical support request. Our team will merge threads accordingly. Thank you!