Okay, starting an extended debugging session today. Hopefully I’ll be able to narrow down on what’s going wrong. I’ll probably send a sequence of messages out rather than a single massive update.
First, DNS:
- Yes, I’ll convert my local BIND server to only use my ISP’s dedicated upstream DNS servers rather than ever going directly to the Root Servers [A-M]…ROOT-SERVERS.NET. The default /etc/bind/db.root contained pointers to those and that’s pretty insane for my usage.
However:
- The TTLs on the *.roonlabs.com and *.roonlabs.net DNS entries are insane:
1m accounts5.roonlabs.com 5m bits.roonlabs.net 5m devicedb.roonlabs.net 5m identifier.roonlabs.net 5m imagecache.roonlabs.net 5m metadataserver.roonlabs.net 5m push.roonlabs.com 5m updates.roonlabs.com
- Wow. Just Wow. The number of times that we do DNS lookups on account5.roonlabs.com is simply staggering. And I presume that every single one involves a call back to the Mother Ship. I presume that these are to prevent people from using the Roon Licensed Software illegally, but man, put a timer in there and only check every five minutes or so. Yech. As Roon becomes more popular, this will not scale. Your servers will crack under the load. Someone needs to take this on as a priority task immediately if you want to survive success. Here are the counts of the number of times DNS lookups were flagged in the Roon Server Logs going back to December 25, 2018 (about 12 days):
1019 accounts5.roonlabs.com 450 bits.roonlabs.net 75 devicedb.roonlabs.net 2293 identifier.roonlabs.net 19 imagecache.roonlabs.net 4672 metadataserver.roonlabs.net 784 push.roonlabs.com 26 updates.roonlabs.com
- Even with a transient DNS failure, that shouldn’t result in the server completely going out to lunch. Maybe if you haven’t been able to contact the License Server in a reasonable amount of time, say 5 minutes if you want to go crazy, sure: throw an explicit diagnostic and stop working, but only then. And even with that, how would I listen to my own music stored on my local storage when there’s a network outage? Honestly, I think you need to come up with a far better Licensing Scheme that doesn’t require going out to the network. The Roon Software should work if all the local components are connected and healthy even if we can’t get to the InterNet … There are many far better — and scalable and stable and fault tolerant — mechanisms for ensuring that someone has a valid License Key.
Okay, on to more debugging …
Casey