Others from the Roon team may chip in, but the loading performance is all down to server-side caching. With a relatively small number of users at the moment and, I’m guessing, occasional cache clearouts, it’s going to be luck of the draw as to whether you hit something that has been cached or you hit something which hasn’t and therefore needs to be retrieved and processed.
We’re well aware of the performance/latency requirements here.