Roon Search Overhauled [2022-12] Update

I’ve answered the “why can’t we run the servers at home” question before, so I’ll paste it here–

I very much doubt that any significant number of people have the infrastructure to run this stuff at home. We’re talking about roughly 100 CPUs and a few terabytes of RAM just to spin everything up and serve the first request.

The most costly part would be the elasticsearch cluster, as that scales with the size of TIDAL/Qobuz catalogs, so you would need about as infrastructure as we are running even for a very light request load.

We’d be shipping 500GiB of data to you each day to update the streaming catalogs + machine learning models. The cost of that data transfer alone would be around $1200/mo. Even if we could somehow pay that once for all users using this mechanism, we would still need hundreds of users doing this just to offset the bandwidth cost.

And then…we’re talking about managing 50-100 backend services and about that number of deployments per day across the team. You can’t just spin it up and hope it stays running. To keep stuff like that going requires cluster management, monitoring tools, etc.

And then we would have to figure out what to do with all of the “Secrets” required to make our stuff work. We can’t publish private keys to third parties. Not our own, and not those of our partners/vendors which live on our infrastructure.

And then we would have to figure out a way for you to replicate certain resources that can only be cloud-based, like object stores, pub-sub queues, load balancers, and more, since there are many aspects of our infrastructure that are provided by GCP.

Replicating and maintaining our our infrastructure elsewhere would be a full time job, and that person would require a team on our side to productize it, document it, and support it for that kind of use. And then everything we do from now on would be slowed down by the need to support this use case.

Also, I’m certain that we would have to substantially reorganize some things to be able to do this without running into privacy laws, as many of those are quite strict about transferring data to third parties.

This project is so expensive and impractical, that to justify it we would need to think that it would double our user base. Realistically, if we released it, I doubt that anyone would actually go through the costs and trouble to successfully operate it, given the complexity.

8 Likes