Make roon more independent in regard to metadata requirements with streaming services

I was meaning to post this already for a long time, but when reading the Apple/TIDAL article I just needed to ask this.

My question/plea to roon is to make roon more independent to the needed cooperation in regard to the metadata access with possible new streaming partners. Streaming is the future and roon needs to integrate with more streaming services and still provide the USP which is in their rich metadata.

What the Apple/TIDAL article shows is that the world will change quicker and quicker and we might see TIDAL disappear (or not). There might be others willing to cooperate with roon (or not). I can also understand other streaming services like Qobuz/Spotify not willing to cooperate with roon (every business makes its own decisions).

Without a more independent solution roon might be left out in regard to streaming services. So in the long run the rich metadata with it current implementation/requirements can become the achilles heel of roon.

Please read this in a positive setting. I am a lifetime sub and enjoy roon, but this is a real concern on my part and probably others.

Maybe one of the roon guys can give their take on this.

1 Like

Part of the issue here is that generally speaking Roon/Tidal integration works so well that we don’t see what is under the hood. I suspect it’s quite a lot and any generalisation to lower the requirements of entry for other services would see a different user experience.

Perhaps @brian or @mike could tell us whether there is room to move in that direction.

though I’d rather pick a “poorly” integrated streaming service offering me all the music I like instead of a perfectly integrated one I spend most of the time skipping through stuff I don’t like at all :stuck_out_tongue:

:wink:

1 Like

I think about this quite a lot. My concern is, that as amazing as the Tidal integration is, there may be no other company willing to work with Roon with their current requirements for integration.

I’m not a programmer (beyond the basics) but have been heavily involved with developers writing code for tools at my company. We integrated with various APIs indirectly, pulling whatever we wanted and forming our own database so we could interact with it as we saw fit. Obviously I don’t know what magic happens with Roons Tidal integration, but in basic terms I struggle to see how the same thing couldn’t be achieved using any streaming services API - albeit indirectly - as long as the API was half decent.

By that I mean Roons servers trawling through the streaming services systems 24/7 via their APIs, building a custom Roonified database that Roon then uses.

Maybe I’m missing something but after this work was done once, the bulk would be there for adding other services. I don’t see how it would mean a lesser experience.

Of course I could be oversimplifying but I really hope something along those lines is possible and materialises, so we can have a bit more choice - not just for scenarios like Tidal disappearing into the world dominating monster that is Apple.

1 Like

It’s a valid concern. I’m not sure it’s as difficult to get data dumps out of companies as you would think. We’ve done it several times in the past with different players (not all at Roon…).

We have been working on the problem of broadening our streaming options since long before this latest rumor. It is a priority for us to diversify here.

As for the 24/7 trawling through a streaming service API idea…lets do the back-of-the-envelope on that.

A typical streaming service has ~60m objects (albums+tracks+artists) today.
Say it’s available in 60 countries.

On a streaming service, objects look different and have different visibility, copyright information, etc. in different countries. Lets assume we hit the same update interval that we use with bulk ingestion from TIDAL today–24 hours. That’s 3.6 billion hits per day, or 42,000 hits per second.

Not impossible, but that’s quite an infrastructure to maintain on both sides. That “doing the work once per country” thing really puts a damper on the party.

When our TIDAL update interval was wider (2-4 days), it led to widespread complaints, and even at 24h, though the complaints are much less frequent, you can still see us drift out of sync each day if you’re paying close attention. Copyright rules change day-to-day at streaming services, and displaying stuff that we are not allowed to display is a big no-no.

I think you’ve got this already, @hifi_swlon, but in case others are thinking it: here is no on-the-fly way to build the Roon+TIDAL experience. We need to efficiently answer questions like “which performances of Beethoven’s works are visible in Brazil”, and TIDAL’s data set (nor any other streaming service…) has no idea about works and performances. The only way to figure that out is to “spider” the Roon metadata universe starting with the set of TIDAL tracks available in Brazil, and pre-compute notions like “work visibility”, “performance visibility”, “performer visibility”, and “album visibility” based on the track-level rules from the rights-holders.

About 35-40% of the time we spend generating our data set is spent managing copyright data for TIDAL. It’s a surprisingly intricate and resource-intensive problem to solve given how transparent it is from the user perspective. The database tables that hold the rights data have billions of rows. Not nice stuff. It’s a challenge to manage that data efficiently when it’s nicely delivered in bulk. Dealing with it one record at a time…ouch.

6 Likes

A fantastically insightful post as always Brian.

I can’t get my head around the regional issues, but it sounds grizzly. But in terms of the data crunching, wouldn’t the API have provision to only query things that are new/changed? So once the first ‘Big Crunch’ is done, it’s a bit less intensive from thereon in?

My only comment there in that case would be, how come there’s still only one provider in that Services Tab? :smiley: :wink:

Zero pre-existing streaming service APIs are built this way, so to work with something “off the shelf” we would have to poll the whole data set. If a company is willing to build custom API to our spec for syncing up, that’s fine, but it’s generally easier to just dump the data to an FTP site.

Three reasons:

  1. Roon has been out for 13 months. Many other companies are just now comprehending that we are a real thing and not about to go out of business. It’s a lot easier to start and progress discussions now than it was a year ago.

  2. We’re a small team–too small to have every area of our product moving forward at full tilt at the same time. That said, we’ve hired 3 people in the past month. Two have started, and the third starts in a couple of weeks. Hiring and ramping up employees is time-consuming too, and that’s why releases over the past two months have been on the lighter side, but it’s the only way to get things moving faster, so we have to do it.

  3. The streaming business is complicated. Roon really only makes sense with a lossless service with a big library. There are only a few. Some are tied up in exclusives, some don’t have great name recognition, some were just rescued from bankruptcy, sometimes labels and lawyers get involved. It’s not always clean or simple to do this stuff.

7 Likes

All valid answers, and @brian the voice of reason as always :smile:

If 35-40% of your data time is spent working with TIDAL, I’d dump it. Put that time towards something else. Maybe I am still in the previous decade, but I think Roon’s strength is what it does for our existing libraries, not what it does for streaming services.

@fritzg, dealing with TIDAL data is ~40% of the time that it takes to run our daily metadata ingestion cycle. Computer effort, not human effort.

I must disagree.
In my opinion the real strength of Roon is the intgeration of local library with streaming service.
It’s so great to not longer have to think in this categories. It’s only music and I can hear it if I want and don’t even have to know if it’s local or not.

4 Likes

no one is complaining about Tidal being integrated in such a good way

just… asking to make life easier (if it helps) for other streaming services, as Tidal does not fit everyone’s tastes :wink:

btw… been using an Aries for less than a month
I always use it as RAAT so never really bothered with Lightning DS (Auralic’s remote and player)

This morning, though, as there’s a new (beta) firmware I tried it and to my big surprise, performing a search, got results from both my music library and Qobuz :slight_smile:

so integrating Qobuz in Roon… doesn’t look to be mission impossible :wink:
(Qobuz consenting, I obviously mean)

Is this still active? Any update on this? What are roons plans? I also think that the Tidal only solution is a big show stopper for onboarding others. I personally don’t like the Tidal catalogue. For me, Tidal is more focussing on main stream pop music and is not adressing people looking for high quality records. The Tomahawk solution looks interesting…

Totally disagree, integrating my own database with a streaming service was the absolute selling point for Roon to me. Streaming audio is the future, in fact for most people it’s the present, kids nowadays don’t even know any other way to play music. So yes, even more and better integration with Tidal is highly appreciated from my side. I wish the “play radio after last song in que” feature also includes playing Tidal content wich I did not add to my library. That’s a very nice way to have a personalized radio in the background while discovering new music at the same time. But make it an option, not default bahaviour, otherwise some people might get mad.

1 Like

Wow, replying to a comment made over a year ago. You’ll be happy to know I now see TIDAL as a big part of my streaming life. Of course I get it for $5/month since I am a Third Man Records subscriber.