Distributed, mobile, cloud: my Roon priorities 1 - 86

Roon Software Discussion
81

I should acknowledge and emphasize that the complete vision I’m sketching here is demanding and costly to build and operate. Assuming the Roon team shares this vision (and there have been public hints that they do), choosing and prioritizing is a crucial step. And I leave that to the team. The essence of engineering is optimizing under constraints.

Wonky section:

In particular, I talk about the requirement for multi-master with intermittent connectivity, i.e. the ability to add music or edit metadata even when temporarily disconnected. Multi-master is famously difficult and many implementations have failed. But I believe some of the deepest challenges do not apply in the Roon case.

In computer science we talk about transaction processing. The typical example: you transfer $100,000 from one bank account to another, this has to be treated as one transaction, even though the process does two operations, deduct $100,000 from one account and add it to another; if something goes wrong in the middle you don’t want one of the operations to succeed and the other to fail because that would lose the money, or double the money depending on the order of operations. This has to be true even in the case of computer crash, or earthquake. And while the transaction is under way, if another program or person looks at the account balance, it cannot see $200,000 or $0. The solution is well known, even when the accounts are on different computers. But it is challenging with intermittent connectivity, and at planetary scale.

Roon has a database. With a simple implementation we could have problems. For example, I star an album while disconnected, and at the same time my wife deletes it: when I reconnect Roon has to deal with the confusion.

But I don’t think this is serious. It isn’t banking. And Roon already supports multiple endpoints and has to deal with the possible conflict. It is more difficult under intermittent connectivity, but the consequences of possible confusion are no more serious.

82

Interesting. Maybe it was because of entering a mature market with mature encumbants. But that makes the need to evangelize the mission of roon even more important. We aren’t the same. We are roon. We help you see, touch and hear your music. Let me tell you how we do that…

83

Hopefully roon could do something elegant and simple. A hierarchy of operations? Storage is cheap so is deleting merely removing a reference to a location and if someone else manipulates that file then it doesn’t get deleted.

The synchronous edit scenario would be a very small use case (I’d hope). So having something like office or google docs or share point would be overkill for it. Last edit wins is simple, but could be problematic.

Maybe a github model. You go offline and you’re effectively working on a fork of the library. Maybe your wife and your profiles are forks from the outset? When you’re back online it’s like committing changes.

Git could work. Sorry. Just thinking loud.

84

Yes. These modern paradigms accept the possibility of conflict and inconsistency, because in modern distributed and large scale configurations attempting to eliminate them is futile. The solution is to learn to deal with eventual consistency.

A colleague of mine wrote, the essence of business is having to apologize. Imagine you are Amazon and you have only one example of a unique vase in the warehouse. If you have the possibility of inconsistency in the database, you may sell that one vase to both Alice and Bob, and then you have to apologize to Bob. Or you don’t have that problem, you sell it to Bob, and then the forklift in the warehouse drops the vase and backs over it — you have to apologize.

1 Like
85

We may as well throw ML into the mix. Like the comment earlier about the social aspect.

The product can anticipate what you want to do or listen to - like recommendations and radio. But when it “guesses” you should be able to a) get back from the state it got you do (undo / remove) and b) easily edit to fill in the remaining 20% it didn’t get right. I guess this is a bit like the software apologizing. Sorry we don’t have that track - would you like us to get it again.

I still reckon the play buttons should remember your preferred actions and default to them. Learn the user and anticipate them a bit.

86

Good example. It’s pretty clear that people who aren’t serious/passionate about music aren’t going to care too much about the relationship graph. So this feature — central to my Roon experience — is effectively a niche feature.

Similarly, younger, less affluent folks aren’t likely to care as much about multiple zones on the LAN, since their domiciles are smaller.

This all points to the “Roon everywhere” concept as the potentially big, non-niche feature for the future (but maybe not the near-term future, because of current technical limitations).

87

A few years ago, I was working on a side-project that was more-or-less like Google Keep (before it existed) but with bigger ambitions. It used a decentralized versioning model modeled on git to facilitate offline use. I learned a few things about repurposing git’s database model while trying to make that work…

Git is a little bit too clever. It uses every weird unix trick in the book to be as fast as it is. It requires low-latency access to the database (i.e. local SSD). Take it out of its home environment, and it suffers. It’s really a brilliant design. I respect it a lot–but doesn’t transfer well to other domains.

Roon’s approach to consistency could be extremely lax, even poorly executed, and it would be nearly immaterial to the product/experience.

This product is not very conflict prone. Metadata editing is very rare. Editing “on the go” will be rarer. Independent people editing the same thing asymptotically rarer. Disconnectedness is becoming rarer. And all of these will become even rarer over time. Can’t be totally ignored, but should not drive the design.

The important stuff: favorites, bans, ratings, play history, social interactions all have trivial conflict resolution schemes that work well.

–

7-8 years ago, I built a mobile app for Sooloos (that never saw light of day). As you may (or may not) know, Sooloos (like Roon) keeps lots of information about the library indexed in RAM. Since I was targeting a circa 2009 smartphone that was obviously not going to work, so the scheme worked like this: have the “fast” core generate a little embedded database that was optimized for exactly the queries the mobile app needed, then sync the whole thing as a bulk transfer on each sync with the core.

Like git, another clever trick with a lot of limits.

This had some really interesting offline possibilities…having the whole music collection on the phone–even if not playable. I could pick (or unpick) stuff for sync on the subway and have it transferred next time I was at home. One night in a bar, my date spent the night scrolling through my music collection and discussing my choices at length.

At the time, it didn’t scale. The database got too large, and the sync too slow. Too much tension between performance and capability–every little extra piece of data cost too much in the sync time. Technical constraints pushed the product to suffer.

Some of the hard problems concern how we approach “the whole library”. The right experience is obvious–same thing I was going for back in 2010: same library everywhere, regardless, with variable content access.

Still, it’s hard. Maybe the cloud (and requiring a connection for full access) is part of the answer. Solves some problems, but it isn’t a silver bullet. It’s not as if cloud storage has latency an order of magnitude better than a phone. It’s not as if cloud CPUs are an order of magnitude faster than a phone. And the phone has a locality advantage managing just one user’s data.

The distribution of library sizes in our user base might be surprising. Lots of people at the bottom, some in the middle. Large ones are rarer than you’d think. There’s 2 orders of magnitude between the median and the max library sizes in our user base.

7 Likes
88

@brian - thank you for the post - very enlightening. The roon blog doesn’t get loads of posts - but this is the sort of thing that I really enjoy. You guys are experts in your field and to get an insight into your thinking is great. I think one of the goals of roon was to be as open and transparent as possible - within reason.

I did a similar project a while ago and agree with you on the git front. Sorry - I’m all about the experience. The user doesn’t care about the clever stuff under the hood as long as it works, and it doesn’t impose some awkward workflow on the user to make them fit the technical solution behind the scenes. Git was less “can we use Git” and more that building something like Google Docs simultaneous editing is a hammer to crack a nut, and a nut that we aren’t going to encounter too often. It is a niche use case. The lighter weight the solution, the easier it is for the user to use (generally). Sure it also increases chances of conflict in those very rare occasions. Like Anders says - thats the time for the software to apologise. Using his Amazon analogy we don’t say “Hey how could you sell out of it? Why isn’t your system connected to the supplier so they had more units manufactured on demand, and then shipped across the world…”. Could it be done? Sure. Should it be done? Don’t think so.

I think the thing that has changed from the time I did this in a mobile environment is the cloud. When I started out it was a case of the client had to be reconnected at least once every 28 days to the server and would get its license information and sync when it did so. And sync was like shipping database logs and then playing them. With the cloud and wifi / 4G we are seldom truly disconnected. And when we are, it is for short periods of time. Also we don’t need to physically connect like the old client / server models. So the log becomes more a message queue to the cloud, which can sync back to the core at leisure and vice versa. I don’t think it is “my wife just changed the tags on an album so if I pull down on my phone I should see that instantly.” This isn’t “Hey I’m sending you this important doc now that I need you to look at…”

Build the lightest weight solution that works. Much like building race cars - often the “simple” solutions are the best. But designing simple, is hard. And much like racing cars, the job is done when you’ve removed everything that isn’t needed… and then the process of starting to find new ways of doing things so that more things can be removed begins :smiley: - that’s what ML is for me. We built a car, and now we take new tech which means all those buttons there that are confusing and distracting to the driver can be removed…

89

:rofl: Beautiful, a scenario I hadn’t considered.

“…conflict…should not drive the design.”

Agree, I mentioned it just to put it aside. I think multi-master is essential, a simple form of multi-master illustrated by the scenarios I mentioned, and lots of people panic because it is hard, but it is less hard when conflict resolution is not a driving consideration. I wouldn’t want to give up on that, disallowing “like” on an airplane. Or more seriously, disallowing adding a new album while on vacation just because the home server shut down.

90

This, becuase if you want to attract and connect with the ‘new’, young audience referenced in your OP then Roon needs a payment model that will draw them in. And like it or not, a monthly subs option is the best way to do that.

91

I can understand the business case to offer a subscription model on a yearly basis or lifetime basis for new startups. Once established though, I suspect adding a monthly subscription option would definitely attract a new demographic. The other shoe would be the dropping of the lifetime option - something that has already been hinted at by the Roonies.

92

The monthly model will massively increase churn. The burden of support will be a lot higher. And the methodology of killing software out of subscription will need to be pretty robust. That is a lot of work!

93

You may well be right. Perhaps the answer is the compromise model that has a contract that is annually renewable, but with monthly payments.

94

I agree completely with the sentiment. Unfortunately, I’m way past middle-aged, but the rest of the profile probably fits. I also suspect many of Roon’s core customers fit this profile.

I became a lifetime subscriber as soon as Roon beta was released because Roon’s promise perfectly matched what I had sought for years.

  • Bit perfect (24 bit) audio delivered to multiple end-points
  • Well integrated metadata based on All Music Guide (Roon’s primary metadata source) reviews and database, and cd quality streaming (again, well integrated).
  • Best-in-class library management tools (especially hyperlinks)
  • Acceptable ease of use

The Roon team would benefit by building on these core strengths, and not trying to be something for everyone. Trying to out Spotify Spotify is a losing proposition.

I am in complete agreement with AndersVinberg on his list of ways Roon could enhance his quality of life. It would be a home run if Roon could deliver on making “the Roon experience” available anywhere, anytime, with no hassles or glitches. I’m not sure Roon can deliver on this promise, though (at least in the near term).

For me, I think I can get everything I want with a relatively simple enhancement to Roon. I don’t need Roon controlling Tidal streaming 24/7. All I need is an easy way to copy music I own in my Roon Library to other media storage devices. I want to be able use Roon to filter, make playlists, etc. and then easily copy the Roon filtered music files (with tags) to my portable music player or another hard disk. Also, tools that make syncing my Roon library at my vacation home would be of great value to me.

For me, the most important aspect my mobile library is audio quality (the first on my list of why I am a lifetime Roon subscriber). I would still be using Sonos it supported 24 bit audio. I would be streaming Spotify if I was happy with its sound quality.

If Roon can deliver bit perfect 192/24 audio everywhere,at no incremental cost, I’m all for it.

On the other hand, my work around would keep me happy and it seems like it would be trivial for Roon to implement.

  • For my second home I bring a hard disk of what is new in my library and copy my new music to my NAS at my second home. It would be great if Roon could make it easier to identify my “new” music and make it easier to copy that “new” music to my other NAS.

  • I use a high quality portable music player for a resort or hotel room (Ayre acoustics Pono Player was a great value with DSD and 192/24 and very good SQ for $400) - if Roon just made it easy for me to select and copy my music to my portable music player my life would be better.

  • Boat - same portable music player

  • Office - Roon works great with a pc, a good DAC, and high quality (mine are Focal) active monitors.

  • Hotel Room - same portable player

  • Airplane - same portable player and high quality headphones

  • Car - same portable player with aux input into a good audio system

  • Running - same portable player with high quality ear bugs

  • Working outdoors with power tools on my property - same portable player with ultimate ears custom fitted ear bugs and noise protection - outstanding audio and you won’t go deaf!

  • Exercising in the gym - a Roon endpoint in my gym

I want “serious music listening” in all of these situations. Otherwise I’d rather not listen to music. I’ve tried listening to my iPhone, but not for more than a few minutes at a time.

Based on what I’ve read in this forum it seems like I’m the only one with these priorities. That seems odd. I would think that others were attracted to Roon for similar reasons that I was.

Spotify will always win for those who value what they offer (especially the social networking stuff - you need their huge user base for that to be of any value).

Sonos will win with easy to use multi-room play.

I would encourage the Roon team to keep doing what you do well - improve on what differentiates you! You’ve been very disciplined in pursuing your vision so far.

I couldn’t be happier with Roon to date. I’m getting more than I expected and nearly all of your development has been right on target. Stay on course and build on what you’ve done.

Please don’t forget your core user base whose loyalty is based on delivering against the value proposition you put forth a couple of years ago.

7 Likes
95

Exactly. These are by far at the top of my wishlist. As it is (I mentioned that earlier), I use a custom Python script to select what to put on my DAP (QP1R) for travel, and I use a Synology USB copy program to create a clone of my music files for my (non-Roon because RPi-based) work system.

2 Likes
96

Hi Brian,

I would really love to see this sort of implementation just for Tidal albums, as a stop gap until bandwidth issues allow local content streaming to really catch up.

Am finding I am using Tidal a lot more than previously in parallel to my static collection. Playlists for mobile use, etc. And this just isn’t supported within Roon. It’d be really nice to consolidate my work on the Tidal playlists within Roon, seamlessly.

In addition, I’ve just ordered a Fiio X7ii DAP for on the go use and will be limited to the Tidal app. I’d REALLY like to see a Roon dedicated app for Tidal on the go. The modern DAPs are fully embracing streaming nowadays too :slight_smile:

Cheers!

97

Great topic! Took me a while to read through. Had many thoughts on the way to get to the latest post and will not try to reconstruct them now. What sticked was all about the Roon experience that should become ubiquitous. For me it’s the database. This is the part I’m sticking on when I would be asked what would I loose when Roon closes and eventually would give us 2 weeks to take with us what we can. I would try to find a way to save as much as possible from that database I can reproduce anywhere else. So, Roon becoming ubiquitous for me is mainly a question of how can I use my Roon database ubiquitously.
How can I get my Roon experience that is somehow stored in my database to any place?
How can I use my Roon database with any streaming subscription, not only Tidal, but also Qobuz, AppleMusic, Napster, Spotify or YouTube? Yes, YouTube as I see many younger colleagues using YouTube as their main content provider, it’s free and exhaustive.
How can I use my Roon database with my mobile devices, when offline and when online?
How can I use my Roon database for playing the largest quantity and highest quality possible in my current environment (airplane, taxi, own car, holiday resort…)?
Roon needs built-in mediators that facilitate to reproduce my music wishes, choices, my concrete selections, my sound setups (filters) in any environment I am or move. For playlists there is souniiz that allows transformation from streaming service to streaming service. Similarly, Roon should run such a matching tool in the background that transforms my playqueue on the fly with all my available sources and services I have at the current location, being on local storage, in internet or in the cloud.
All my currently available data, being subscriptions, freely available, licensed (e.g. ripped or downloaded files) should be used to allow me ‘working’ on my Roon database. This includes selection of preferred quality (audio and interpretation), modifications on metadata, selection of what play next, through which output…
The dimension of location would need to be included. Roon should remember in history what output means I used, what equalizer setup was linked with. Next time I’m back in my holiday appartment Roon knows which of my subscriptions work, which DAQ I prefer.
I should be able to add to my Roon database data that I gather while listening to music, being internet articles related to what I search and read while listening, notes I take reg. quality, or market places I scan to find where I can purchase what I listen in better quality…
This is roughly the experience I whish from Roon and I get now in a fraction of it (limited locations, limited access to my subscriptions, limited access to my own sources). But still, it is only Roon that gives me by now this fraction of access to be able to enjoy such experience. This experience is what no streaming service or other offer so far and in future can take from Roon because Roon cultivates my database. The more they offer to add to this database the more I will depend on them. This is the downside.

3 Likes
98

Step 1, fix playback on iOS.

Step 2, allow servers to be accessible online and expand RAAT so it streams via WAN.

Step 3, PROFIT!

Your phone becomes the endpoint regardless of where you are providing you have a cellular connection and your Core is online. On a boat? In a car? No big deal. Access your Roon Core anywhere and playback via your mobile device, regardless of what it is or what it’s plugged in to.

Offline access totally goes against the UX of Roon as a whole (and when I say “offline” I mean, when your Core is offline, or when you don’t have access to it). Picking and choosing a selection of albums you want to access offline is a nice idea, but seriously just export them to iTunes or whatever it is you use. I want access to my entire collection when I use Roon, as this is how the experience is currently shaped. The times I go bush or catch a flight and don’t have access to the internet I’m happy to use iTunes to play music I’ve exported (Roons export functionality is truly excellent IMO and we’re all pretty familiar with it by now).

I honestly don’t think this needs to be a major rework of the experience or a redesign of the app. Currently we can access Roon anywhere on our LAN. Add in the multitude of pieces that will let us connect to a Core outside of our LAN. Plex handles this very well as an example. I know it’s not as simple as all that and I’m not claiming that it would be easy to achieve technically. I’m claiming that the UX doesn’t need to be changed in a big way.

But yeah, kinda need to be able to playback on our iOS devices as that’s going to be the cellular connected device that has access to our Core.

99

The topic I don’t see being discussed here is robustness of the core product. For me, it’s been slow going to get the family to use Roon, but slow and steady progress is being made on that front. The enthusiasm killer is the ‘Core not available’ type messages on iPhone/iPad.

With the latest drop of core (iMac) I now have no connectivity (again) when the display sleeps and when I wake it we get ‘no audio devices configured’. Opening the settings and waiting rsolves the problem - but it should not be this hard.

At every drop (I at least) seem to get new issues. The core before this week’s drop completely locked up the iMac it runs on. In my experiecne completely killing a Mac is a challenge.

The correspondents who point out the ‘experience’ are on the money, IMHO. There are many things I’d like to see included in this list (and many not!), but they all depend on a robust and usable system.

100

Interesting, I was just musing on the reliability of my system. I have not touched, or adjusted, or rebooted, or restarted my Roon Core (a NUC running Windows) since I set it up, must be a year and a half ago or so. I have frequent power outages, and everything restarts when power comes back. I used to have WiFi issues in some rooms, all went away when I installed Eero.