Roon 1.4 - Radio Feedback

A “Why This?” Button that revealed the logic involved would be useful for identifying odd connections and potentially fascinating to look at.

3 Likes

Along these lines is the HEART button. I found out I was using it wrong. As I was listening to a Tack Radio stream, I was HEARTing tracks that I liked for that stream, and I was unHEARTing songs that I didn’t like for that stream. I was thinking that this was causing the Roon program to learn what I liked and didn’t like about that particular Track Radio stream (this is how most other Radio streams work). But I found out recently that I was HEARTing and unHEARTing tracks in the main library. I had to go back through my main library and reHEART all the songs that I liked.
I guess the Roon Radio system is not a ‘learning’ system, there is no feedback feature?

I thought the Thumbs-up and Thumbs-down buttons were for fine-tuning the Radio algorithm… Of course, this is just an assumption on my part, that function may not yet (or ever) be implemented… So I just use them to cue the queue at the moment.

The thumbs up/down only modify the current Radio session, (at least that is how it used to work, I think. :hushed:)

1 Like

That was my understanding. Haven’t tested it since 1.4 though.

I just tried it again and it still did the same to me as before.
I pushed the START RADIO button for 3 Doors Down. The song Here By Me played (this song is in my LIBRARY Tracks list). As a test, I thumbed down this track. After that, in my LIBRARY Tracks the track Here By Me now also is thumbed down. So further testing I told Roon to play the entire Seventeen Days album (which has the track Here By Me) and when it came to that track, it skipped it (because it was thumbed down).
So, yes, if you thumb down a track during a RADIO stream, that track will be thumbed down in your LIBRARY Tacks list and will be skipped for other play functions.

Can you post a screenshot of the thumbed down song in your library?

.sjb

I guess the proper labeling is “Favorite,” “Ban Track,” and I’m not sure what to call the empty heart, maybe “Neutral?”
After I took the screenshot I returned Here By Me back to NEUTRAL. Just like I had to go through my LIBRARY and return a bunch of tracks back to NEUTRAL because I had BANNED them during earlier RADIO streams.

Something very odd going on here - “Thumbing down” in the Radio function is NOT the same as “Banning” a track; at least here on my system. I’m often building up a queue and selecting what will be added to it by using the “Thumbs-up” and “Thumbs-down” buttons in Radio. Yet when I go to the Track browser in the library, and use Focus to list all the “Banned” tracks - there aren’t any…

Are we talking about the same buttons?

1 Like

I’ve never seen the ‘thumb up’ and ‘thumb down’ buttons on my Roon. For my system it’s always the ‘heart,’ by default the heart is always just an outline (to me this means that the track is neutral, I don’t love it and I don’t hate it), then if I press the heart once it turns solid and it is now a “Favorite,” then if I press the heart again it turns into the no entry symbol and it is now “Banned.”

OK. I looked again, I do have those buttons, but those ‘thumb up’ and ‘thumb down’ buttons are for the next upcoming track, correct? Once the track is actually playing, there is no ‘thumb up’ and ‘thumb down’ buttons.
I may not know whether I want to ‘thumb up’ or ‘thumb down’ a track until it is playing, but then I only have the heart button.
Or is the ‘thumb’ buttons actually controlling the currently playing track? If they are controlling the currently playing track, they should move the buttons away from the ‘playing next’ info.

I think the function offered by Thumbs Up and Thumbs Down is phenomenal. But the symbology is a little off and misleading.

Thumbs up and down merely determine if that song goes in next on your Queue. It does nothing to teach Roon whether you like the song or not.

As to the heart symbol, it not really learning per se either, but you can sort/search by whether you have used the heart on a track, and if you ban a song using the heart-banning symbol, it won’t play in radio.

The closest thing to learning your preferences in Roon is the counter of how many plays a Track gets. But the queue building feature by having Roon select songs to suggest and then you can use the thumbs to build your list that way is very cool and unusual.

Actually, it does prevent that song from being chosen again during that Radio session. It may have other impacts that I am not aware (like a modification of future track selections in deciding Artist or Album for that Radio session). Currently, if you do not like the song selected after it starts playing the only thing to do is to advance to the next track.

Yes it does stuff like this within the current radio session.

1 Like

Wow, I didn’t know that. Is that documented somewhere?

But it would only work if you thumb up or thumb down a song before it plays.
Why not allow a thumb up/ thumb down button before, during, and after the track plays? I don’t sit at the computer terminal indefinitely, sometimes I’m away from the computer for a few minutes doing something else while listening to the music (wow, I know, amazing right?).

1 Like

Nope. That’s not what documentation is for.

It’s not a very good idea for anyone but the developers/product people working on Radio to develop deep intuition about what it’s doing under the hood. It’s an area of active development/change, and will continue to be indefinitely. If you build precise expectations about it, they will be broken, probably soon.

That is a reason why we are not going into deep explanations here about the underlying implementation, and also why we are not going to “bare the guts” of the algorithm to a bunch of sliders and make it tweak-able. The actual knobs on the inside would not make a good product or user interface, and they change too often to be a stable target for that anyways.

Another thing that’s super important to understand about “Radio 1.4” is that it represents only about 15-20% of our actual Radio project. Internally, I actually call what you guys are using now “offline mode radio”, since that’s how it fits into the bigger picture. I call it that because what you’re using right now is the algorithm that–in the future–Roon will use to do radio within the library when no internet connection is available.

“Radio 2.0”–the whole thing, not just this small piece of it–is intended to be a proper modern radio feature–radio that utilizes information about all of the content on Earth, not just your library, a la Spotify, Pandora, etc.

At this point in time, it’s not possible to do radio properly without relying on large data sets in the cloud. Yes, I know some people back in 2008 did cool stuff within a local library (Mirage, MusicIP, …), but we would be fools to place a big bet on such a narrow-minded approach in 2018. The world has moved on to larger scale implementations for this, and we must too.

These radio systems are complex and multi-faceted. To work right, they must combine aspects of metadata similarity, auditory similarity, mood/style analysis, usage-based relationships, while leveraging explicit and implicit feedback. More than one of these areas involve machine learning and working with data at scale. Different people in this thread have identified some of these aspects, but they are all just ingredients. Work on the larger-scale project is underway, of course.

This period we are in today, where we are relying on “offline mode radio” for everything, is an opportunity to work out the kinks for offline mode and make it as good as possible within the design constraints, so we welcome your feedback, and will continue to iterate on it.

As for the current radio feature–there has already been one iteration on this feature released to the public in January, and second iteration is entering alpha testing soon. These ideas are being applied to the larger-scale system as it’s worked on as well, so hopefully we won’t see these same issues reappearing down the road when we turn on the bigger feature.

13 Likes

Hi Brian. Thanks for the insights. This may be one of the most exciting posts I’ve seen on the forum, and it’s also a little disconcerting. More the former than the latter but I think it’s fair to provide feedback on both points.

This is the exciting part. Seeing a very powerful algorithm reach beyond what is in a single library and beyond what Roon displays as metadata will be a very nice listening mode indeed. Who knows, maybe this is how SkyNet started, or this is how the aliens take over without firing a shot. But in the interim, I’d love to see what you can put together and how it does in serving up tunes.

This is a little disconcerting. I like Radio 1.3 but I’m not so attached to it that I cannot possibly conceive of living with changes.

That said, I do think people should, if they want to, have the right to understand how their actions affect the software’s response to playback. I mean, you get this feedback because people CARE, a LOT, about what music is served to them. That manifests itself in feedback not only because they want to be helpful in the development process but also because they want a modicum of control over how it’s done – how their listening session will go, and to gain that control, one needs to understand how one’s actions create responses within the software.

I’m not saying I object to what it does. But I’m not sure I agree with your statement that is not what documentation is for. If it’s to help us enjoy your product, it should help us understand how to use it and how what we do creates responses.

And in terms of continually changing it, I think that affects how people feel they can control it as well. I really don’t see an issue with a couple of settings that allow people to use the version they’ve enjoyed the most. It feels a little like you are saying that users have signed up to be taken for a ride of sorts. In a way, yes, I am enjoying the software’s evolution. But in matters of what I want my shuffle or radio algorithm to do, I think many users will desire more control than this implies.

You’re entitled to your opinion, and it’s your product. I suspect you’ll get some push-back on this point, as above. If there are no settings on your ultimate radio product, users will not be able to vary what they want based on their mood or desire to explore versus stay close to home. Maybe that will work out fine, and I assume there will still be seeds of some sort. But as above, people work really hard on their collections to gain a feeling of usability over it, and one manifestation of that, one way to provide that sense of control, is to be able to change parameters on a shuffle or radio.

So this news affects me two ways: (1) it creates a real sense of anticipation to see how SkyNet Roon Radio will be a revelation in AI DJ-ing, and (2) I am going to need to build in my own methods of creating listening modes in case Skynet starts sending terminators back through time.

2 Likes

We’re not averse to a simple control or two, so long as they are kept broad and goal oriented–but nothing like what you proposed above where knobs control the internal mechanisms directly.

We tried to build a slider into the current radio feature that controlled the tightness of the picks, and it was a real struggle to make it work well, so it was pulled from the release. The slider in our prototype ranged from 0.0 to 1.0, with 0.0 as “tighter” and 1.0 as “looser”.

One problem is, depending on the seed and the diversity of quality picks available in your library, the useful range of the slider varied a lot from session to session. In a huge library where you’re radio-ing in a part of your collection that has a lot of depth, most of the slider is useful, though below about 0.2 or so it starts to feel repetitive from session to session even though it wasn’t actually repeating tracks. If you’re in an area of your library with a poverty of picks, though, anything above 0.5 starts to feel so random it’s useless, and everything less then 0.3 or so is virtually the same…but if your library has a good depth of picks (say 2-3000 valid picks available), the slider was useful all the way up to 0.8 or so. In the cases where the useful range is narrow, moving the slider didn’t have a meaningful effect, because there just wasn’t enough content to create clear subgroups of “very related” “somewhat related” “loosely related”.

Another problem–even when it does the right thing, it sometimes feels broken. The slider was controlling a statistical probability of picking certain kinds of stuff, but it doesn’t guarantee anything about specific picks. Sometimes you’d move the slider to “tighter” and the “up next” track would go to something that–to your mind–felt looser. Sometimes it was, and sometimes it wasn’t. Sometimes our personal concept of what is tighter/looser contradicts what the data supports…and the slider, as implemented, made those situations apparent in an unpleasant way.

So the slider wasn’t that good, and it didn’t make it to production. To be clear–we are not in principle against the idea…it just didn’t work well enough to make the cut. I think it will be easier to make something like that work once we are incorporating TIDAL content–since there will be a lot less variations in radio behavior caused by differences in personal library makeup at that point. But we could also fail on the second attempt, too, so no promises :slight_smile:

I don’t think documentation is about baring the internals for all to see. It’s about helping people use the product effectively when we can’t make it self-explanatory. We like products that don’t require documentation. When docs are required, it means that we have failed to make sufficiently intuitive product.

We can do a lot better job helping people who want to understand in discussions like this one.

I’m generally not shy about sharing internals, but I think that there’s a high likelihood that if I explained exactly how this stuff worked, this conversation would devolve even further into specification-lawyering and backseat-driving and we’d stop getting useful experience feedback. I’m not about to put more fuel on that fire–there are already too many supposed silver bullets in this discussion.

Plus, I already know that once we have this stuff truly done, the explanations will be totally different, so I don’t want to prematurely build a bunch of public intuition about how this stuff works yet. It will actually be fun to talk about the machine learning stuff. It is probably the most interesting project I have been involved in so far, and it has implications that reach far beyond radio.

Yes, we are all on a ride together. In 1990, if you bought a piece of software, it stayed the same until you bought the next version. This is not that world anymore. Roon is a service, not software–and that is not just a technicality, it’s how we approach the whole thing.

Part of that service is the client software updates. Part of it is the cloud services, which evolve more or less independently. Part of it is the business relationships we maintain. Part of it is the choices we make to cultivate a useful ecosystem around Roon, and part of it is that Roon changes over time.

Not everyone is going to agree with every change, but keeping every possible permutation alive isn’t possible, and–if you think about it holistically–it isn’t good either. It’s really confusing for new users to deal with all of the unnecessary choices, and it consumes a lot of resources to maintain a bunch of nearly-dead versions of things for the people who are reluctant to adapt. Those are resources that don’t go into addressing other problems, or building new stuff.

10 Likes

Thanks Brian for the thoughtful response - it is fascinating to see inside the development process and what did and didn’t work for a pending release.

Interesting point. I wonder if you could calculate the pool of songs from which a shuffle or radio would draw, and then present slider settings that match. E;g; if there are 3000 tracks, 5 incremental settings available, if 1500, then 3 increments, and below 500, you pop a message that the pool of tracks is too small for meaningful adjustments. But of course this would mean setting it every time you shuffle, which would not work for “set it and forget it” folks.

A “why this track?” capability might help with this. Amazon does that when they pop certain products.

I don’t disagree with the way you phrase this. My concern related specifically to the thumbs up and thumbs down functions and your response which I inferred to mean that documentation isn’t for telling us what those functions are. I think there are many users who would want to understand what this does (other than the obvious of creating the queue), at least on an approximate basis. The exact algorithm, no, I wouldn’t even understand that. But if I am affecting more than the decision on what song goes into the queue, I’d want to know that.

Hey, it’s hard for a leopard to change its spots.

Sure, however you want to characterize it. I don’t disagree which is why I’ve stated concerns about putting all my object-by-object metadata into the Roon database alone as opposed to reading it from file embedded metadata. That said, I do hope that if there are radical shifts in how Shuffle and Radio work, that users have a way to use versions they liked better. Maybe that’s impractical, and I am not change averse, but there is something to be said for letting people have the version they felt worked the best for them.

Anyway, I am looking forward to seeing the SkyNet version of Radio!

2 Likes