Untangling artist/artist/artist

Roon Software Discussion
1

I have suggested a more liberal approach to which artist should show as the “owner” of an album. In today’s world, there is a single artist, but that is limiting. In principle, I would like to see all “important” artists in the artist list and even in the album list, so that an album would end up in multiple places. But in practice, it is not easy to figure out which artists to include – we can’t got crazy, don’t want 10,000 artists.

But I have found two cases where the system just misbehaves right now, because of the way the metadata is set up.

I have a number of albums by Norma WInstone, and on some of them she performs with Klaus Gesing and Glauco Venier. All the albums are listed under Norma, except one which is listed as "Norma Winstone / Klaus Gesing / Glauco Venier. Roon parses that string and recognizes the three artists so I can click on them, but it chooses Norma as the owner. But the results are random: when I look at this triple-owner album, the other albums by Norma show up under “By This Artist” but there is no Appearances list because the system doesn’t have any other album that is listed by those three artists. Ok, this is logical but annoying. But if I look at the other albums by Norma, this triple-owner album doesn’t show up. And there are two other albums that in fact have those three artists but are not listed that way, listed under Norma only, and the system doesn’t recognize that it is the same group. And what’s the reason this album doesn’t show up under Klaus and Glauco? Random. The only because I can make sense of this is because I know the structure well. I.e. the metadata database is in my head. This is not a rare case, I have lots of albums with multiple artists listed like that.

Another example: I have lots of albums by Miles Davis, but some by the Miles Davis Quintet. The Quintet albums do not consistently show the Miles albums under “By This Artist” or Appearances. It actually show a few albums that are listed under “Miles Davis” but have titles like “Relaxin’ with the Miles Davis Quintet”. It’s random stuff, hard to unravel. And I think it’s because of the insistence on choosing one owner. With a looser attitude, things would make more sense.

Of course, if I navigate through Credits, I can find everything. But that is an explicit action, I don’t do that every time I look at an album.

So this confusion about the owner undermines “serendipitous discovery” and brings us back to navigation based on my having the database in my head. Which is what everybody else does and Roon aims to solve.

So maybe these two specific bugs will encourage a rethink on this ownership issue.

3 Likes
2

Actually, it is worse. The Norma/Klaus/Glauco situation leads to trouble. Two albums are listed under all three artists, and as a result all three artists show up in the Artists list. But Klaus and Glauco have only one album listed. When I look in the Albums list, they only show up under Norma. And when I look at an album that exists in two versions, 44 and 88 kHz, the “Other versions of this album” sometimes shows two versions, sometimes three, depending on which I am looking at.

So it isn’t just me, the system gets confused too by this situation.

3

You have a great eye for detail, and you’ve identified some real issues. I have an advantage because I actually have the database in my head, so let me explain a little bit.

Under the hood, we treat the main artists of an album as an ordered set. The linking that you identified as string parsing is actually first-class in the data model. We do support multiple owners–but–as you’ve noticed, we haven’t always used the data in a consistent fashion.

It wasn’t always this way–virtually all common models of music metadata have a single artist (represented inconveniently as a string), and for the majority of the time when this software was evolving (1999 till perhaps mid 2014), we did too. Transitioning to a model where an album had multiple artists didn’t come without the requisite friction, kicking, screaming, and transition pains.

I’ve always felt a little bit icky about making it an ordered set–but I haven’t found a way to escape ordering. Without ordering, we can’t make the display of the list repeatable, even in a future where the user is capable of editing the set of artists, and might justifiably want to control the order in which they appear.

The abusive thing is that we use the ordering for other purposes.

The design decision that says “a browser will contain exactly instance of each element” is a very old one, and something that we periodically challenge, but haven’t moved on for a long time. There are some potentially nasty UX consequences of moving to a browser that accommodates multiple instances. One is that if you have an album by “Miles Davis” and “The Miles Davis Quintet” it’s very likely that you’ll be able to see both on the screen at the same time, which many people would identify as “clutter” even though it’s technically correct.

I think we will eventually encounter a convincing reason to create “multi-browsers”, and that some of them could be very interesting. We haven’t gone there yet. Maybe it’s a setting for people who prefer cleanliness to correctness.

I believe that we also use the first artist to drive “By This Artist”. This is stupid, and an evolutionary artifact. Likely, the code for determining that list took one artist as an import and I’m guessing somewhere there is an album.artists[0] where there shouldn’t be.

“Miles Davis” vs “Miles Davis Quintet” is a troubling problem. Our data model supports the notion of multiple owners (i.e. if we exposed an edit UI, you would be able to edit all of your albums to be “owned” by both), but our data sources (the best of which also support multiple owners for an album) are woefully inconsistent in populating the data.

There are also people who would call it a bug, or at least redundant, for us to auto-populate a bunch of albums as “Miles Davis / Miles Davis Quintet”

Personally, I don’t find “Miles Davis Quintet” the be useful information, because it’s not an easily understood term. He had several well-formed quartets over his career, each of which is a distinct group. Lumping Relaxin/Steamin/Workin/Cookin with Miles Smiles feels disingenuous. If we’re going to get it wrong, I think I’d rather err on the side of calling everything “Miles Davis” if we can’t accomplish a clear naming scheme for his various groups.

The last thing you pointed out (with alternate versions) is just a bug. If I had to guess, it’s the consequence of a recent performance optimization that narrows the search for alternate versions to reduce the number of pairs that must be considered…using main artist information.

You’ve given me some food for thought. I’ve made some notes. Expect this to improve over time, and thanks for taking the time to think about this critically and give this sort of feedback.

2 Likes
4

A long time ago I had a teacher who told me there are only two data structures, trees and chaos. He recommended trees. And for a long time, I followed his advice and built systems based on trees. But eventually it became clear the world is more complicated. If we have to deal with chaos, we have to learn to control chaos.

You’re right that I’m really raising two separate issues. The Norma Winstone situation: you do support multi-owner but you have some implementation flaws. But the Miles Davis Quintet situation is different. The Quintet is not a robust or durable identity. It is just descriptive, belongs in the album description but not as an artist. Hadn’t thought about that.

But what is the right way to handle it? In principle, we should honor the artist’s (or publisher’s) choice. So Norma Winstone, John Taylor and Kenny Wheeler played in a group that got a name, Azimuth, and we should use that name. But for some reason, the Winstone/Gesing/Venier trio didn’t get a name, just played under those three names. Not anybody’s trio, just the three individuals. Implying a democratic situation. Norma Winstone also recorded an album under her own name, with John Taylor and Tony Coe: the use of her name indicates a leadership situation. These are the artist’s prerogatives and we should respect them.

But The Supremes evolved into The Supremes with Diana Ross, and later Diana Ross with the Supremes, and eventually Diana Ross. The evolution of artistic and commercial and political power may have been significant to the people involved, but I don’t care and I don’t want the albums to show up in four different places in the catalog (if I had any Supremes albums).

So it is tricky to decide. But in any case, we have two different issues.

  1. The system needs to support complex relationships including multiple owners – the world is a DAG.
  2. Curating the metadata to represent the data in a sensible way, respecting the artists’ choices.

Curating is labor intensive and boring. The solution should not be based on that. When we have to, we’ll do it and the system should support it. But automation is the key.

If I wanted to be a curator, I would have been one.

1 Like
5

It is just descriptive, belongs in the album description but not as an artist. Hadn’t thought about that.
…

So it is tricky to decide. But in any case, we have two different issues.

  1. The system needs to support complex relationships including multiple owners – the world is a DAG.
  2. Curating the metadata to represent the data in a sensible way, respecting the artists’ choices.

You have convinced me that we need two fields for album->artist:

  • List of primary artist links
  • Editorial artist string

How we use them in various situations is a series of product and UI decisions, but having access to both, separately in the data model, seems like an obvious increase in expressive power.

Doing it this way neatly solves some other issues. A huge user acquisition problem: there are many people who will not seriously use the app unless it preserves certain aspects of their pre-groomed content.

The knee-jerk reaction is to ask for a big red button that puts the whole app into a “my stuff” mode. This is reactionary, and I don’t think that many of them actually want that–they want a sane blending of their stuff and our added value–a much harder balance to strike.

We have inoffensive solutions for letting people elect to “prefer” their data for simple fields like album/track titles, cover art, release date, and user-defined genres.

We must also solve it for albumartist–and letting them blow away the artist links and replace them with links to data-less artist entities like “Miles Davis / John Coltrane” that lack global identity does not lead to a good outcome, even if it’s what people think they want.

Separating the concepts makes a very clean solution to that problem. The user’s data can go into the string, and the artist links can co-exist separately.

This also may help solve a problem related to handling poor quality source data in our metadata databases–in some of our metadata sources, we get a list of main performers, in some we get a string, and in some we get both side-by-side. Currently we are doing evil–we try to normalize everything to a list of links. It has nasty consequences.

We’ve had our heads in this data for years, and we have not, for whatever reason, ever considered making both concepts first-class and separate. This won’t be fun to fix, but we have to fix it.

6 Likes
6

Yes, that sounds right.

I was going to caution you about not requiring metadata grooming, just allowing it. You know that, of course. On the Meridian forum, somebody pointed out that importing 1,000 albums into Sooloos takes 5,000 minutes while Roon takes 10 minutes during which you can go off and have coffee. I think this is because Sooloos massages the original metadata it finds to get it to conform to a restrictive data model, and if this massaging goes wrong (which if often does) you are in deep trouble. Roon needs to do less massaging because of its richer data model: if the metadata is poor the situation is still not catastrophic.

You know this of course. And I think this change will address one limitation that is behind some of the problems that crop up.

Having the right conceptual data model is key. With that, we can take our time to fine-tune the algorithms to deal with all the wacky edge cases.

For example, the “work” concept in the data model is brilliant. I have filed an issue where the algorithm for works doesn’t work, but the algorithm can be tweaked.

Interesting problems. And very good solutions.

7

Actually part of the Sooloos import issue is that the music files themselves need to be physically copied to the Storage. With Roon the files are already there and Roon is merely identifying and importing the metadata for those files.

8

I agree with the notion of respecting the artists/ publishers, but at the same time allowing user override. As a simple Blues/ Pop/Rock related example let’s take the case of Fledtwood Mac in its many incarnations. There was what could be termed the Peter Green era, Bob Welch and Danny Kirwan and subsequently the Buckingham/Nicks era. Many of the artists that have made up FM over the years also released solo albums or collaborated with other musicians/ bands over the years.

Using Logitechmediaserver in the past has meant that how my music was represented was entirely at my discretion. I ended up making heavy use of albumartist and artist. This meant that I would for example be able to assign albumartist as Fleetwood Mac whilst assigning individual artists to the artist field should I wish to. Doing this for FM may be a little superflous, but it means I can easily browse to all albums that Peter Green appears in - solo albums, collaborations with others as well as those FM albums he appears in.

From a browsing perspective, browsing album artists should be distinct from artists and only yield primary artists, whereas browsing artists should yield both albumartists and artists that have never released an album in their own right. This way I get the best of both worlds.

When browsing a band like FM the band members for that particular album should appear in the album details and selectable as a means of drilling down to that artist as an entity.

Browsing artists should present lists distinguishing (however described) between:

  • Their own albums (primary artist/ albumartist)
  • Albums they’ve appeared in as band members
  • albums they’ve guested on
  • collaborations
  • compilations/ VA albums

In the case of collaborations I often assign all artists as albumartists so the album appears under the primary artist listing of each artist.

Having a means to modify/ amend Roon metadata en mass to give effect to such a scheme or any other a user chooses to adopt is in my view essential for those that want it.

2 Likes
9

Interesting ideas for the pre-Roon world of LMS, squeeze etc. I understand what you were trying to do.

I think the Roon is different though. You can click on FM and then dive into Green, Buckingham, Nicks etc and see all the tracks and albums that they appear on. You can see all other bands and artists they were associated with.

I don’t think you need to manually force these links like you did when you had a flat structure like album artist tags only. Roon is doing much of this for you.

10

Yes, this is a central value of Roon and of Sooloos before that. Being able to browse the contributing artists leads to serendipitous discovery which is the reason for all this. Navigating a simple tree of album artists and albums is ubiquitous today, everybody can do that including my phone and my car. The world is more complex than that.

I agree with Nick that Roon changes how you do it. No longer a need to curate the metadata because if Roon, or its metadata services really, get it wrong, I.e. If they set up a structure that is not ideal, we can still navigate and find everything,

But I’m still not really happy with the navigation. We can only browse by the album artist. Contributing artists require other navigation techniques, like Appearances and Credits. All good, clickable hyperlinks. But I would like to be able to browse by the other significant artists. This means the FM albums would appear under F and under Peter Green. The problem is identifying who is significant, I don’t want all the people credited like the engineer and the A&R guy and the “current distributor” (the very title indicates it is ephemeral).

Different ideas for finding “significant” have been discussed.

Note that the first idea is automatic in Roon: any artist who is “album artist” at least once shows up in the browse list, and then you see other contributions. This takes care of most cases.

The problem is artists that only appear as sidemen in my library, they are not elevated to the browse list, so you have to look for them. In many cases, once you click on them, there are more albums out there, some where the guy is leader. Often happens with drummers and bassists and such, and many categories in classical.

This is the interesting challenge.

11

[quote=“ncpl, post:9, topic:1206”]
I don’t think you need to manually force these links like you did when you had a flat structure like album artist tags only. Roon is doing much of this for you.
[/quote]Agreed, I’m generally happy with the way Roon handles this kind of thing, however, it would be great to be able to browse other contributors without having to first get to an album they’ve contributed to. Additionally I don’t believe Roon should try to distinguish who is or isn’t ‘significant’ - what/ who is significant to one person may not be to another. Every musician should either be an albumartist and/or artist and accessible without having to find something they’ve appeared in.

12

The “user decides” approach of course gives you all the power. The problem with that approach is that it turns us all into curators and librarians. I did that for years with Sooloos, and it was grueling work and boring. The best part of Roon, in my view, is that the power of the user interfaces with “appearances” and “credits” is that it isn’t necessary to have the metadata exactly right, it works anyway. So I can relax and let the system do it automatically.
Somebody wrote that importing 1,000 albums into Sooloos takes about 5,000 minutes of hand-grooming; with Roon it takes 10 minutes while you get a cup of coffee.

We agree on the value of being able to browse the secondary artists, not just find them through credits – I have pushed this idea to the Roon team for some time.

But I want a reasonably successful automatic model, to follow up the core value of Roon. Manual override is ok, but the system shouldn’t rely on manual grooming.

And if you include all the credits, with 1,000 albums you get a browse list of 20,000 people, many of which are not interesting (liner notes translator, guitar technician, current distributor, A&R…).

From a practical engineering perspective, I might consider something like this:
0. Allow browsing of any artist who appears at least once as album artist – the system does this now.

  1. All the user to manual “promote” some other artists to the top-level browse list: open credits, click on a bassist, and click on a Promote button; clumsy but a useful first step
  2. Some algorithm that analyzes the data in the library and figures out interesting people to promote, such as contributing artists who appear with several primary artists, e.g. Paul Motian who played drums for lots of people even if you don’t own any albums by him
  3. Some algorithm that looks at Tidal or other libraries, and promotes secondary artists in your library who also appear as album artists in Tidal
  4. Cloud-based correlation: the metadata of all Roon users is uploaded to the cloud (anonymously) and analyzed there, so an artist that appears secondary only in your library but is primary in many other people’s libraries get promoted. Tricky to deal with privacy issues, may need opt-in, which might trigger a “tragedy of the commons”.

Something like that might work.

1 Like
13

[quote=“AndersVinberg, post:12, topic:1206”]
And if you include all the credits, with 1,000 albums you get a browse list of 20,000 people, many of which are not interesting (liner notes translator, guitar technician, current distributor, A&R…).
[/quote]In my post I referred to musicians as opposed to the data points you refer to above (which I agree are uninteresting). Such data points should not qualify as artists and if exposed should be dealt with as separate entities for those that may find them of interest.

14

This all begs an interesting (and difficult) question: what makes someone an “interesting” artist?

Right now, we have a concept like this: artists can be “main artists” (meaning, they have an album), “interesting performers” (meaning, they are credited in a performance capacity on an album or track), or “others” (people only credited as composers or in production roles). The artist browser only shows “main artists”. “interesting performers” are available for navigation and as focus criteria items, and “others” are dead links within the app–they have no pages and can’t be focused on, much to the chagrin of people who consider production credits to be as important as composers, conductors, and violinists. This is something that will obviously change as we become more well-rounded in our handling of those use cases.

We went down a similar path with composers–trying to make judgements based on the content in your library about which composers are “interesting” enough to appear in the composer browser, or to have their works called out on album details, or …

At the moment, we have three (internal) levels of interestingness: a composer can be “uninteresting”, “interesting”, or “super-interesting”. The classical guys are always “super-interesting”. Everyone else with composition credits can be any of the three categories. The criteria is based on several factors, but the most dominant is the number of recordings of that person’s work appear in your library that are performed by unaffiliated musicians.

Uninteresting composers basically don’t exist as composers in the system. Interesting composers can be searched and navigated to as composers. Super-interesting composers cause their works to get the “2 PERFORMANCES” treatment on album details wherever they appear, and appear in the composer browser.

The goal with all of this was: get really important guys like Harold Arlen and Richard Rodgers to be treated as well as the classical composers even though we don’t have proper data for them, and get less important guys who are still noteworthy enough to support composer-based navigation to allow it. And at the same time, to manage the noise level so you’re not looking at performance/work structure for Bob Dylan on his own albums just because he recorded three versions of each song. Oh, and we also want to help you navigate cover recordings of Dylan’s works if you have a lot of them, but not if there’s too few to make a reasonably well populated navigation experience. And this should vary based on your collection. Maybe in your collection, Nirvana is just a grunge band, but I happen to be a connoisseur of jazz covers of Nirvana songs, so Kurt Cobain is an important composer to me (really).

Whew.

It was a sloppy set of goals, not all of which arrived at the same time, and we did a lot of iteration to get here and…I’m not happy with it. Both in terms of the end result in-app today, and in the entire line of thinking that we will somehow come up with a model of “interestingness” that any two people can agree upon and understand.

There’s some confusion about why “X performances” doesn’t always show up on the album screen (answer: it only shows up when there is an interesting composer). There is also confusion about why certain tracks end up being deemed works (answer: because allowing them to be works allows some people to navigate based on composer/work structure that they find interesting). There’s also, I think, an annoying level of noise for everyone with large collections because in large collections more composers meet the criteria for “interesting” or “super interesting” than in small collections. There is confusion about why all people with composer credits don’t appear in the composer browser (answer: you’d have 10,000 composers in the list and only 236 of them would have images or interesting data to navigate. Have you seen the work browser? It would be like that…but even worse).

We learned something that shouldn’t have been surprising: no matter how much you tune, someone will always be unhappy with the result. Some people want every navigation path to work, no matter how boring or pedantic, and others consider any mention of composers or works whatsoever to be clutter or noise.

I think our next go at this will be crisper and more predictable. We might separate the idea of works/performances from the idea of cover recordings. Why? The same data model works for both, but navigation patterns that work for work/performance level data fall over in the context of cover recordings.

I haven’t mentioned “alternate versions” at the track/performance level–but that intersects with all of this because it creates another difficult judgement call: how different does something have to be to be a distinct performance of the same piece of music. Does re-mastering count? I would argue no, but there are definitely people out there who will expect to navigate that way, too, just as I want to navigate all of my recordings of “Stella By Starlight” by different musicians in different decades.

Complicated stuff. I’ve been thinking a lot about what “round 2” will look like. I’d really like to get us to a better place. We’ve learned some things from the first iteration.

2 Likes
15

If your data model allows you to easily implement a means of having every navigation path work, but the behaviour as to whether this is enabled or not is user configurable would that not cater for most tastes?

Not sure how feasible but differentiating between composers for Classical vs Popular music may also cut clutter in browsing and have two separate presentation formats . Keep the works component for Classical and where Popular music is concerned show albums and songs composed by the Popular artist. If a user wants to find all instances of a song in their library, who write it etc. they can browse using the song as entity?

Separating Classical composers would also enable presentation of all classical composers to the user incl. their significance classification…they could then tweak as they please and Roon will behave accordingly because it’s part of the data model?

16

If your data model allows you to easily implement a means of having every navigation path work, but the behaviour as to whether this is enabled or not is user configurable would that not cater for most tastes?

I think it would make people who are 100% uninterested in composer/work level data happy, and would do harm to everyone else.

One of the loudest complaints when we turned up the volume on navigation paths was that rock/pop tracks were ending up with work pages and composer credits that were very boring to look at, and too easy to get to.

Just because I like listening to Jazz, and require composer/work navigation to understand it, doesn’t mean I want a high noise level on popular music by shoving composer links and X PERFORMANCES all over the place. This is pure gold, though:

Screen Shot 2015-06-08 at 3.17.46 AM.png754×741 105 KB

Not sure how feasible but differentiating between composers for Classical vs Popular music may also cut clutter in browsing and have two separate presentation formats .

As much as this hurts the part of my brain that likes to keep editorial judgements out of the data modelling process, I think that something like this is probably going to be necessary. I’m not sure I want to judge people as “classical” or “not classical”–I fear that there are too many blurred lines in real life for that, but I think we could probably judge works or songs that way with little collateral damage.

Separating Classical composers would also enable presentation of all classical composers to the user incl. their significance classification…they could then tweak as they please and Roon will behave accordingly because it’s part of the data model?

We’ve tossed around some graphical treatments for dedicated classical browsing modes. It will happen eventually.

If we keep significance classifications, they will eventually be editable. Right now we’re still not sure about them as a long-term approach.

Something always in the back of our mind when considering editing is: once we make something editable, we’re stuck with it forever, because our users will have invested time in building that data. Everything related to editing requires really careful consideration.

1 Like
17

You raise a very, very good point.

Am I correct in surmising that you obtain genre metadata from Rovi at album level? If that’s the case doesn’t it by definition provide the distinction required to enable the presentation layer to treat Classical one way, Jazz another (if not aligned with classical) and everything else the same? This could eliminate the need to consider works or songs that aren’t classified Classical or Jazz?

18

Very interesting. Didn’t know you did that. Deserves further thinking.

19

@brian Picking up on the old discussion of how to handle artists and groups…

I recently acquired “Now This” and the metadata lists this under “Gary Peacock Trio”. I have several albums by Peacock, but the metadata lists them under Peacock himself, not the trio. So this means that when I look at the Now This albums I see nothing else listed by the “same artist”. And if I click on Gary Peacock Trio of course I see only the one album. And there is way under the Trio to see membership. So the only way to navigate on is to go back to the album, and open up Credits, and select Gary Peacock.

The navigation steps are a bit clumsy, but that’s not the issue. The problem is that it hides the relationships and thus impairs discovery. If I know about Gary Peacock I’m ok, but I wouldn’t discover.

So I hope you can implement the idea you discussed last. Some time – I realize you guys have a lot of top priorities.

20

This one is high on the list, because it enables us to solve an important problem. Separating the “album artist” string from the list of primary artists for an album is going to help our metadata to co-exist with metadata from file tags gracefully.

Once the app supports the new model, we’ll need to do a pass on the backend data to make it more honest. I believe that in the new world, the album artist string should be “Gary Peacock Trio” and “Gary Peacock Trio” and “Gary Peacock” should be the linked primary artists, but the backend data seems to have been de-cluttered, and only “Gary Peacock Trio” remains.