Music File Management

Roon Software Discussion
41

The product intent is that you do not worry about this stuff at all–because most people on Earth do not worry about this stuff at all. Your friend is doing it right with his sloppy library :slight_smile:

It’s a good long-term bet that software will get smarter about this, not dumber. Roon is a lot more capable of dealing with the messes that humans make than the things that came before, and I’m sure we won’t be the last to take steps in this direction (nor will we stop working on it).

Roon is all about automating metadata cleanup.

If there are particular things we’re not getting right repeatedly, we’d really like to take that feedback and improve. The idea is, you should be able to drop your files in and get a good experience for almost all of it, and perhaps do some small tweaks to fix the rest if it is annoying you.

My music folders are totally unorganized, like your friend’s. They follow 2-3 different inconsistent formats from different periods in my history of collecting music. About 30% of my library is a raw database dump from Sooloos that is totally incomprehensible to humans and mostly void of file tags. Even in this situation, Roon makes decent sense of 90% of it.

The only editing features I use are:

  • “Merge Albums” if for some reason Roon can’t put a multi-disc set together
  • “Merge Artists” (I’ve used this 9 times)
  • Adding artwork if it’s missing

I touch file tags or directory structure perhaps 2-3 times per year. It’s so rare that it is barely a consideration.

I don’t care, because I’ll never move to something that isn’t as good at handling my messy library as Roon is.

Comprehensive editing features, detailed documentation about directory structure, etc, is for the subset of our users people who treat music metadata grooming as a hobby or who cannot tolerate a statistically tiny number of mistakes in their music library. If that’s not your hobby, this stuff is best ignored.

A long time ago @AndersVinberg made a great post about this that I can’t find anymore where he laid out the tradeoff: Roon makes it so you can spend almost 0 time/effort on metadata cleanup and enrichment. In return, you tolerate a small level of imperfection. This is pretty much how all automation of previously human activities goes. Computers are good at different things than humans are, so the results are different. Maybe Roon doesn’t get every box set right–but there’s a 0% chance that I would ever go hunt down biographies for every artist in my library manually.

@ncpl once shared the story of how much labor was involved in bringing his huge library into Sooloos (which required individual confirmation of each Album’s metadata upon import), whereas Roon can do it overnight while he sleeps.

9 Likes
42

@brian
indeed I did. It was about a week of almost constant effort. And I had to do it twice in the years that I had that set up.

43

One thing I’d like to see is some intelligence applied to filling in the gaps for composers/songwriters. My library is admittedly large, and as a result exposes holes in your upstream provider’s metadata. There are many instances of songs with no composer entry in some albums, but the same song performed by the same artist elsewhere in the library has a composer entry. Roon doesn’t consider these the same song, particularly if the performer is not the original composer and performer combo. End result is you don’t always find all performances of a song when Roon groups performances, not all instances have lyrics available, not all instances are surfaced when viewing composers etc.

3 Likes
44

One area that I do think you’re getting it wrong is the drag and drop import behaviour when the user selects to copy files to a selected location. It’s the directory naming convention that I’m talking about here. Given that when we export files from Roon the files end up in a human readable /Artist/Album/Track format, why is this not mirrored with imported files and instead put into a folder named with the import time? Maybe this is of use to Roon itself?

45

We have had a lot of (mostly bad) experience trying to automatically organize people’s files automatically/implicitly.

The problem isn’t what happens when we get it right. It’s that the small part of the time when we get it wrong, the files are as good as lost because they are filed in a nonsensical place.

This is why we removed Roon’s “Organized Folders” feature a while back, and (at the same time) designed the drag/drop stuff to be 100% safe and predictable–so no-one will ever lose track of a piece of imported content.

Roon is very tolerant of moving files around within a folder. If you rename the directories to something that makes more sense to you later, it should do no harm. Others have mentioned using Roon’s export feature to create clean directory layout from content after you’ve confirmed that Roon handles it correctly.

This is a tough problem–song titles are far from unique. We have a flexible model for handling compositions–but in the absence of additional evidence, linking up composer-less tracks based on song title feels dangerous to me.

We will never be a manual metadata curation shop–it’s not our expertise to hire armies of people to clean metadata–and looking at the other companies in the metadata space, no-one is investing in this model, while many of them are automating away those kinds of people with machine learning. There is a good reason for that.

I have the same pain point in my library–but I’m not seeing a clear path to resolving it yet without better sources of external data (which is something we’re working on). I’m curious if you can see data in our system that is there already, but which we are not leveraging to this goal.

46

My experience with Roon is that this statement absolutely depends on the type of music you are listening to. Being in the classical music user minority, I’d be absolutely doomed with Roon if I’d have a messy library. That’s exactly why a lot of people who do not use file tags in the first place want Roon to offer folder view in my opinion… I’m not one of them.

20 % of my album collection of 8000+ albums is unidentified by Roon, i.e. if I wouldn’t have a proper library structure and tags in place, this woul be a mess with Roon.

On top of that, out of the 80% of identified album, Roon adds inconsistent and messy metadata, because lots of classcial metadata, provided by 3rd party is just crap. That’s not Roons fault, for sure, but that’s the experience I as a classical music lover get.

My library was in a good (surely not perfect) shape when I swiched to Roon. Since I have switched to Roon I am dealing with my file tags much more often than before though, in order to get rid of messy metadata that Roon puts into my library…

47

As best as I can tell, there are zero good data sources for classical metadata, and especially for box sets.

I eval’d another possible (classical-specific) metadata source last week, and it was junk. This is pretty much the story every time we look at something new. We’re willing to pay…but not if the data is useless.

We’re never going to hire an army of drones to classify/clean/document this content. The labels don’t publish it in a clean/normalized way. No-one gets it right without a ton of metadata grooming. It’s a big problem.

48

I guess the answer depends on what you term as data in your system that is there already. I have definitely seen instances of a performer / track title combo with composer metadata attached and instances of the same performer / track title combo with no composer metadata attached in other albums. For me these are instances where it would be reasonable to infer that given the existence of one or more instances of performer / track title combo with composer metadata, that same combo without composer metadata could be augmented.

49

That’s an interesting piece of confounding evidence that I hadn’t previously considered. @joel, maybe this is something to take into account when munging performances (at some level of confidence).

50

I absolutely agree

That’s also my experience and the reason why I am putting so much effort into tagging. That’s my personal choice, independent of Roon.

Consistency among publishers is a very big problem, yes. I’m not criticizing you for the lack of proper classical metadata. That’s just the situation as it is.I think you can only “fight” that by allowing more control for the user in these cases. But I guess that contradicts your vision what Roon is supposed to be.

And box sets are a mess not only because of the metadadata but also because of current navigation und display limits within Roon even if the data was manually corrected by the user. When I’m on an artists page in Roon and click on a box-set album the artist is particpating in, Roon brings me to the album page on discs 1… How should I know on which disc out of 50+ the artist is performing? :wink:

51

Yes, this is a tension…

This is fixable. We’ve been trying to work out the UX for this for a long time…the problem is, you want navigation from artist->performance, not artist->album, which is what we currently provide. We’ve been playing with concepts for this, but we haven’t found one that is 100% right yet. It is in progress.

2 Likes
52

To state the obvious: If it could be properly curated, Klaus and people like him could be the answer to this problem. I’m hardly an expert in crowdsourcing, but I do know there are quality control techniques that could apply here. Why not a library to which metadata from well-tagged albums could be submitted by trusted Roon users–then cross-checked with 1 or 2 other trusted users. At that point it might be worth Roon’s time to spend a little bit of curation time resolving minor conflicts if necessary. Utilize Roon’s amazing users to BUILD that database that’s missing. Management then gets to decide whether that’s a proprietary Roon asset or the foundation of some open-source thing.

BTW, I have a small, diverse library–about 2200 albums, classical, jazz, pop/rock, in that order of frequency–and about 250 are unidentified. In fairness, some fraction of that (perhaps a third) are individual tracks that have found their way into my library–demo tracks, test tracks, etc. So my “unidentified” rate appears to be right around 10%.

1 Like
53

I honestly don’t mean this in a snide way–but how is this different from the usual, official, correct practice?

54

@Jim_Austin

I don’t think that quote is from me, but in general I do agree with it.

I think we have discussed crowdsourcing in a different thread and it’s a nice thought. However it is really hard to implement without a very strict guideline on the data structure used and on the data maintenance rules. And even then you need a kind of data gatekeeper… That’s a herculean task for anybody to be done in their sparetime.

When I look at the things that occupied me tagging-wise the last couple of months I think for example about grouping of classical works.
Are the op.9 Nocturnes form Chopin a single work entity? Are they three separate works? Rovi (or Roon by doing its magic) doesn’t seem to have made its mind up about it, since it offers composition information for both variants. What about Beethovens op.18? Maybe that’s six separate string quartets for most of us but in terms of op.# it’s the same problem.
So should you intriduce an additional level of hierarchy here? In the past I have decided on this grouping for my own library on a case-by-case basis. In some cases this is now killing me with Roon in my mix of identfied vs. un-identified albums…
If you want to move to crowdsourcing to need to agree on a consistent way dealing with this among lots of other things.

And then you may come to discuss localization, since we are not talking about songs anymore which are called the same all over the world.
I admit that, even if I speak english, I am having problems getting used to english work names for the pieces of music I know most of my life under a different name. I love my A-Dur instead of A major for the keys… and so on. I think Rovi has all its metadata in english, but if you rip with dbpoweramp for example you come across a very interesting mix of languages for your standard tags…
I for myself have accepted that there never will be a really reliable and consistent source of metadata for classical. That’s why I try to convince the Roon guys to give me more control about the data. The structure Roon offers is very good already, it’s just its magic that’s bugging me sometimes. I have come to accept that what’s good for most of the Roon users is not good for me.
I still raise my voice from time to time but I am also a realist as far as priority and implementation speed is concerned for the Roon team.

1 Like
55

Sorry for mis-quoting. I copied brian’s comment from your post.

I completely recognize this as a nontrivial task. I’ve got no idea whether it’s even possible within Roon’s person-power constraints. But–considering that such a database does not exist, I think it’s worth some thinking about: It should exist, and how else are you going to build it?

op. 9 Nocturnes: independent works. Op. 18 quartets: Independent works. I could defend those claims, but what’s the point? Really one just needs to make a choice. Once. There are, however, some tougher calls like Schumann’s Symphonic (and Posthumous) Etudes, which seem to have no fixed structure at all. They’re played all sorts of ways. In that case, you can’t just make a decision: You have to somehow allow for a single work to have varying structure. Seems challenging, but possible. I don’t think it’s that different from when one track on a CD won’t copy because the disc is damaged: Even without that 3rd movement, it’s still a Beethoven String Quartet.

Cheers

1 Like
56

Hey, I resemble that remark! I recognize I’m an outlier (and it seems like a lot of classical fans are as well). All I really want is a way to fix (and then display) errant/missing metadata and good documentation of same. Having to use a third-party metadata editor isn’t an issue for me.

(And it’s not so much a hobby as it is a personality disorder.)

4 Likes
57

You’re too hard on yourself. :joy:

58

Yes, this is so right. I want crowdsourcing to be a solution, but when I start to think about how this would be structured and QC’ed, it seems nearly impossible, especially when you take into account that you’d be dealing with a bunch of OCD types, each of whom would have a different but strongly held idea about the correct way to tag. Language preferences and style rules are only the beginning, as you’ve pointed out.

1 Like
59

Crowdsourced database allready exists for many years www.discogs.com . Works because of a very large user base.

60

Yes, and it’s incomplete and definitely not up to my style standards. I use it, but only in an ancillary way.

If you really wanted a good crowdsourced music-metadata database, you’d need a bunch of fairly skilled volunteers, QC’ed by means of a fairly elaborate hierarchical structure, not unlike Wikipedia.