Bulk override of Roon metadata

Is it possible to stop Roon overwriting my own carefully constructed metadata? The “Revert” function is much too cumbersome to be used on anything other than the odd album. I have 10,000+ albums and I’m looking for a bulk override facility to allow me easily to revert to my own “Artists”,“Album Titles” and “Track Titles”.

Not today, but a “Prefer Local Metadata” feature is in the pipeline for album titles, track titles, and artwork.

In general, the idea of “Prefer Local Metadata” is very simple and well understood for fields that work the same in Roon and in file tags.

This idea is more complex for fields like artist. The problem is–in Roon, artists are first-class entities with lots of supplementary metadata. The fact that your content is hooked up to this metadata is the source of much of the power that Roon offers.

Files don’t contain links to Roon artist entities, they just contain textual artist names. Replacing your artists in one fell swoop with lame metadata-less “artists” based solely on file tags would remove a lot of useful functionality.

What I think you really want here is a way to get your artists in line with your expectations while keeping as much metadata as possible–this is a more complex task, but I think it’s a worthwhile goal for your collection long term.

Keep in mind–you have a large, complex, heavily groomed collection. I’m sure it took you a lot of time and effort to format your media collection to work well with your current software. Switching to any app that supports a relational metadata mode like Roon’s ls going to take some effort and iteration and time to get right.

I’ve looked through some of your other posts. Some of the things you are missing are there (or almost there) already–for example, you’ve embedded quality information into title strings in situations whereas Roon manages it directly. Maybe the problem is that we’re not displaying this quality information in more places in the UI. Where would you like to see it?

It’s also clear that to meet your needs, we will have to improve conductor-centered browsing. Again, I think this is best approached as a first-class feature. We have conductor data–we just need to figure out how to turn it into a good browsing experience.

I’m a little bit less clear on your situation with boxed sets–it sounds like they may be getting split up into multiple albums within Roon as a result of your filing system. This is definitely going to cause problems. In Roon “album” means “release”, not “disc”. If your releases are getting split up into one album per disk, this will hamper automatic identification and result in confusing behavior. Tracks are grouped into albums solely based on the file tags + directory structure. It’s likely that your custom filing system is confusing Roon a bit. I suspect that this is surmountable.

What does the directory structure look like for box sets in your collection (a screenshot of explorer/finder will probably provide the most insight here)? Maybe we can do a better job of recognizing it and taking it apart.

Thanks for trying out Roon. Complex collections like yours are a project, but something we intend to solve. If you’re willing to bear with us and keep the feedback coming, we have every intention to get you to a place where you can enjoy your collection and Roon together.

Brian

It’s a brilliant product, don’t get me wrong, but I genuinely have
something that is beyond piecemeal amendment within Roon. My album
structure is very straightforward - I simply follow the output that
dbPoweramp produces. If you are interested I will go into it in some detail
offline and send you some examples of the problems I have. I don’t know
whether you ever attempted to bulk import using drag and drop into Sooloos

  • works quite well for pop music but for classical music it produces a mess
    and the only solution is to delete everything and start again on an album
    by album basis. The dbPoweramp plug in overcame this problem as it
    completely bypassed the Sooloos lookup procedure. The Roon problem is
    similar. Let me know if you want me to provide some examples - I still have
    another 3000 classical albums to rip so I have some incentive to get it
    right.

I genuinely have something that is beyond piecemeal amendment within Roon

Absolutely.

My album structure is very straightforward - I simply follow the output that dbPoweramp produces.

If your directory structure is exactly what comes out of dBPowerAmp and we’re still throwing your discs all over the place, it might just be a matter of not recognizing the directory structure a little bit better. @jeremiah should be able to help you dig into this a little bit deeper. We definitely want to help here.

I don’t know whether you ever attempted to bulk import using drag and drop into Sooloos

Thousands and thousands of times :slight_smile:

The dbPoweramp plug in overcame this problem as it completely bypassed the Sooloos lookup procedure.

This is something that Meridian did sometime after the point where they took over the Sooloos products several years ago. You may disagree with some of what I’m about to say, but I want to explain where we’re coming from here, and how our goals and approach differ from theirs.

One of our primary goals is to automate the grooming process so that as many people as possible can experience the joys of a groomed collection. We are all picky groomers ourselves, but we are also software engineers, so it’s in our blood to automate and to generalize. We want solutions that scale well to most users and most content, and then some mechanisms for manual fix-ups when we get it wrong.

Another goal is to represent the metadata directly and precisely. This is why we keep track of performers, composers, works, and performances as first-class entities, and why we don’t like format information in the title fields–by treating this data in a first-class fashion, we can make better use of it. Extra data in title fields feels (to us) like something that users do as a work-around when they weren’t furnished with better alternatives.

Together, these goals give Roon a lot of power. The automatic identification (is supposed to) fiigure out what your content is, and then once the content is linked to global identifiers, we can improve the coverage and richness of metadata over time.

If I had to place Sooloos, Roon, and ID3 tags on a scale how powerful the metadata model is, it would look something like this. I’m not spitballing here–I designed and built the Sooloos 2.x database. Roon’s is an order of magnitude more complex:

|-------------|--------------------------------------------------------------|
ID3        Sooloos                                                          Roon

Sooloos metadata is essentially ID3 tags plus some extra album level data, an improved genre hierarchy, tags, and some very lightweight display hacks for work/part grouping and composers. The product design is great, and to a large extent, makes it seem more complex than it is. Really, though, it’s a pretty tag manager with a few album-level extras.

By supporting first-class works, performances, composers and so forth, we’ve done more for classical listeners than anyone else has yet attempted to, and it is still only somewhat usable. This is a little bit disappointing given the effort involved in getting this far, but It’s a difficult problem to solve, and it’s going to take time. We’re committed to doing it right.

Classical content is difficult for a number of reasons:

  • The data is inherently more complex. Popular content is largely understood as artist->album->track + genre tags. Classical has all of those things plus works, performances, forms, instrumentations, conductors, composers, and so on.
  • Identifying classical content is hard. Track titles, work names, composer names, and catalog systems are poorly canonicalized when compared to popular content, and it’s becoming clear that more bespoke work is going to be required to reliably identify classical content.
  • Acquiring data about which classical releases exist in the world is also hard. Many releases have no readily available data from any of our metadata sources. It’s becoming clear that we will need to generate some of this data ourselves, or find a way to empower our users to help collect it for us from their own content.

While we recognize that Meridian’s solutions to these problems work for some people, we are disappointed in how Meridian elected to approach this problem.

They recommended bypassing identification for difficult content instead of making the identification work better or trying to acquire (or produce and make available) better data. They recommended (and got involved in) extensive manual grooming on a user-by-user basis. Instead of supporting a first-class treatment of composers, works and performances, they stuck with textual splitting hacks based on track titles, and they never made a serious attempt to link up metadata in the Sooloos systems with globally meaningful identifiers that allow for improvement and evolution over time. We have bigger, longer-term ambitions when it comes to this stuff.

In the near term, we need to provide better ways for you to get at the data that’s already in acceptable form in your file tags and directory structure. In the medium term, we need to get our identification engine to the point where it’s reliably identifying the majority of your content. You probably also need some new browsing features to help you cut up the data in all of the ways ways that you need to. It’s not going to happen overnight, but I think we can get there.

1 Like

Thanks for the detailed response. Much appreciated, as are your efforts to sort out how to deal with classical music. I can see the Roon data is an order of magnitude greater than anything that has gone before. Your comments about extra data in title fields are spot on - we only do this because of inadequacies in existing systems but, to use a cliche, “we are where we are” and many of us we are bedded in with sub-optimal systems for very large collections. If there is an easy way to convert these compromises, I am all for it, but I don’t have the stomach to re-edit all my database at this stage. I think you could, however, let us display on the front page some of the information that has been relegated to the back pages, such as album version and sample rate. I have a screenshot to show you what I mean but I don’t know how to include it in the post. Also, how about, on the top page, the option to select albums by right click and then in the edit drop down box the option to revert to user metadata - this would at least avoid having to click through three layers on an album-by-album basis and allow me to re-file the worst misfits. I will of course get in touch with Jeremiah and explain my filing structure and give him some examples of problems. Incidentally, the top page would benefit greatly from an alphabetic index along the bottom to allow one to get quickly to a particular artist.

1 Like

So I just got my MS200, and set up my Roon (flawless setup process – kudos to the Roonies). Then I watched Roon willfully ignore all my carefully groomed meta (I know – by design :wink: ).

Now I am literally unable to easily locate about 1/3 of my collection.

We (obsessive meta groomers) need a 3rd setup mode besides Organize and Watch. Maybe it should be called Basic, or Hands-Off. If Roon would use the existing metadata for Artist, Album Artist, Album, Title, Genre, Composer, Album Art (i.e. folder.jpg or embedded), and Year – without overriding even a single instance on any file – then those of us with large groomed collections could at least get up & running and find our music.

Figuring out what limitations might have to be lived with (or not) in this ‘Basic’ mode can come later, but at least our music would be easily found.

I don’t want to be a buzzkill – this has the potential to be an incredibly powerful, seductive, and rewarding tool. But with much of my collection playing Hide n’Go Seek due to the “Watch” setup, it’s more like a cool toy than a tool.

Spectacular possibilities exist – but right now just feeling utterly frustrated & disappointed…

We’re going to do things to make the system friendlier for people who groom metadata. We’ve designed some solutions, and many problems have yet to be solved. We will do this carefully, because the consequences of getting it wrong are severe.

It’s a really challenging problem because on one hand, your data is very clean, but on the other, it’s not very expressive. Getting your data into Roon too simplistically will limit your options with it long term. And, from where we’re sitting right now, importing it properly without compromising the expressive power of Roon’s data model seems like, at best, a daunting and very manual task.

I want to take an opportunity to explain some of the complexity, to help you understand what we’re up against, and how frustratingly in-expressive metadata tags can be.

Artist
Album Artist
Composer

These should be lists, not items, and should never contain a composite like “Miles Davis / John Coltrane”

Composers should be associated with compositions, not tracks or albums, and compositions spread across multiple tracks should be structured properly.

The other problem with these three is that human names aren’t unambiguous identifiers. There are two guys named “Miles Davis” (one of them is a bassist on a Kanye album, and one is the one you know), two named “Bill Evans” (one is a low-rent fusion saxophonist), two named “Avishai Cohen” (contemporaries in the nyc jazz scene), a half dozen groups called “Air” and so on and so forth. It’s a tricky problem to figure out how to map artist strings (especially composite artist strings) into meaningful artist identifiers that can distinguish people and groups that share a name.

Album
Title

No trouble here–it’s just display text. Whew!

Genre

Genres belong in a hierarchy. The way most file-oriented music software uses genre is closer to what we would call “tags”.

We have a genre hierarchy, but you may not agree with it, or you may find it incomplete, or you may disagree with how genres are assigned to albums or artists automatically.

If your genre system were the same as ours (or could be simply mapped), it would be trivial to import your genre assignments, but it probably isn’t.

Also, it’s likely that in your collection, genre is a track level field, whereas in our world, it’s an album and artist level concept–this creates further opportunities for ambiguity and friction.

Album Art (i.e. folder.jpg or embedded)

No trouble here.

Year

Roon has four separate year fields:

  • Recording Start Date
  • Recording End Date
  • Original Release Date
  • Release Date

When sorting by “date” we prefer original release date, then fall back through release date, recording end date, recording start date depending on what’s available. This ensures that “Kind Of Blue” is in 1957, and not in the year that it was remastered.

There’s a real question of what your “Year” field means–it probably doesn’t mean recording date, but does it mean original release date or release date? What about someone else’s metadata who groomed using different rules?

You’d probably expect to be able to sort by your “year” field, but it only may (or may not) represent an “original” release date.

Spectacular possibilities exist – but right now just feeling utterly frustrated & disappointed…

So here’s the problem. You created great clean data…in a system that’s really, really lacking in expressive power.

I understand that the first impulse is to say “just let me keep what I have”. A lot of people are going to say that, and will probably be frustrated as we carefully move towards that goal. The problem is–if we just did it in the simplest way possible, the product would lose most of its power in the process.

2 Likes

Wrt this thread, how can we best leverage your excellent system for date? In mp3Tag, there is a standard Year field. I also see under new/extended tags a couple of fields labeled origyear and date. In Tag & Rename there is also Year, of course. Looks like something called Release Time is available as an extended tag. Of course, values entered in Year look different in mp3Tag as opposed to T&R, and I’ve never understood that.

Anyway, what does the Year field correspond to in Roon’s 4-variant version of date?

Could one fix mistakes in the 4 variants by creating/utilizing any of these additional tags in mp3Tag, populating them w/correct info, and then enacting a change (not necessarily globally) in Roon?

I love the current 4 variants, but, as always, garbage in…garbage out. Looking for insight on future ability to fix incorrect info…

Anything ambiguous like YEAR goes into “release date” (which, by default is hidden by Roon’s metadata for identified content). You’d only see the “garbage in” if you choose to prefer-local-metadata on that field.

Some tagging schemes have original year, recorded year, etc dates which could populate the more specific places. It’s an item on my TODO list to find all of those and map them.

Right now, when you sort a browser by date, you are really sorting by “original release date” and using the other fields as fallbacks when that isn’t available. We will add a secondary sort method that sorts by actual “release date”, for people who want to preserve pre-existing date sorting (prefer local release date on all albums, then sort be release date).

Could one fix mistakes in the 4 variants by creating/utilizing any of these additional tags in mp3Tag, populating them w/correct info, and then enacting a change (not necessarily globally) in Roon?

Today, we only populate release date from tags. Part of this ongoing metadata project is expanding the set of tags that we support.

However, in the near future, I don’t see why not. Roon already re-scans tags when file modification times change, so it would just pick up the changes, populate the “local metadata” layer with them, and they’d bubble up to the surface.

1 Like

Hi Brian

I’m all set to download and trial Roon, eventually completely migrating from Sooloos with a library of over 15,000 albums, 11,000+ of which are classical. But before the trial I thought to share my ideas about classical music in a server after spending over 6 years grooming Sooloos. Sadly it seems a switch to Roon will lose most of that.

First up the COMPOSER is the most important for me in classical albums, and in the surname, not the complete name. So for albums with works of only one composer the lead entry is e.g. BEETHOVEN: Concertos 3 & 4. [And I note some complication with “Williams” so use V.Williams to distinguish that composer. Similarly with Bach, I only use “Bach” for J.P. but CPE Bach etc for others in that family]

What to put in the Artist field is challenging but I decided for concertos the ARTIST (full name) is more important than the conductor. It is tempting to add conductor and orchestra but that can add extra data which complicates entries and can make them too long to read anyway. With Symphonies and other works the conductor name goes first in my Artists entry (but missing the “Sir” that occurs with some) and I’ve decided to often omit the orchestra name, particularly with those who conduct various orchestras, the full names of which are also often too long anyway. That data could be in the “Credits” field anyway although that area has not been of particular concern for me. The orchestra names and lead artist names are usually clearly visible on the cover graphic anyway.

For solo piano the artist is the easy entry but then identifying the composers in the album name is impossible if works are from various composers. My solution has been to use “Various” as the lead album name to indicate various composers are involved. The remainder of the title might then read e.g. Various: “album title” or Various: name of major work involved. Roon could easily identify the situation where one composer is involved and place the COMPOSER SURNAME at the start of the album title. It could also easily identify when an album contained more than one composer so place “Various” or “Mix” as the leader but, to be consistent could only follow that by the Album name, something unsatisfactory for me but unavoidable.

Bottom line for me is that I agree with the other poster that EDITING IS ESSENTIAL for classical fare and is a great feature of Sooloos. But there are many features of Sooloos not so good with their search function the messiest. And the Meridian hardware is not as reliable as its high price suggests.

I’ve utilised the tag feature extensively, and successfully, with Sooloos and have added multiple tags to many albums. e.g. a Vivaldi CD could be tagged VIVALDI, Baroque and Early, Concerto, Classical. A jazz album might be tagged Jazz, Jazz piano, Mood, Popular. Multiple tags are needed because, as you discussed, there is no one word which can describe an album. Something like this cannot be expected to be auto inserted by Roon apart from the basic Classical, Popular, Jazz etc so again, there must be an ability to edit entries so individuals can massage entries to their particular needs and tastes.

And I also appreciate the ability in Sooloos to edit the cover graphics. In quite a few I have added data to more clearly identify what the album in a set contains. Again, the subject of editing arises.

The above might be interpreted as me being a happy Sooloos owner and user but that is not really the case. I’ve had many hardware issues with it and consider Sooloos an overpriced and clumsy system that Roon has the potential to completely replace. It appears to have already done this with most genre but has a challenging road ahead with classical.

That said Brian, I appreciate your task ahead with release 1.3 and the earlier suggestion of a poll to vote on features to include could be a good idea. We are all different and have different requirements in our server systems so the most important feature is ability to edit and massage it for our own particular needs.

Not sure the above will be helpful for you but …

John

OK, I’ve downloaded and am impressed as I posted elsewhere.

To me, only two things are currently missing:

  1. Editing
    2, Tags.

Tags enable us to organise albums to our individual taste so e.g. I can randomly select tracks from the tag "mood"or “Baroque and Early” or … whatever to suit visitors, oneself etc. If Roon would detect and download tags from Sooloos that would be great but I’m unsure they are transported in export so maybe that is just a dream.

But if editing is invoked with tags an option, I’ll be prepared to go through the 15,500 albums to retag them!!

Bottom line, a brilliant job so far Brian & others.

John

Ah, discovered tags ARE are already in Roon :slight_smile:

1 Like

Hi John,

Improved editing is coming in 1.3 (I sound like a cracked record, I know).

Roon currently has Tags which are stored in the Roon database, as distinct from file Tags.

I don’t know what the position is regarding importing Sooloos tags. I don’t think it can be done. @ncpl might be able to confirm.

There are some current techniques that can reduce the workload associated with retagging. Once you can filter an album view using Focus to a group of albums that you want to have the same tag, then select one (right click or long press) and then use Select All. You can now tag all the Albums at once.

The Focus filters are flexible, but they may not filter groups exactly as you would like in order to tag. If file tags, folders or other identifiers make it possible to group such albums outside Roon, then you can create a separate Watched folder called Work and move (not copy) the group to that temporary location. Then in Roon you can use Album/Focus/Inspector/Storage Location to filter all albums in that location, Select All and tag. Move the albums back to usual Watched folder after tagging.

Remember to take regular backups of the database so that if something goes wrong you can revert.

You will find the User Guide a great help in reducing the Roon learning curve.

Thanks Andy.

I’m allergic to learning new programs but I’m seduced into studying Roon as it is obviously so good.

At this stage I plan to use this PC with its modern ASUS Z87 PRO motherboard as the core with the 6TB HDD inside and hope the Roon activity will not interfere with my otherwise relatively light use of the PC. When finally set up, playback will be via the MS600 and its various analog and digital outlets around the house. Final purchase will be a tablet but I’m hoping to find a suitable one with a much bigger screen than the highly recommended Google Nexus 9.

It will be over a week before the 6TB HDDs arrive for the data transfer (one will be for BU) so I’ll take my time studying that User Guide - wow there is a lot there!!!

John

I’m not a huge Apple fan, BUT, Roon on an iPad is a very good thing. If big is what you need then the big pro tablet would be high on my list.

Again, compared to the C15 price it is a relative bargain…

I agree fully - I’m not an Apple fan despite cutting my PC teeth on the Apple II (and that led my youngest son into an IT career, currently in the US).

But thanks for the suggestion Nick, something worth serious consideration and something I will certainly follow up. Portability is unimportant here as it will never leave the house. It will be a dedicated Roon remote!!

Looks like the switch to Roon, even with an expensive tablet, will be far less than the used C15 I was once considering and of course I will end up with a far superior end product.

To me, Meridian shot themselves in the foot by refusing to communicate with those supporting them (at high cost). Quite a contrast to the nice relationship here.

John

1 Like

Hi Nick

I just spotted this tablet:
https://pt.aliexpress.com/item/Chuwi-Hi12-Windows-10-Tablet-PC-Intel-Atom-Cherry-Trail-Z8300-12-inch-4GB-64GB-1100mAh/32603121578.html?isOrigTitle=true

I duck in case people here take me for an Aunt Sally because it is Chinese, but then iPads are made there are they not?

I contacted them to see if they could guarantee it would handle Roon. That is all it would ever be used for.

Comment?

TIA

John