Missing Composition

Mike_O_Neill · February 3, 2018, 2:02pm

This example is one of many in these box sets, and if I but knew it may more.

I have 20+ Beethoven Piano Sonata sets.(Obsessive yes…)

Most show piano sonata no 1 as Piano Sonata No.1 in F Minor Op.2/1 ( I used Musichi initially to standardise this order)

These 2 show it as Piano Sonata No.1 Op.2/1 in F Minor, i.e. reversed

The Composition Tag is common in all cases being Piano Sonata No.1 in F Minor Op.2 No.1, set externally by JRiver

The bulk show in the composition browser. I just happen to know there are 2 missing, I cannot see them anywhere in the composition Browser

They are from the sets

Beethoven Complete Piano Sonatas Michel Korstick and ditto Rudolph Buchbinder

I assume if I could see them I could merge them …

I have manually added Movement to the tags which caused the sonata to “split” on the main page (With and without Numbers and Roman Numerals)

Both of these sets are “Unidentified”, I have tried Jaikoz to see if MusicBrainz can see them but no joy

So How do I merge these so that I can see a full set in the composition browser?

The same holds for the remaining 31 tracks on the box set

It would seem that Roon doesn’t have these albums in its Db , so how can I manually fix it , internal to Roon or external

Mike

tripleCrotchet · February 4, 2018, 10:18pm

When this happens to me I usually align the title tags to a pattern that has been identified in another performance, or whatever is on allmusic. Maybe you have already tried that? It’s a lot of work for a box set and my experience has been very hit and miss. Sometimes it is worth the effort and you get an id. Sometimes not. In this case I couldn’t find either of those sets on MusicBrainz, freeDB, allmusic so maybe there is a reason why there would be little point in that.

I have examples where an identified album does not have identified compositions (no matter what I do). But I also have examples of the reverse where an unidentified album has identified compositions. I think that is mostly when I have another identified album with overlapping performances. I cannot really see a pattern but I would also like to know if there is something that can be done to force a composition identification.

tripleCrotchet · February 6, 2018, 2:13am

I cannot comment on the Buchbinder but on the Korstick I have got a different identification than you.

I am working off a much smaller library and I actually got an identification for No. 1. But then I got a lot more unidentified. The pattern is that compositions on an unidentified album are only identified if you have another identified composition on something else. As it happens I am carrying around a few box’s with some of the more famous like “Moonlight”. Out of the 32 Sonatas on the Korstick box this is what was identified:

Nos. 1, 2, 4, 6, 7, 8, 14, 23, 29, 32

That’s 31.25%.

The pattern I see is there would be a zero match if that was the only box I had. Looks like a curve. The bigger your library the better your chances of an Id. I guess roon has more information to draw on in that case.

I am not sure why I got a match on No. 1 when you have so many more performances. I haven’t even got the allmusic format. I’ve got this pattern for the Korstick:

Piano Sonata No.1 in F minor, op.2 no.1 -

I only had one other and I have this pattern for the Cave:

Piano Sonata No. 1 in F Minor, Op. 2 No. 1:

Roon identifies this pattern:

Piano Sonata No. 1 in F minor, Op. 2/1

Mike_O_Neill · February 6, 2018, 7:59am

The big drawback is twofold, it’s not even recognition. With best will in the world Roon can’t know every album.

The inability to edit Composition to force a match

The inability to force the use of file based data for Composition

I can’t even get the Composition to use my matched set in JRiver and in the file

It seems to me where Roon fails to ID it should at least offer an alternative to force a match

Mike

tripleCrotchet · February 6, 2018, 9:48am

@Mike_O_Neill, you read my mind.

I don’t know why it is being driven so much from an album identification either. That is a bottomless pit the metadata publishers will never get round to publishing. But in many cases rare albums with small runs may still have famous, often very famous, artists/composers/compositions. It would be nice to get the benefits of id’ing on those levels as well even when it is unrealistic ever to expect an id on every album.

Maybe there are underlying reasons but the restrictions on the “prefer file” concept feels a little arbitrary as well. You can currently “prefer file” on several screens at album, track even composition level. So I don’t really know why it has stopped there or why for example you can only “prefer file” for album art and artist art but not for an album review or artist bio if you have it. I think that has been asked for several times.

tripleCrotchet · February 6, 2018, 5:26pm

@Mike_O_Neill, I don’t know if you have tried this but it is possible to do an excel export that might help us all get a handle on this.

Try going to the composition screen, doing a selection (I selected all my Classical), go to the 3 dots menu at the top, choose export to excel. This is what I get:

You can now see that there are a lot of gaps in the rovi keys. That seems to decide whether there is a composition identification or not. When I filter for all the rows with empty External Id’s I can now see all the Korstick Beethoven Sonata’s for which I got no identification:

This is my small travel library and I can see that from 1,904 Classical works, 494 were not identified. Approx 26%.

I couldn’t see a way of doing it for non-Classical so I just did the same exercise for my entire Classical + non-Classical library. I find from 5,973 works of all kinds, 4112 were not identified. Approx 69%.

I wasn’t surprised how big that last number is as Composer fields are often blank in Pop/Jazz. There also needs to be some perspective. I suspect many are not interested in composition identification for all genres. I’m certainly not. But I am for Classical and also Jazz Standards, American Songbook, Traditional Folk and Singer-Songwriter Covers, maybe a few others.

This is also a quick and dirty analysis. I haven’t eliminated double counting because of my miss-understanding of the roon DB. But I think it is indicating orders of magnitude and why some of us are having a more negative experience with roon than others. I know @Klaus_Kammerer1 comes from a technical background so all this excel stuff should be a walk in the park. This can help put a number to why some of us are having a different roon experience to others. I would be curious what both your numbers are.

The other obvious thing is a feature for importing these excel’s back. It’s not for everyone, populating the gaps in these spreadsheets, but the frustrating thing is that you have rovi keys I didn’t get. Maybe I have a few you didn’t get.

If there was a way to import these excel’s back into roon then a lot of our issues would be addressed it seems to me with a “rovi exchange”. I couldn’t see an “import excel” on the feature requests.

Klaus_Kammerer1 · February 6, 2018, 6:00pm

never played with the excel export
will try tonight…

Mike_O_Neill · February 6, 2018, 6:34pm

Me neither ,I just dumped it but it doesn’t seem too helpful

I’ll have a more detailed look in the morning

tripleCrotchet · February 6, 2018, 7:12pm

Just to put some meat round this.

I actually have the rovi key for @Mike_O_Neill’s missing Korstick Beethoven. It’s rovi:MC0002426224.

Mike must have all the rovi keys for my 20+ missing Kortick Sonata’s.

This gives us enough keys for a positive Id on every composition in an unidentified box. Very powerful. I would really like that. And actually it is those with smaller libraries that benefit the most. At the moment for example I only have 10 of the Beethoven sonatas identified but with Mike’s help it could be all 32.

The question is:

How can we exchange the key’s?.
How can we get them to roon, so roon could use them?

My preference would be just to import the whole Excel back. There are a lot of empty fields and it would be a lot easier to populate them by a batch process like this. But it could also work via the ROONTAGS mechanism in a current feature request. For example: ROONTAGS=“rovi:MC0002426224”

Klaus_Kammerer1 · February 7, 2018, 2:01pm

I played around a little bit with the excel export. I’m not sure I can make anything useful out of the external ID field apart from checking identified vs. non-identified albums.

I initially thought that I could track identified compositions on unidentified albums, but that didn’t work out. But I made another interesting observation.

I have this album with Schubert Piano Trios… unidentified. WORK and PART tag empty to begin with:

Still Roon shows me a grouping for the composition. TITLE tag being:

So to me this looks as if Roon would parse the TITLE track and identifies the common string before the colon. The name of the composition displayed is equal to that string.

Now I am maintaining WORK and PART tags as with my other versions of D 898. I am using the following tags:

Now what happens in Roon:

You notice three things:

There is now a number next to the composition (17), i.e. the composition has been linked to an existing Roon composition.
The text is not equal to my WORK tag.
The Track is showing my PART tag

So it looks to me I’m giving Roon a trigger to match for existing compositions when maintaining the WORK tag. If it finds a match it will merge.
What I find remarkable that Roon is able to group it even without WORK tags, if the TITLE string allows it. At the same time Roon is not able to match based on that grouping, although it is much closer to the internal composition name than my WORK tag.
If I assume that Roon tries to identify the catalogue number and matches it based on that, I do wonder why it can not do that in the first round.

And even after the matchup, the external ID field stays empty (which seems logical, since the album is not identified)

tripleCrotchet · February 7, 2018, 2:46pm

I notice this as well. One of the biggest problems I have is that if I jump through these hoopes to force a match, I cannot prefer my TITLE or WORK tags. This is important if you have a lot of versions and you want to distinguish different scorings (i.e. Briuckner). I also have a lot of Classical Guitar, most of which is transcriptions,. So I don’t want a Guitar piece called a Piano piece (e.g Albenez) but I cannot do that.

I think the main use of the External Id’s is if we could somehow exchange them and roon could then use them. In many cases roon seems to be able to find a match in one library but not another. If we could exchange these key we could all benefit.

Mike_O_Neill · February 7, 2018, 3:12pm

Hi Klaus

A slight snag …The format of the Title is also not standard, WHAT IS ??

For example the MusiCHI tagger allows for Compoistion = Text To the Left Of that is Colon or Hyphen
in desperation you can Copy Title -> Composition and then manually clean up

Realistically to standardize properly you would have to parse the Title and split out, and then Clean Up the individual elements

Instrument - Piano
Form - Trio
No - 1
Key F Minor
Nickname - “Moonlight Sonata”
Opus No. - Op10/1
Catalog - K331 (Mozart, Liszt Scarlatti)

And then Reconstruct it from those elements in the desired (Configurable (?)) Order

Given that that the title is is no standardized format and is in effect free text that would be a very tall order indeed for the developer (I am a retired Microsft Dev and I can assure you it would be very tricky. Maybe I should dust of Visual Studio and try )

For every recognition of an element you’ll get all sorts of associated anomalies too

When I did my Compositions I leaned very heavily on MusiCHI which coped extremely well , but still an awful lot of manual GRUNT… I only did the Major Composers in my library to start with

I suspect this is why Roon leans so heavily on external data sources to identify and create compositions (I assume) to do this exercise would be a major undertaking

Cheers

Mike

Mike_O_Neill · February 7, 2018, 3:20pm

I suspect how Roon does that matching will be one of their best kept “Trade Secrets”. Certainly Song Kong & Jaikoz generate the MusicBrainz ID which seems to help in tricky cases, but I agree best used when all else fails.

Perhaps more interesting is how their 3rd party source does that extraction in their master composition db , if they have one. Or is simply manual slog to create a db of all Composers all works etc which is what MusiCHI tries to do . If that was the case you would expect better consistency.

I do like MusiCHI , you probably get that idea, its worth 20 Euro to take the slog out of you Work/Part stuff you mention. You get a 28 day demo if you want to try.

Mike

Mike_O_Neill · February 7, 2018, 3:22pm

I agree , being able to make Composition and Movement (Work & Part etc) as a tag to “Prefer File” would help

You could put in the had work of standardization in the knowledge that it would be honoured

Mike

tripleCrotchet · February 7, 2018, 3:37pm

The rovi API is actually on line. The keys seem to follow a standard format: AA9999999999.

http://prod-doc.rovicorp.com/mashery/index.php/Data/music-api/v1.1/composition/description

MC is composition but there are different ones for title etc. So unfortunately there is no “join” in the different Excel views as far as I can see. But I think these keys do give some insight and I will use them to be more systematic about prioritising and grooming gaps. It’s still a hopeless task but I like to understand the scale of it. I am not optimistic roon would be interested in an import feature for the handful of people that would use it.

Shame. If it were possible to autopopulate these Excels from a pool of Excels before re-importing that would increase matching for everyone. But I guess that is getting back to the old CCDB/Acuraterip processes.

Klaus_Kammerer1 · February 7, 2018, 3:54pm

I used that approach on my tags but I got rid of it after a while. I now have 95% of my files tagged with a work, but not all with a part (operas mainly).
So if I select “prefer file” for composition on a global level, I screw up my operas, because Roon does not consider the WORK tag as long as the PART tag is not filled. At the same time, Roon still does some matching. I never got a “clean” sitaution on my works eben when “preferring file”.
Roon intervenes somewhere…

and don’t get me started on Bruckner symphony version information… I just gave up on that in terms of tagging it.

Mike_O_Neill · February 9, 2018, 10:43am

I wasn’t suggesting WE tag with all the elements but as a database design the lower you can separate out those elements the more flexible the “rebuild” to a string = Composition.

Have you looked at MusiCHI tagged, it allows all sorts of text manipulations that are really good for splitting out the Part with minimal effort

There’s a 30 day demo if you are interested, I use it a lot and it has helped clean up some Roon issues

Mike

Mike_O_Neill · February 9, 2018, 10:47am

@klauss_kammerer1
@Tony_Casey

Hi guys , a good debate, I wonder if we should draw it all together as a conclusion from the threa to @support

Klaus_Kammerer1 · February 9, 2018, 12:22pm

@Mike_O_Neill sure - I think most of it they know already, but it couldn’t hurt to collect it.

One other thing that I found out is that for single track compositions it seems to be required to copy the WORK tag into the TITLE tag in order to give Roon the trigger to find a match.

I have several albums where I have the work tags consistent and equal to the Rovi work name and they don’t match in single track cases where WORK tag and TITLE tag are different. (when “prefer file” is set)