Beets to manage library

Tinkering
1

After a few false starts I think I’m up and running and starting to understand Beets. Its quite the rabbit hole. Lots to digest, but seems to reward you handsomely on the other side.

https://beets.readthedocs.io/en/stable/index.html

Like Roon, I wish I had started down this path sooner. Any users out there? Thoughts?

I seem to be obsessing on how exactly to uniformly organize my digital mess. Roon does a great job untangling so I probably could just leave it alone, but the OCD in me won’t allow it. So many digital nuggets locked in the files. Having a good time mining them. Would welcome any feedback on lessons learned form others with 20+ years of digital files ripped and purchased with many different naming conventions. Just getting all the album art named alike seems like a victory.

My goal is to keep all unique versions/releases of an album I have and to truly find exact duplicates and remove them. This is where I seem to be leaning right now in my folder naming convention. Feedback, criticism and ideas welcome.

Up till now I’ve never had a good way to see the MoFi and other High Zoot labels. Just as interesting to see the BMG Club label. CDs for a penny anyone? :grinning:

RoonNow
RoonNow759×563 112 KB

1 Like
2

I’ve been looking at this for a while and am minded to get on top of my collection which has become a digital hoarders mess this last 10 years. Just read the intro docs and am interested. Will give this a go soon and check in on here.

2 Likes
3

I’ve been at it since about 2004. So many different ripping strategies, sources, etc. The entire collection is a mess. Roon is fantastic at managing that. However, if you go to merge some new stuff and there is overlap it is becoming impossible to do without automation. I knew beets was the right tool. I just avoided putting in the time. I would have saved me weeks of my life had I started 3 or 4 years ago. It’s simply amazing the granularity you can accomplish.

1 Like
4

Should perhaps have been clearer. Things were relatively fine when it was just rips, which I’ve been doing a similar amount of time. It’s buying downloads with variable metadata when things started slipping. Since using Roon, 3 years is all, I’ve got worse, just ripping and copying to the NUC and let Roon deal. It’s now a bit all over the place…

1 Like
5

While I don’t use Beets to manage my music collection per se, I do use it primarily for importing downloaded albums.

Beets does the following work for me with a single command (roughly):

  • unpacking the download archives
  • arranging the albums and the artwork into a folder structure
    • e.g.
      • first letter
        • artist
          • album
            • subfolder for artwork
  • add additional metadata (especially genres and contributing artists)
  • add lyrics
  • add additional artwork
  • add ReplayGain tags (for other players besides Roon)
  • automatically follow artists on Muspy
  • adding albums to my Musicbrainz collection
  • adding a checksum for the files
  • etc.

Besides that there is the possibility to:

  • convert files on export (e.g. for mobile devices)
  • find duplicates
  • find missing tracks
  • change file permissions
  • etc.

If you are interested in my configuration (for inspiration), you can find it here:

Meanwhile I am glad that I made the effort to deal with Beets more intensively. It took some time and nerves, but I have this configuration running for half a year now and I am very happy about it.

Additionally I pimp the imported files with Songkong.

Adding additional metadata can in some cases add value and in some cases rather the opposite. This is something everyone has to figure out for themselves.

1 Like
6

Sounds like you have a much better handle on it than I do. It is incredibly powerful. You can configure it to your hearts content. Basically if you can think of it Beets can do it. That said it is turning into a full time job for me. I can really see the benefits once you have it configured the way you want and are only all several albums per month.

BUT to get it where I want it and turn it loose on 9K albums has been a lot more work than I thought. I’m sure I am overthinking it, as I tend to do. I want every version of every CD I’ve ripped to be there and appropriately identified by the folder name. Pretty straightforward with newer artists. Proving to be very “hands on” for older artists. I’m hyper focused on my precious Blue Notes of which there are tens of versions for many. The auto-tagger and various thresholds makes it tough for me to put it into quiet mode and step away. Again I’m probably overthinking it.

I can easily see a day when it is done that adding new albums/artists will be a breeze. Short term pain for long term pain I suppose.

7

I’m not a programmer either and I got my desired one with a lot of time, inspiration from other configurations and of course a lot of trial and error.

As you say, with Beets and the appropriate plugins almost everything is possible, but it requires some patience until everything runs the way you want it to.

It’s also very important to try out a newer Beets configuration on a test music library first, and of course to make regular backups of the actual music library. You never know.

I must say that the folder structure is not quite as important to me. I leave the organizing to Roon or I edit albums with Roon. Of course, I also attach importance to a certain order, but perhaps not quite as intense as you.

As I already mentioned, I use Beets mainly for the initial import of the files and then run Songkong over it again. I find the graphical interface or web interface a bit more comfortable. There are just functions that one program can do or can do better and the other just not.

I had originally tried to implement all the functions I wanted with Beets, so that I really only had to use one software. But at some point I reached my limits and now I use just said configuration with two programs.

Maybe someday I will have more time and muse and will optimize or finish my Beets project. The capabilities of Beets are also growing steadily, so that possibly in the foreseeable future more or simpler options will be available.

You should also consider the Pareto principle in all this tinkering. At some point you reach a point where you can only achieve very little with a lot of time. I am also one of those who often overstep this point, but I have also learned a lot.

Currently I’m very keen on spicing up my music collection with images using Valence Art Director. This project may continue until the end of this year until I am reasonably satisfied.

However, we should not forget with all the tinkering that actually listening to the music is in the foreground (which I also forget from time to time) :wink:.

2 Likes
8

I’m really going back and forth on this. Trying to strike the balance (and failing I fear) between good enough because Roon handles it so well AND I want it to be perfect. Also “perfect” seems to be a moving target every time I realize there is one more things I can do with Beets

I’ve already changed things a bit…but at some point part of my Coltrane went from this…

image
image516×513 11.2 KB

To this:

image
image759×563 78.8 KB

3 Likes
9

That’s quite a deep rabbit hole you’ve chosen sir, :+1::joy::rofl:

Good work.

2 Likes
10

I can understand your problem, but since I don’t use folder-based music management software anymore, I don’t put so much emphasis on such things. After all, that’s what Roon is for.

I also have some cases where it looks similar, but I have to admit that I don’t worry about such things anymore.

Surely there is a solution with Beets, but I think you have to ask the real nerds in the Beets forum.

1 Like
11

I agree browsing/playing by folder is antiquated.

I have my doubts as to Roon being around longer than my digital library already has. Roon is fantastic at telling my I have multiple versions and the disc count for maybe a 2-cd deluxe version and the standard 1 CD type. I’m thinking this is a more long term archival format. So anytime I need to know one version from the other OR what version I actually have I’ll know by looking at this:

Living in Darkness (1981) - FLAC 16-44 [1987 Posh Boy ]

That this Agent Orange Album was originally released in 1981…probably on vinyl, but I have the CD version from 1987 on the Posh Boy label. Which if you really care is the original CD release not any sort of reissue. I do remember buying it back in the day to replace my LP because CDs were so much “better”…and always regret selling all my vinyl in the early 90s. Good when you know the history and it matters to you.

Though I know a lot less about the history of Alana Davis I’m reasonably sure by looking at this one:

Blame It on Me (1997) - FLAC 16-44 HDCD [1997 Elektra Entertainment 62112-2]

that I have the original CD because (1997) matches the [1997 Elektra…] date. (Original release date) [My version date].

Some of this work has been a trip down memory lane, but I do think I nearing the end of any benefit. Space is not becoming an issue as I’m always making copies “just to be safe”.

2 Likes
12

I agree with you, but I hope Roon will be around for a few more years. But who knows that.
Perhaps all of my work and time that I am currently investing in Roon will be obsolete in a relatively short time.

I also agree that your strategy is currently a longer-term one. But maybe your headache will be taken care of in a short time by an Al. Who knows.

Maybe we shouldn’t worry so much about the future and just live in the present?

2 Likes
13

Hello, can you share your config? I like your name formatting.

1 Like
14

Here you go. Not sure if it will make sense. A lot of it is commented out. I’m not much for “house keeping”. There is an inline function I lifted from over at the Beets forum and modified for my own use. It is what figures out the bit rate…and format if I remember correctly. I’m very happy with the results. Now that the bulk of my library is done, it is smooth sailing for the small group of news ones trickling in.

I’m quite shocked there was not more interest or commentary on this thread. I figured it was the perfect rabbit hole for the Roon crowd.

plugins: fetchart embedart convert scrub replaygain lastgenre chroma web inline importadded discogs 

# directory: /data/_TestCombined
# directory: /data/asis
directory: /share/Music/BeetsPostProcessing # *** Remember to change this for HighRez, etc.
# directory: /share/Music/Standard
library: /config/musiclibrary.blb

art_filename: cover
threaded: yes
original_date: no
per_disc_numbering: yes
    
paths:
    default: $albumartist/$album ($original_year) - $format $bitdepth-$album_samplerate $media [$year $label $albumdisambig$catalognum]%aunique{}/%if{$multidisc,$disc-}$track - $title
    # default: $albumartist/$album%aunique{}/$track - $title
    singleton: Non-Album/%asciify{$year - $album ($label) $albumdisambig %aunique{albumdisambig}}/$track - %asciify{$title}
    comp: Various Artists/$album ($year $format $bitdepth-$album_samplerate $media) [$label $albumdisambig $catalognum] %aunique{}/%if{$multidisc,$disc-}$track - $title ($artist)
    albumtype:soundtrack: Soundtracks/$album ($year $format $bitdepth-$album_samplerate $media) [$label $albumdisambig $catalognum] %aunique{}/%if{$multidisc,$disc-}$track - $title ($artist)

# Inline plugin template
item_fields:
  multidisc: 1 if disctotal > 1 else 0
  my_samplerate: round(samplerate / 1000)
  #my_samplerate: str(int(samplerate / 1000))
  is_flac: 1 if format == "FLAC" else 0
album_fields:  
  format: |
       formatList = []
       for item in items:
           formatList.append(item.format)
       return formatList
  av_bitrate: |
       total = 0
       for item in items:
           total += item.bitrate
       return round(total / len(items) / 1000)
  album_bitdepth:  |
       total = 0
       for item in items:
           total += item.bitdepth
       return round(total / len(items))
  album_samplerate:  |
       maxsamplerate = 0
       for item in items:
            if item.samplerate > maxsamplerate:
                 maxsamplerate = item.samplerate
       return round(maxsamplerate / 1000)
       #total = 0
       #for item in items:
       #    total += item.samplerate
       #return round(total / len(items) / 1000)
       #return str(int(total / len(items) / 1000))
  is_1644: |
       bd = 0
       sr = 0
       br = 0
       for item in items:
           bd += item.bitdepth
           sr += item.samplerate
           br += item.bitrate
       bd = round(bd / len(items))
       sr = round(sr / len(items) / 1000)
       br = round(br / len(items) / 1000)
       return 1 if bd == 16 and sr == 44 and br > 320 else 0
  display_year: |
       if original_year != year:
          return str(year) + '(' + str(original_year) + ')'
       else:
          return str(year)
        
import:
    quiet: no #Testing this come back to it!!!
    write: yes #Writes tags to files/tracks
    copy: yes
    move: no
    delete: no # Be careful!!!
    resume: ask
    incremental: no
    # quiet_fallback: skip
    quiet_fallback: asis
    # timid: yes #no is default always ask no matter the percentage match
    log: /config/beet.log
    duplicate_action: keep #“remove” means remove **old** item
    # duplicate_action: remove #“remove” means remove **old** item
    #incremental_skip_later: no
    from_scratch: yes

importadded:
    preserve_mtimes: yes  # After importing files, re-set their mtimes to their original value. Default: no.
    preserve_write_mtimes: yes # After writing files, re-set their mtimes to their original value. Default: no.

#match:
#    #strong_rec_thresh: 0.05      # shold be 95% acordign to docs 0.10 = 90%
#    strong_rec_thresh: 0.15
#    preferred:
#        countries: ['US', 'JP', 'GB|UK']
#        media: ['CD', 'Digital Media|File'] # Use CD fist processing standard rez and Digital Media first for HighRe
#        #media: ['Digital Media|File', 'CD']
#        original_year: no
#    ignored: missing_tracks

match:
    strong_rec_thresh: 0.05
    #medium_rec_thresh: 0.25
    #rec_gap_thresh: 0.25
    #max_rec:
    #    missing_tracks: low
    #    unmatched_tracks: low
    distance_weights:
        source: 2.0
        artist: 3.0
        album: 3.0
        #media: 1.0
        media: 0.5
        # mediums: 1.0
        mediums: 0.0
        year: 1.0
        country: 0.5
        label: 0.5
        catalognum: 0.1
        albumdisambig: 0.1
        album_id: 5.0
        tracks: 2.0
        missing_tracks: 0.9
        unmatched_tracks: 0.1
        track_title: 2.0
        track_artist: 2.0
        track_index: 0.3
        track_length: 2.0
        track_id: 0.3
    preferred:
        countries: ['US', 'XE', 'JP', 'GB|UK']
        media: ['CD', 'SACD', 'Digital Media|File'] # Use CD fist processing standard rez and Digital Media first for HighRe
#        #media: ['Digital Media|File', 'CD']
        original_year: no
#    ignored: missing_tracks

musicbrainz:
     extra_tags: [year, country, media]

# files matching these patterns are deleted from source after import
clutter: ["Thumbs.DB", ".DS_Store", "*.m3u", ".pls", "*.jpg", "*.pdf", "*.log", "*.png"]

lastgenre:
    auto: yes
    source: album
    count: 3

embedart:
    auto: yes

fetchart:
    auto: yes
    
replaygain:
    auto: no

scrub:
    auto: yes

replace:
    '^\.': _
    '[\x00-\x1f]': '' #was underscore originally
    '[<>:"\?\*\|]': '' #was underscore originally
    '[\xE8-\xEB]': e
    '[\xEC-\xEF]': i
    '[\xE2-\xE6]': a
    '[\xF2-\xF6]': o
    '[\xF8]': o
    '\.$': ''
    '\s+$': '' #was underscore originally

#path_sep_replace: _

asciify_paths: yes

web:
    host: 0.0.0.0
    port: 8337

discogs:
    index_tracks: no
3 Likes
15

So here I am as a new user starting with a local library for the very first time.

I have been playing around with messy tags, random directory structures, different filenames and so on. Roon has been set to the default settings. I can not believe how good it is in importing and tagging my music.

There’s just a couple of albums with multiple discs I had to merge and even then… it worked flawlessly. Amazing. My library looks so well organized!

So now I am wondering, should I spend time cleaning up the directories, filenames, tags etc. if Roon does the job so well?

16

Roon is amazing at making sense out of chaos…but as far as I know it only does it within Roon. If you are happy with the results that is all that matters. I don’t think Roon actually can/will re-tag or otherwise clean up your underlying mess.

I have a lot of albums with multiple versions. I want to know the difference at a glance. Beets helps with that.

Note here with better tagging I can quickly see which of My Blue Notes are the RVG Editions.

RoonNow
RoonNow1186×538 144 KB

With a little effort in Beets you can even make all the “similar” artists show up in the same Root Artist Directory. For example Neil Young, NY with Crazy Horse, NY the Monsanto Years or any other NY variant call all show up in single directory “Neil Young”. Again does not really matter for Roon, but if you ever leaver Roon or just want your mess organized Beets is the way to go IMO.

Depending on how messy Beets can also remove unwanted file types when it is processing your library, import better cover art, name all the cover art in the say way, etc. If you can dream it Beets can probably do it.

1 Like
17

Nice!

So I checked, Roon does not edit my files. Which is good. There does not seem to be functionality either which is fine for me personally.

I guess I’ll play around with Beets some day just to give my data some structure. But it doesn’t have any priority.

1 Like
18

Another things Roon is good for is letting you know if your albums are identified. If not they are probably tagged wrong…among other things. So, again if you leave Roon, you will have problems. You might find them in Roon by searching Dead Kennedys but it might not show up that way in another player. Say a DAP you take on vacation.

Beets is pretty good at finding what your album actually is and fixing those issues.

Go to Focus in Roon and find “Identified” then click it to find the inverse. Red means unidentified…not very obvious but par for the course with computer programmers (of which I am one).

image
image1217×609 56.2 KB

400 our of 10,000 at one time is my current situation, pre beets, I had over 2,000.

19

Interesting!

408 out of 411 for me. I guess the missing ones were the albums with multiple discs I had to manually merge.