Remove Duplicates?

It is inevitable - duplicate songs are created - and with a large collection it is difficult to find and remove them. I’m talking about permanently deleting them from the drive, relieving the drive of some space for storage of future music (not just “hiding” them from Roon).
When a duplicate song is deleted, you want to delete the lower quality file, not just “first come first served.”
I’ve been using Jaikoz and SongKong - two excellent programs - but I’m curious what others here are doing (in case there might be an even better solution).

JRiver. Don’t know if it is ‘better’ in any way because I haven’t tried the others. But I use JRiver to manage the library and this is the sort of thing it was born to do.

My Synology has a duplicate file reporting option…but its so cumbersome to go thru and remove and with disk space pretty cheap I just cant be bothered :grimacing:

Actually use roon for this. Roon has already “clumped” the tracks into albums, so I simply choose which version of the album I keep. Usually they all stay since they are from different masters and sound different or have a bonus track. Disk space is cheap as wizardofoz says.

I’ve recently gone through this process.
I merged my FLAC library with my iTunes/AAC library, and ended up with thousands of duplicates. Roon can show you which albums are duplicated, and then you can just delete the lower quality version.
Doing this in Roon, I’ve managed to ‘reduce’ my library from 10k albums to just over 5k.
Yes, HDD space is cheap, but the lower album/track count/total is probably a bit easier for the Nucleus to manage in its library.

4 Likes

Hi Martin, I know this is an old post but I have a question for you.

I am just about to install Roon and wondered if it is worth trying yo ‘cleanse’ my music libraries prior to allowing it to search for my music? I am just nervous that I am gong to end up with a massive headache if I don’t do this first. You mention in your post that Roon can show you which albums are duplicated. When Roon does this is it easy to delete duplicates (lower Res versions) directly through roon or do I still need third party software to find the files and delete them ?

Luke

It’s reasonably easy to do that in Roon. For example you would go to the Album page, select Focus, Inspector and click Duplicates and you can see all duplicate albums. Then you would select Focus, Format and click MP3. This would highlight all your MP3 albums that are duplicates of another resolution. You would then select all these albums, Edit>Delete.
Roon isn’t the best program for curating a library, but it has pretty decent capabilities.

4 Likes

Hi Luke,
I think it’s always a good idea to have a ‘clear out’ before you build your Roon library/database, so if you know which duplicates you have before you install Roon, then I think it’s always a good idea to delete them before you start with Roon.
But as Scott says, it’s also easy to identify them via Roon once the library is built.

1 Like

Thanks for the advice mate. What is the best program for cleansing (finding duplicates and separating lower quality files for deletion) ?

Any advice would be appreciated :crossed_fingers:t2:

Thanks for the info mate - I shall give it a go :+1:t3:

Hi Luke, What was the program you ended up going with to remove Duplicates?

Cheers

Paul

Hello,

May I ask you how to show and delete the duplicates?
Thank you in advance

Keep in mind, I believe when you delete tracks from within roon, it does not delete the folder they were in.

Creating a new mess in your audio storage folders.

You could search for empty folders and delete them, but…

They are not always empty, they usually contain artwork, PDFs, playlist files (.m3u), .txt files… basically anything that was there that isn’t the audio file. :face_with_raised_eyebrow:

FYI. the best (and only?) serious software (windows) that analyzes the acoustic fingerprint and quality of all you files and let you keep just the best ones is Similarity (similarityapp.com). It is very fast, responsive and great value. Like many I spent years testing dozens of crappy apps that didn’t deliver, until I found this. Hope this is useful for the roonies… If you’ve found a better one, please share.

I’ve been using Jaikoz and Songkong for many years, and the programs are constantly being updated. No complaints with either of them - and I highly recommend them both. I use Songkong on my music files (all lossless, and nearly all flac) before copying them to my Roon server.

Yes, I also use SongKong, but they are very good identifiers/taggers, not meant to get rid of duplicates. If you try to deduplicate by name, how do you confirm which is of better quality? Can you delete thousands on a single click?

Yes, with Song Kông you can delete thousands of low-quality tracks with a single click. Jaikoz gives even more user control, but I rarely need to use it.


You are correct. I should have ommited the last statement. What I meant, and I have mentioned in different posts before, is that it is not safe to take decisions based on the “container”. For example, if I purposedly convert a bad 64bit mp3 song that got somewhere to high quality flac, I will be able to cheat on Roon and change the playing color. If I have a better instance of the same song, lets say 256 mp3 and convert it to 320 mp3, Roon will identify it as a lower quality piece. If you hear both, you will immediately note which is which, and they don’t correspond to the container. As far as I understand from jthink, their fingerprint will be very similar if not the same (do not recall that they analyze true quality - perhaps I am missing something?). In that position, when you press the button, the 320mp3 will be immediately deleted and the FLAC will be preserved. At the end you will end up with the “best” containers in your collection, but maybe the worst quality songs inside.
I don’t want to start another long discussed thread here, jost mention that quality is much more than bits and encoders. Ideally we should sit down, rate and filter which versions we prefer by ear - and am sure most of us do - but when we talk about hundreds of thousands of songs we are in another arena. Pick up a prolific artist with dozens of albums and many hit compilations. It is evident that many old versions of the same song, came out of the studio sounding really bad, or that might had a problem with the vinyl press. Anyway, we have cautiously ripped them to the highest digital quality (lets continue with FLAC) and are in our collection. Add to them newer Remastered versions which also converted to FLAC. At the end we have many versions of the exact same song, in several shiny and perfect FLAC files that undoubtely have different sound characteristics (quality) inside. As there is not enough life to do it by ear, we have to rely on newer AI enabled sofwares we can trust. I doubt jthink offers that detail of discrimination, as it is not advertised and is not the main purpose. Maybe Paul Taylor can add here with some luck.
I talked about this in a previous post that Roon sould add this kind of analysis before lighting any of the color codes that right now, only indicate the song container, not the REAL quality of the file. In the meantime most are happy watching a green to purplish light while playing our favorites, believing that we are listening to the puriest version in the stadium… Going back to the topic, just wanted to recommend a nice, solid and intelligent software I use to help getting rid of the bad duplicates; I would surely have saved thousands of hours if someone kindly did this recommendation some years ago. Cheers…

1 Like

Hi, this is rather unlikely scenario, but the Preferred Deletion Criteria is orderable so could put bitrate higher than audio format in deciding which to delete. Delete Duplicates can move instead of deleting duplicates, if you want to be more cautious you could also run in preview mode in order to detect the duplicates and then manually check the results and delete the your preferred option