** Running Roon Core on iMac 27 with 3Ghz i7, 32GB Ram, 1TB Fusion Drive All media referenced in this post has been consolidated and scanned from single external USB drive, in a deep/messy folder structure (see below). **
My apologies in advance for the length and number of questions/issues covered in this one post. Bottom line, we need more detailed documentation and work-flow recommendations for using the “Export…” function especially for large libraries.
I have been intensely grooming a 100,000+ track lossless (ALAC/FLAC/WAV) library for the past 45 days. The source of these files include previous music server exports, downloads, CD Rips, etc. The library “on disk” was a mess and full of duplicates, partially tagged, multiple formats, lots of “various artists albums”. You name it.
I have SUCCESSFULLY invested 200+ hours of tedious work in cleaning up the library within Roon and intend to perform a full library export to new external disk, with the specific goal of outputting a cleaned up version of my library / files on disk. ( I could write a book on this adventure and now consider myself a POWER ROON USER as I believe i have exercised EVERY single feature, numerous repeated 1000’s of times)
Now the real question(s);**
I have read every article/post/thread i can find on “Exporting”, however there does NOT appear to be any detailed documentation or descriptions of exactly how export works or a good work flow to use when preparing for a “whole library export” and/or re-import If I have missed an existing reference, please point me in that direction, otherwise, here are some of my more specific questions about using “Export”, that I have not found good references on;
What is the EXACT behaviour of “Clean up duplicates”? Does it only prevent export of EXACT duplicates from the same album ID?. This is not discussed anywhere I can find.
I have read that performing very large exports (I will be doing about >80,000 track in total) can be problematic even on a high-performance rig such as mine. Is there a difference in performing multiple smaller exports vs. exporting entire library? (e.g. Albums view / select all). I would assume that certain duplicate detection logic, etc might be affected if not operating on full data-set. I was thinking about doing export batches, for example; “Album View/sort by title”, Select all albums beginning with letter “A”, perform export, then select all albums beginning with letter “B”, perform, export, etc. etc.
Any tips here ?
Can you detail how export works with respect to overwriting existing files (if same content is exported) vs. when files might be duplicated on disk?
When might exported files cause files on target export disc be deleted.?
I noted in a quick test that a various artists album that had been un-intentionally split into two albums exported into two separate folders (one for each portion of the album). After I merged the albums in Roon library so that I had one album and re-exported, I ended up with a single album/folder (a good result), but begs some questions about what happens under what circumstance, especially in what cases, might files be moved, deleted, duplicated and/or orphaned.
Lastly and very importantly, any guidance, best practices, etc on how I should plan to be able to use my fully exported library files (to new volume) as my new master library. e.g. rescan my export.
I have read numerous posts that seem to indicated mixed results when doing this.
I have made many 1,000’s of “edits”, including a lot of “Fix Track Order”, etc, album merging, multi-disc fixing, album art additions, etc. Lots of work to get albums AMG identified and full meta-data.
When I complete my grooming I will have a working library that is VERY clean/perfect within Roon but will be a MAJOR mess on disk. I am hopeful that I can be successful at the whole “export” “re-import” process without having to go back through all of the grooming i have done.
Can you describe what data is preserved when simply re-scanning files sets that have been moved (via export). Does Roon write AID’s or TID’s into exported files to help with recognizing files. What is preserved in Roon DB / Cache that will help with library reconstitution, etc.
Thanks in advance for any insight you can provide here.
No help with the export I’m afraid, just a curious reader. Are you exporting and bringing back in to rearrange the files on disk? Is there a particular reason for that. Just fascinated by the reason.
My primary reason is to get my Master Music library reasonably organized on disk even if this is not necessary for Roon operation, but for being able to quickly access files from or utilize them more readily in other applications, etc. It has been long overdue and to also get rid of all of the duplicates and split albums that I currently have. Sort of same reason that most libraries have the books organized on shelves by author or genre, even though a good “card index” would allow you to find any book regardless of how any physical organization might be. Yes, it is a bit of a “vanity” justification. You would probably agree if you saw my overall directory structure. I would have benefited greatly from the now deprecated “organize my files” feature that Roon took away because some users used it inadvertently despite plenty of “warning dialogs”.
OK, So I have completed my full export of 63,732 tracks that represents my completely groomed library. I was succesful in doing this in a single “select all” export that ran for 34 hours.
I learned a key lesson about using the “Album Version” tag, to force Roon to export albums with “multiple versions”. By default it was only exporting the “Primary Version” of multi-version albums, however if you edit album and input a custom “Version” label or if the album had it from tags, you will get a discrete exported folder of each version. This was important for me to make sure I was exporting 100% of my albums.
so the big question remains;
If I “disable” all of my watched folders and create a new watched folder to the root of my fresh full export, will I end up in 100% the same place I am now after Roon rescans the file set ? I would be leaving the current/final database in place for this.
P.S. I did confirm that my export job track count exactly matched the Roon/Summary track count e.g. ALL TRACKS exported.
There is no way that this question can be answered. It will depend upon the settings you use to make the import. There is a lot that is going on with ID, organization, metadata, etc. in the Roon process. I’m not convinced that ALL of that info and organization is captured by the export process.
I predict there will still be nits that will have to be fixed after the import. There are just too many moving parts.
But best of luck.
James_I
(The truth is out there but not necessarily here)
6
I use export primarily to make temporary sets of music for mobile use for example.
Exporting doesnt export your artwork other than to embed the cover only as a weirdly named file within the music file itself. Exporting will completely ignore any folder convention you may have and strictly works on an artist by artist basis. And as james mentions, tags will be lost or need to be reassigned to the exported files i think too.
I can understand your wish to export to a “clean version” for Roon to refer to, but be careful not to throw the baby out with the bathwater so to speak
1 Like
James_I
(The truth is out there but not necessarily here)
8
I think it is an interesting experiment to see what happens and how well it works. And there’s certainly nothing wrong with getting another copy of your library since storage isn’t a huge cost factor these days.
But I am curious about what is achieved if the goal is to have the library within Roon look just like it did before the export/reimport. Sort of a test to see if Roon can preserve everything in its library through an export and reimport process, but I’m not sure what improvement this brings to the experience of using Roon.
But it might mean you can take the library to some new location, and reconstitute it without a backup restore process?
I did a small export test. As I recall, it exported my Roon edits. It did not export all the credits (mostly production variety). And I did not see any of Roon’s tags.
But, it is easy to replicate.
James_I
(The truth is out there but not necessarily here)
10
This is one of my principal concerns with Roon. The design (or, at least lack of current tag functionality on import and export) makes Roon its own silo of information rather than being more easily inter-operable with other music players.
While the structure of a library probably has to be re-built in any music library management software (if you want custom patterns of browsing), at least all the metatags that relate to the grooming of specific artists, albums, and tracks is by and large portable from one piece of software to another, if done right within the file embedded metatags. But not so with Roon.
Since Roon doesn’t provide any way for the custom embedded metatags to be imported or recognized within Roon, and Roon doesn’t export out its tags, tagging (of custom metatags) done outside of Roon has no impact on Roon, and tagging done inside of Roon doesn’t carry over. That makes it necessary to duplicate work. Wish they’d fix that and it is a pretty easy fix.
Well, some of the silo is due to their agreements with other content providers. But I doubt that tags are proprietary. Maybe the tag structure; dunno.
I can at least understand why Roon isn’t jumping to make it easy to leave. But, I feel your pain.
It’s kinda like the Hotel California: you can check in anytime, but you can never leave.
James_I
(The truth is out there but not necessarily here)
12
Understood - while it would be really cool for lyrics, reviews, etc., to go with the export, that is not my expectation. Just the tags we apply - i.e. i want my own work product to be portable, in and out.
At least with approaching 100k subscribers, Roon stability/staying power isn’t an immediate concern. But still, inter-operability ought to be part of the Roon design ethos.
And if we are stuck, then at least I want my pink Champagne on ice!
Well it’s three years later, I am still faced with this same issue and an external 5TB HD that is 3 YEARS OLDER. Yes I have ROON Backups and all of the media in other locations. I’ve also just successfully migrated Roon Core from an iMAC to a dedicated headless PC running Roon Server by simply unplugging the USB HD and moving it over to the new unit and updated the “Storage” paths and everything came back with the RESTORE. (nice)
NOW…BACK TO THE ISSUE.
I need to clean up this HD before it dies and to migrate only the USED media to new location. If any of you could see the folder structure (using 10 storage paths to isolate content) and all sorts of various folder structures NOT TO MENTION LOTS OF ORPHANED FILES THAT I CANNOT EVEN IDENTIFY. That drive was originally used as the scratch pad drive to consolidate all of my original media from MANY sources/locations and dealing with file/folder based grooming/de-duplication and organizing by FORMAT/Resolution, etc…and was the original IMPORT to ROON before I knew what I know now (after extensive Roon editing/grooming)
IDEAL GOAL: Use the Roon Export (ALL TRACKS) function to get all of my USED media consolidated and organized in Artist/Album/Track folder structure (this is huge goal). And THEN USE EXISTING LIBRARY TO RE-AQUIRE the association to the Exported Files, preserving everything in my library. The Export ALL Works very Well in terms of what I get on disk
Simply Put: Export ALL Track to new HD, then USE Those files with current library.
OPTION 2: Use a third party app such as “Tagscanner” to MOVE and REORG all the Files into the Artist/Album/Track type folder structure (this will not provide as clean a folder structure since Tagscanner, etc does NOT have the benefit of the detailed library data to create proper folder/file strucre. e.g. WAV Files, untagged or poorly mp3 tagged files will still result in some messyness) and then PRAY that ROON can track the relocated files.
Early in Roon Days, they had a “ORGANIZE MY FILES” Feature that is EXACTLY WHAT I NEED NOW, but it was deprecated due to misuse and probably blowing up too many peoples setup. (private message me if there is some hidden runtime flag to re-enable this feature please)
Would love any input on this subject. I was quite impressed with my first migration/restore from backup mentioned above and that is making curious as to how I can accomplish my goal using some combination of Export/Backup/Restore and Roons file tracking system. Granted, I literally just unplugged same HD from iMac and used it on PC (so all files reminded in same relative locations).
Wel, My original HD is another year older, but I took it offline and am treating it as my emergency backup. I never found I way that I was confident in to perform the export all/re-aquire move. e.g. I didn’t try, as I received no feedback or input on if/how this might turn out. So I’ve been running fine with the scenario I described, but my latest additions to the newer HD are currently at risk. I think about it every time I am am in my equipment room, or when I experience other HD/NAS drive failures. Let me know if you figure out anything that might be useful. I will probably try to clone my newer drive just to get a snapshot backup. Of course my ROOM library is well backed up.
