Importing content from Meridian/Sooloos Systems

Continuing the discussion from What does Roon offer? I’m coming from the Meridian world:

I wanted to pull this out of the other thread, because it’s an important topic to address.

The Meridian/Sooloos system is one of the best music products on the market today–but since designing and building that product, we’ve gone through three or four tear-it-down-and-do-it-right cycles. At this point, the way Roon approaches metadata is several huge steps ahead of where the Meridian system is today.

A good way to think about the Meridian system is as a standard artist->album->track database with a bunch of “extras” tacked on at the album level, like reviews, credits, genres, and tags. This was a step forward from what other people were doing at the time, but compared to where we are today, now it kind of feels like how iTunes felt to us in back in 2007.

We’ve completely re-thought how music metadata should be represented. Instead of treating it as a three-level artist->album->track hierarchy, we represent it as a web of relationships. Tracks relate to their albums just as before, but both tracks and albums can have artists and credits. A track can also be associated with a performance of a work, which is composed by a composer. Unlike in the Meridian system, which essentially only supported rich data at the album level, we have “extras” at practically every point in the system. Performers have artwork, biographies, birth/death data, countries, periods (think Baroque, Renaissance), and so forth. Works have composers and composition dates, forms and instrumentations. I’ve really only barely scratched the surface here. At this point, Roon is managing approximately ten times as much data for a given music collection than the Meridian system does.

We’ve also made import significantly more automated. Instead of sitting in front of Control:PC or Control:Mac making manual grooming choices during import, the Roon software adopts a less intrusive approach. After you tell Roon where the audio files live, Roon sucks it in and indexes it in the background. Done.

As I’m sure you know, an automatic process isn’t going to get everything right 100% of the time (although, I think you will be pleasantly surprised at how good a job it does right out of the box). To help with this, there are a couple of wizards in the app that walk you through fixing badly structured or mis-identified music. If all else fails, there’s an option that allows you to turn off the automatic behavior and revert content back to the metadata that it came with.

The Meridian system keeps your audio files in an obscured proprietary filesystem layout (something that Roon will not do, btw). Accessing it directly to extract the content is quite complex for an outside piece of software. One of the most irritating issues is that files imported from CDs using the meridian system are untagged, and sometimes not even broken up into individual files for each track! Thankfully, their product provides a lossless export feature that allows you to export the files, tagged with metadata and artwork exactly as they exist in your Meridian system. This is the best way to get media out of the system.

Depending on what mechanisms you used for grooming, there may be some data that isn’t included as part of the export. We are encouraging Meridian to embed as much data as possible via that export mechanism.

Once you’ve got the files out of there, you can import them into Roon with minimal effort. During import, Roon holds onto the metadata from your files verbatim in its databases. This allows us to implement that feature mentioned above where we revert to file-level metadata and ensures that no data is lost as part of our import process, so that you don’t lose access to anything important that might have been embedded in those files.

If we were to simply replicate your existing content verbatim within Roon with every aspect preserved, you’d be missing out on most of the value that Roon offers to music collectors. While we don’t want you to lose access to carefully groomed data, we also know that in many cases, the Roon system is capable of automatically producing metadata far more detailed and accurate than what the Meridian databases are even capable of representing. The best result is going to come from a mixture of the two, and we’re working to make that as simple to achieve as possible.

6 Likes

Brian,

Are names stored as “First name Second name” or as two separate fields? I like to present all my artists and composers by surname first, it makes searching a lot easier and maintains consistancy between classical composers and modern artists.

The one time I struggle is with whether Howlin’ Wolf, for example, should be filed under H or W. In my case I’m happy to leave Mr Wolf under H.

We display names a first-last always, but under the hood we have sort keys for both last-first and first-last sorting.

Choosing which mode to use for each browse situation (composers, artists) is a user preference. Personally, I sort the composer browser last-first and the artist browser first-last.

1 Like

Hi All

Is there going to be an album or track limit in Roon?

Russ

Not a hard limit, no. How much collection you can effectively manage will depend on the hardware spec. How many albums do you have?

Edging over 300k over here.

Edging over 300k over here.

Albums!?

Tracks. Albums are a smidgen over 20,000. Lifetime of collecting and listening.

There’s no technical reason why Roon shouldn’t be able to support a collection of that size, but you’re definitely in outlier territory.

With that much content, you might run into memory issues or instability early on, especially during the initial import, but we’ll get them sorted out as quickly as possible.

Looking forward to it. I’ve got 16GB of ram on the mini-itx it’d run on, so I’m hopeful it’ll manage. My library is filed 0-9, A, B, C … Z so I guess I could always run the initial scan on a subset and append scan thereafter?

25,000 here also and I know of quite a few others with similar. Also running 16GB RAM already.

Wow, 25,000 albums. At 70 min an album, it would take you 3.39 years to listen to all of your music, if you listed 24 hours a day. Almost a decade at 8 hours a day. That is a lot of music. What album or song do you have the most duplicates of? Cannot imagine there are many with larger collections.

It’s never about how long; always about choice and moods.

Thank you for the thoughtful response. I do want to say that I find this talk about Meridian and their obscured proprietary file system layout somewhat disingenuous. You just left the place and now it’s like you never touched Sooloos? :smile: I understand Roon will be better and more open and I look forward to seeing it. Still, it seems to me some Roon devs should be uniquely qualified to extract Credits (e.g.) from Sooloos albums.

We may have just left the company, but that database format was designed in 2008, and some of the other details I alluded to are even older. No-one who takes product and software development as seriously as we do looks back fondly on work they did 7 years and two product generations ago.

3 Likes

Sorry to unearth an old thread, but I just started the process of migrating a Sooloos collection of 7500 albums from a MD600/Store:One setup to a Sonic Transporter AP. After 24 hours of exporting (native format), I have moved 5,314/94,713 tracks.

At this rate, the entire transfer will take somewhere in the ballpark of just under three weeks. According to the Sonic Transporter, I’ve moved roughly 175GB of data. Any idea where the bottleneck in this process is happening?

I’m using Control:PC to export all files from the Sooloos Storage to the Sonic Transporter over a Gigabit switch.

You just have a lot of data to move. I think you can still use Sooloos during the process.
If you point Roon at the folder you are exporting too, you would see the library grow.
I am no expert though and I’m sure others will give more technical advise.

Just thoughts, Chris

I expect it is the write speed of the Transporter. Can you check that ?

I usually saw about 5000 albums per day into a USB 3.0 external HDD.
Are you running Sooloos core on the MD600 or elsewhere?

You may want to try that for speed and also to use as a backup.

Hi Patrick,

My experience is similar to that of Nick’s.

With all associated hardware C15, MD600, QNAP and the PC running Control:PC hardwired to a 1Gb Cisco switch.

The best I achieved was around 15 seconds per album. (circa 5500 albums per day). This was with a MD600 running 7500 RPM drives to a QNAP NAS (again with 7500 RPM drives) and Control:PC running an i7 CPU with 8GB RAM.

Exporting from the MD600 to a USB3 HDD is a little slower circa 20 seconds per album (circa 4500 albums per day).

In contrast an export from C15 (when I used it for storage and core) to a USB 2 drive was more like 45 seconds per album (circa 2000 albums per day).

You might wish to consider performing the export to a USB 3 connected drive first (you should have a backup anyway) and then once complete copy the exported files on to Transporter (using normal operating file sharing).

Also it’s worth double checking / confirming that the computer running Control:PC is also hardwired to the LAN (if it’s on WiFi that will kill the performance).

I also wonder whether the involvement of an ancient Store unit could slow the whole thing down. (Pure speculation.)