How do you organize your Filing System?

Roon Software Discussion
42

uhhhh… I’m not sure where to begin here… pointers in memory are the basics of a “graph”, and they are a fundamental item in computing.

But this is all splitting hairs… you are talking about DAG vs Tree, and you are right that music is far more DAG-y than Tree-y – but the reality is that there are cycles too.

43

Yeah, we keep saying DAG but most things I have worked on are just DGs.
But “DGs” is hard to say… “Duggs”?

You know, the language police argue that acronyms are initials that are pronounced as words, otherwise they are just initialisms. But that is not a very obvious distinction. ACL, TCL and IDL are “ackle, tickle and iddle”, like a law firm. But I keep telling my Swedish friends that an SUV is not a “suuv”, and an IPA is not an “eepa”.

A long time ago, IBM had a network technology and it needed an application programming interface and the customers didn’t want to code assembler so they built a high level language version, but it was too complicated so they built an entry edition of it, the EEHLLAPI, “eeh-eeh-holapi”. It failed in the marketplace, I think because of the eight letter acronym.

44

Well, since everything uses pointers in memory everything is the basics of a “graph”. Including all database applications and query languages. You might as well say all programming languages are really machine code because that is how they all end up. You could add that they are all graph-based because they all uses pointers.

Good ol’ Vinberg is talking about a particular kind of graph database structure and that is how the data in the database is stored and linked together. The tools you mentioned to me are not graph-based. They don’t store data as a graph-based database would nor do they access data as a graph based database does. Just because the tools are more like a graph database than a relational database does not validate this statement:

45

No. I did not mention database.
You are the only one talking about the database.
Most modern systems use graphs as their data model, but we choose storage technology based on a number of engineering considerations.
This thread is about how to organize the file system, and I brought up the fact that the file system is ill-suited for organizing music because of its tree-structured data model. None of that has anything to do with Roon’s implementation.

I don’t know why you keep on this digression.
Did you misunderstand my reference to graphs, thinking I said graph database?
Or are you trying to impugn my technical competence?
And what’s with “Good ol’ Vinberg”?

46

Well, I’m not that optimistic that Roon is the stupidest system we’ll use. I think that because it’s possible Roon does go away and we’re left with far stupider system that only care about streaming and the masses, we may very well have to deal with dumber systems down the road and that is at least one reason to apply some reasonable and sensible organization to your files. I do hope I’m wrong though.

3 Likes
47

Yet you said this in this thread:

48

Are we actually arguing about this?

OK, the world consist of lots of things which can be modeled with graphs, but not everything, and some of the edges won’t be directed, and there will be cycles. So there.

1 Like
49

@Speed_Racer , what @AndersVinberg is saying is that filesystems are trees for the most part (he’s forgetting about symlinks), and trees are limited. For example, a tree can’t have links between nephews and nieces, a DAG can.

He’s not talk about databases, he’s talk about how the filesystem (or any “folder structure” has a limited data model and thus sucks at navigating music.

The statement is pretty valid (ignoring the cycle issue I mentioned above). The data is not a tree, it’s a DG. This statement says nothing about “databases” nor does it talk about “data structures”. It talks about data and the relationship it has with other data.

50

Are you suggesting that just because the data in Roon is best described by a “directed graph” data model that Roon’s data is not stored and accessed by a tree-based system? Because at their core, both leveldb and RocksDB are log-structured merge tree-based.

If Roon is not using a “directed graph” engine, we users are not using a directed graph system. If Roon were using Neo4j, then we would be…

51

I’m not sure where you are going with this. How Roon stores data is pretty irrelevant here, because it is not limited to a filesystem.

You keep referring back to trees and graphs… but they are just one type of implementation. Graph data can be conveyed completely by key-value stores.

You are using a “directed graph system” because that’s what Roon conveys to you in the UI every time you follow a credit link. It has nothing to do with our database or data structures being graph orientated.

Are you a developer?

52

Oh, I see. Just because it looks like something means it is something. That’s a logical fallacy…

53

Some interesting discussions by Facebook here and here about the realworld engineering choices and optimizations required as they evolved from a relational to a hybrid relational/graph model.

1 Like
54

That second link is awesome… it’s exactly what I’m talking about. Nowhere in there is a graph database of the likes of Neo4j.

I’ve used hexastore ideas to store graph like relationships on disk in a key-value db (to use as an index into a document store).

But once again, this is all greatly off-topic. @AndersVinberg never brought up databases or even data structures, just that filesystems are too limited to convey anything but a singular rigid tree hierarchy.

1 Like
55

Back to the topic:

I’m a bit obsessional about music files organization and although I do enjoy using ROON to wander into my music, I do have a very rigid system for saving music to my disc:

  • I have three directories: “Mp3”, “Flac” and “To sort”
  • My “To sort” directory is a huge mess but that’s okay, no software (or other human!) uses it.
  • “Mp3” and “Flac” are all verified files and have the same logic:
    – First level, “artist name” as it appears on the disc (I have around 4000 for Mp3s)
    – Second level, “{Year} Album name” as it appears on the disc. (If it’s a remaster, I used the original year though.)
    – Third level, the audio files. No other file (no picture, no txt, etc. - everything is stored in tags.)
    If there are discs (like double LPs or box sets) I usually use subdirectories here with the disc number or name, hence going up to a fourth level.
  • I usually only keep one version of a disc. If there are different masters I will try and listen to them all to choose the one I want (sometimes before actually buying it 8-).)
  • Everything is properly tagged following the same logic (ROON can help me find out some rare issues)
  • If there is a collaboration like “Bags & Trane”, I will store it in a dedicated artist directory, not in John Coltrane. ROON can always help me remember about those discs anyway.
  • One exception to this rule is different names for a band. For example “Arcade Fire” will go into my “The Arcade Fire” directory because that’s the name I first knew for them.
  • I do have different directories for, say, “Neil Young” and “Neil young & Crazy Horse”. It’s not the same spirit anyway!

It is rigid, but it makes sense to me and once you’re used to a system, it’s really quick to go by! Also I share my music to friends through a SFTP system, and it’s convenient for them to find out stuff to listen to.
Anyway, I’ll stop boring you now :crazy_face:
Cheers

56

I have 3 main folders in my root

\CD
\HD
\DSD

I use this in DBPowerAmp

[IFCOMP]Various Artists[album][IFMULTI]\Disc [disc][][track] - [title][][IF!COMP][IFVALUE]album artist,[album artist],Various Artists[][album][IFMULTI]\Disc [disc][][track] - [title][]

it will create a Various artist folder for compilation
\Various Artist\Album\track - title

For other it will create the Album artist folder and album sub-folders and sub-folders for multidisc

Single album
\Album artist\Album\track - Title

For multi disc album
\Artist artist\Album\disc 1\track - title

57

Just adding my two cents here at the risk of being repetitive and following @Dannythered’s valuable insights:

In the database world the are relational databases (SQL) and non-relational databases (NoSQL). In the former, one needs to apply great care in predetermining the structure, since changes in the future impact all pre-recorded data. This structure is good for very defined items, like, for example, describing the elements in the periodic table, which rarely change (or, when a new one is found, fit in the same structure, so far).
The latter uses a key-value table, which is otherwise unstructured, and much better when the “final” future state of the database is not known, or potentially likely to change (read new genres of music).

In my humble opinion, I would argue that for music organization, the best file structure is the latter, with an exhaustive set of metadata, which–with the right UI–would enable the user to choose how he or she wants to organize the data as they see fit: resolution, artist, length, genre, year, etc.

I hope I offended no one.
R

58

That’s a very simple and incomplete model of the database world. In particular, you left out graph databases, like AllegroGraph and Neo4j, which are typically triple stores. And strictly speaking, SQL vs. NoSQL refer to whether they have an interpreter or compiler for the Structured Query Language, not their implementation. But even in the relational and key-value worlds, there are enormous differences. Frequently these are due to either the intended hardware for the DB, or the intended application.

59

Understood and thank you for the additional perspective. Of course you are correct.

My comment was less to provide a disertation on database structures, but more to add perspective to the spirit of original question regarding setting up a strict file structure at the onset, versus an alternative more flexible structure.

60

Sure, and I was trying to think of a less pedantic way of responding. Don’t think I succeeded, though. Didn’t mean to dis you at all.

61

I just use dbPoweramp and rip to a single folder CD Music. dbPoweramp default settings take care of the organization so Roon can easily find the files and add to Roon Library.