I think you’re mixing table size with DB block size. In enterprise applications, e.g., Oracle ERP, a ZFS pool may be created for the database, and this pool’s record size would match the block size. Nonetheless, I doubt such tuning would provide any performance improvement in a domestic setting.
Since Roon uses LevelDB this is entirely moot.
Due to this line, I think perhaps they are separate, but can be aligned.
“The ‘block size’ 4KB = io fetch size = page size in most databases”
Can you elaborate on why using leveldb makes this point moot?
LevelDB is a noSQL database., i.e., data are not stored in relational tables. Roon uses key values and this is stored on disk as a sorted list of arbitrary key / value pairs. These files are immutable.
Whilst LevelDB has a block size, I understand this to be a threshold before writing to disk.
Staff Software Engineer for a major internet company, with 30+ years of experience, among other things. And I still would not second-guess e.g. Roon’s implementation of signal processing, it’s not my area of expertise. Databases and performance optimizations though, I deal with those every day.Does that work for you?
Lots of support here is done by the members. Roon support will jump in, if there is something for them to do, but the question, as asked, makes no sense and has no answer.
You could have asked “are there any parameters in ZFS that could be tuned to improve Roon performance?” The answer would still be “not really” but that’s not what you asked.
Sorry, but you are confusing even “table size” (which is dynamic, and different for every table), page size (which is generally fixed, e.g. 8K in SQL Server), and IO block size, which might vary depending on what the database engine is doing. Within the context of this discussion, it does not seem that you understand what you are even trying to ask.
It does not even have tables per se. It’s a key-value store.
The question you ask, and the optimization you seek, isn’t relevant to an in memory database like LevelDB.
There are two reasons for this:
ZFS is concerned with optimizing reads and writes to disk; which is why it will want to match a database’s optimization to a read write to disk. (That is, if a DB is read write’ing in specific page sizes you don’t want the file system splitting those across say heads of a HDD)
levelDB is an in memory database. It’s optimized to read write from memory which has its own optimizations unrelated to anything ZFS is trying to do with a HDD or SSD.
Now, another problem, when levelDB runs out of memory to store the tree it obviously has to page out some of that tree to page in the branches it’s traversing. How do you optimize this? Make it look like swap space. But, honestly, I’m not so sure you’ll have a ton of detail here because leveldb, from my limited history, was never meant to grown considerably larger than the memory footprint of the most commonly traversed branches.
Staff Software Engineer for a major internet company, with 30+ years of experience, among other things. And I still would not second-guess e.g. Roon’s implementation of signal processing, it’s not my area of expertise. Databases and performance optimizations though, I deal with those every day.Does that work for you?
Great thank you - I ask this because it’s impossible to know who is who here given anyone can chime in. I had a look and I can see it actually says that ‘our technical support team will assist’ you. That’s what made me think it was official Roon support. I see after that it also says ‘and community’ which I did not see. But given the use of an actual form - similar to an incident / request system it was an easy assumption.
Also it sounds like we have similar backgrounds in experience. 30+ years in various technical roles for me, currently a solution architect for a large government org. So I do know something about technology, what I don’t know a lot about is databases.
You could have asked “are there any parameters in ZFS that could be tuned to improve Roon performance?” The answer would still be “not really” but that’s not what you asked.
Yes I could have but I didn’t want to. This was meant to be a simple question. I was expecting either an answer with a table, block size or whatever or an explanation that leveldb doesn’t use it or something similar. What I was not expecting was to turn this query into a mishmash of opinions - given that I thought this was official support.
Sorry, but you are confusing even “table size” (which is dynamic, and different for every table), page size (which is generally fixed, e.g. 8K in SQL Server), and IO block size, which might vary depending on what the database engine is doing. Within the context of this discussion, it does not seem that you understand what you are even trying to ask.
I believe I have said above more than once I could have this wrong. But I definitely know what I am trying to ask - perhaps I have not quite the right terminology, but there have been a lot of unhelpful opinion in the responses which I was not expecting. For example calling me an armchair engineer which ironically is the perfect description for a typical architect - I can see the funny side of that. Except I’ve come from the ground up in the implementation space among other things.
So finally I think we’re getting around to answering the question. There are a lot of database types obviously, it is clear I will need to understand how this one works in order to understand why or why not my question is misaligned and then perhaps I can come back with another question.
Thankyou for making a reasonable attempt to answer to my question. And apologies if any of this frustration was taken out on you (or anyone else) it was not my intention. But you know how it is with forums - people like to poke their noses in and talk about things they don’t know about sometimes. It seemed to be some of these responses were like that and were not giving me anything meaty enough to actually do anything with except to fall back that I know that performance issues can exist from this misalignment.
Cheers.
Thanks for the info. I have been unsure about the in memory aspect. I was hoping someone would tell me and it seems that you have. Though as you know a good engineer never 100% believes until is proven for themselves (work for me to do later if I want).
One of my performance issues I have is extremely slow startup times (of the server side). After what you said, I am wondering it it is something to do with loading the 3.6GB or so of data into RAM. I wonder if it is truly all loaded (including the image cache) or if it is indeed partially loaded and on demand because the images don’t seem to behave like they’re pre-cached on the client. In my case the database and server portion sit on a fairly under-utilised 6 way mirror, with the data on a zraid hdd pool.
He didn’t, it was created in Support and Roon staff moved it out. Presumably as they view this outside of support scope.
Oh I hadn’t even noticed that. How nice of them. A whole lot of confusion could have been saved if I had seen that. Probably it emailed me and told me but just too many emails. And thanks for pointing it out as I was imagining yet another uphill battle.
I completely fouled up my ZFS deployment through lack of reading the manual and a misunderstanding of how its cache works. My I/O wait times on disk are silly high. This was many years ago and I’ve never gone back to fix it. I run ROCK in a VM with the block storage managed by the hypervisor. My music files live on a USB drive passthrough directly to ROCK.
I’ve never experienced slow start-up or what I’d consider slow performance. Roon gets 8GB of RAM. It manages just over 19,000 track across almost 1400 albums.
With Roon, if you look over the forums and spend enough time here, you’ll find that KISS method is where you find stability. It’d be easier / safer to move my music files to a NAS or into the ZFS block store but I don’t… Because Roon likes to have direct access to the device for all kinds of reasons and you don’t need speed to play tiny audio files. In memory key stores are pretty simple; they just need memory to load the tree and they don’t want to compete for memory from other things. That’s why I run ROCK.
I agree with this and am also guilty of it. My wife is pretty good at reminding though that… Maybe it doesn’t work because you evaluated it to death? 
That’s when I go back to KISS, put the thing in the corner, and move on. 
It’s working, I know it’s flawed, but it’s working so I’m just going to ignore it’s flaws and keep using it. Good luck. Maybe if I get around to fixing my ZFS I’ll reach out… you certainly seem to understand that aspect of it better than I do / did.
I completely fouled up my ZFS deployment through lack of reading the manual and a misunderstanding of how its cache works. My I/O wait times on disk are silly high. This was many years ago and I’ve never gone back to fix it. I run ROCK in a VM with the block storage managed by the hypervisor. My music files live on a USB drive passthrough directly to ROCK.
This is a great example of something where it’s not something I would personally do given my current known situation, but everyones situation is different and it may be exactly the right thing for you - as such I’m not going to call it out like some people might do. I hate that.
I’ve never experienced slow start-up or what I’d consider slow performance. Roon gets 8GB of RAM. It manages just over 19,000 track across almost 1400 albums.
I do seem to have a library quite a lot larger than this, having spent hours importing CD’s in lossless format from my teenage years - but equally it’s a dual xeon with 96GB RAM, so it’s scaled up a bit, though not exactly the latest tech any more. I have a suspicion I have a larger database than many and that may be causing the issues. It’d be interesting to see how big your roonserver folder is - mine’s about 3.6GB (newly restored - so should be very little free space inside - as opposed to something that’s been running a while, there may be a bit of extra space that could be compacted if so).
With Roon, if you look over the forums and spend enough time here, you’ll find that KISS method is where you find stability. It’d be easier / safer to move my music files to a NAS or into the ZFS block store but I don’t… Because Roon likes to have direct access to the device for all kinds of reasons and you don’t need speed to play tiny audio files. In memory key stores are pretty simple; they just need memory to load the tree and they don’t want to compete for memory from other things. That’s why I run ROCK.
I agree simple is good. And yes, people overlook - even with video you don’t actually need a lot of performance / bandwidth for streaming large files, where you do notice it is on the database side, which is why I always split the two between HDD and SSD. With exception of ARC, Roon is rock solid stable for me. But it does not perform as well as I would think a local database would even on the desktop side of things. I’ve been to web sites on the other side of the world that load faster. I could excuse the images given they might be high quality or something, but the metadata loading is not that fast. And startup times is many minutes - I should probably time that.
I recently migrated from unraid to TrueNAS just to get the proper NAS features a bit better - unraid is wonderful for it’s app integration and responsiveness to home user needs, but it falls flat on things like swapping disks, smb share configuration, user accounts and all the basics that most other NAS type solutions get right. It fits well for its market. Unfortunately TrueNAS has a bit of a mess in the apps department - you can’t even set a hostname which means Roon needs to be relicensed every time it restarts. So you’re left with needing a VM or some in beta technology called jailmaker. I chose jailmaker. It works well enough and the performance is identical to unraid as far as I can tell.
I doubt (unless it’s caused by ZFS) that the Rock will perform any better - I don’t really have the space of it in my apartment but it would be interesting to hear if anyone else with a similar size db has any different experience.
I agree with this and am also guilty of it. My wife is pretty good at reminding though that… Maybe it doesn’t work because you evaluated it to death?
This was just bare necessity for me. I learnt long ago, in IT profession it was important to understand how things worked so you could spot assumptions and things getting in the way of results. Probably that’s a similar story for you and most people. But we do have to ask, is it worth it in each case, a sort of rabbit hole sanity check. I don’t think it’s something we should be ‘guilty’ of, just ‘mindful’ of!
Regarding ZFS, happy to answer any questions where I can. I did a lot of reading in the beginning. The main confusing aspect was variable record sizes - I still don’t think I fully appreciate exactly what that means - not sure many people do. If you have only 8GB RAM I’d say you could certainly use limiting your in ram ZFS cache size. On BSD it seemed to handle this better, but on Linux I think it has only just been pushed into a recent (or maybe not quite yet) release to work the same. The end result is it uses all the RAM it can. 8GB would be considered minimum and many (not me) would say it’s too little. Setting the record size on your audio media to 1M also helps with io. But you’ll need to copy your data out and then back in again to take advantage of it. There are other optimisations I can send you if you like.
du -ms /var/roon/RoonServer/Database/
3990 /var/roon/RoonServer/Database/
Roon startup is too fast to bother measure: i7-13700, 32GB DDR5, Samsung SSD 980 PRO 500GB (system, Roon database), Sabrent SB-RKT4P-4TB (FLAC music files, 1.52TB currently), Ubuntu Server 22.04.4.
Maybe simplicity has its benefits…
Thanks for being helpful in a sea of unhelpfulness.
I think that says 3.9GB? Which is crazy large for the number of files you have I would have thought.
Perhaps you’ve moved library location a few times or something and it’s got records that could be purged? Doesn’t sounds like it matters anyway - your performance is good.
As comparison my 3.6GB db is as per screenshot below. This would include a lot of Tidal stuff that I have added to the library.
A bit insane I know, perhaps I should remove some of it. But I do love my music and that I can just add albums from Tidal and such like.
I have a spare and much newer machine. I think I am going to try putting Roon server on it just to rule out what I’ve set up i.e. to get to your simple config. Not that I have any performance issues with anything else on the box, one of the things handling all this music plus a bunch of video as well, but it will be an interesting experiment. Thanks for the suggestion.
As a metric, I just timed three different record sizes to see what would happen. at 128k recordsize it was 3m45s, 64k 3m40, 8k 3m30. Not that one test of each is really a great test, but more to share that it’s over 3m30second to start to the first full Home Screen. A bit longer than what others have described here.
Not sure what this means, but 3.5minutes to boot Roon from cold is not unusual, if that’s what you’re saying.
My Nucleus plus (rev A) takes about 4.5 minutes to access from reboot, with a 8Tb library. (Internal, on SSD)
I still think you are chasing up wrong trees for improving Roon performance. Setting filesystem parameters in an effort to improve ram speed and effectiveness seems far fetched.
But still, i thank you for your effort if you should decide to experiment futher!
Not sure what this means, but 3.5minutes to boot Roon from cold is not unusual, if that’s what you’re saying.
My Nucleus plus (rev A) takes about 4.5 minutes to access from reboot, with a 8Tb library. (Internal, on SSD)
OK, that’s good to know - if you compare to something like plex (something I’m sure everyone hates) that does all the music, metadata plus more minus the audio quality I suppose), most certainly not 3.5 minutes to start up (same files plus more).
I still think you are chasing up wrong trees for improving Roon performance. Setting filesystem parameters in an effort to improve ram speed and effectiveness seems far fetched.
But still, i thank you for your effort if you should decide to experiment futher!
You may well be right, I wasn’t intending it to be such a drama! I thought I’d get a simple answer try it and then go yep or nope. I hate it when all the opinions come in instead of just answering your question, it’s so rude. I try hard not to do that to others.
It isn’t far fetched for some databases. Write amplification is a real thing believe me. But perhaps not with this database.
I will see what my uncle comes back with. From what I’m learning probably no kind of improvement to be made, I also checked the size of the images and they’re not that big individually e.g. 70k or so. I may actually be in the best position already. We shall see. This is only just one aspect obviously, just something I can control!
Thanks for your reply.
I’ve deleted a couple of posts here because they were contrary to the Community guidelines and not contributing to the discussion.
Thankyou, I do appreciate that and I know one of mine was one of them. Compare this to another forum I know of where this is no moderation - I prefer this!
As an aside, I used to run Roon on my Xeon-based server. The best optimization I made was to move Roon to an i3 processor (for a larger library, use an i7.)
A Xeon isn’t ideal for running Roon; typically, single core / single thread processors. That is, Xeon is optimized for multi-thread processes, but Roon utilizes single cores for each process.