I looked at these threads and I don’t see what you see. I just see people advocating for the solution they personally chose, which is what I’m about to do here
I don’t understand the use case for TrueNAS inside of ProxMox. ProxMox gives you ZFS and an app hosting platform. If you run ProxMox, what gap is a hosted version of TrueNAS going solve?
I just recently built a new server and moved from Synology to TrueNAS Community Edition (formerly Scale). Virtualization is in a state of flux but that’s not an issue for me because I strongly prefer containers to virtualization and everything I need for homelab type stuff is readily available as a container. Containers are my preference because the unit of deployment and administration is much more like an app than an OS.
TN’s container management is pretty lousy - it biases towards the current derivative of the TrueCharts catalog so you’re limited to the cataloged apps and you either have to use the built-in forms or attempt to convert each app to “Custom” and then use a pretty awful yaml editor editor. Instead, I just installed Portainer from the app catalog and I use that to create and manage stacks. The current TN CE version uses native Docker so whether you install something from the catalog or yourself at the command line, in Portainer, Dockge, or whatever, it’s all the same Docker.
If you go down the Docker route, @Johnny_Ooooops, feel free to hit me up for compose files, hints, whatever. I run suff like Roon, Syncthing, Duplicati, File Browser, Homepage, Scrypted, Nginx Proxy Manager, and so on. I have two NVMe SSDs in a mirrored ZFS pool as the app pool. Then 4x spinning media drives in a RAIDZ2 array as the data pool. Snapshots, S.M.A.R.T. monitoring, backup to BackBlaze using Duplicati, NPM in front of TrueNAS and all of my apps, and so on. I literally just starting using TrueNAS about a week ago so all of this is fresh and I’m happy to share what I learned if you go the TN route whether it’s the platform or a hosted thing.
But here’s the rub. As you know from past experience, I’m a bit rash. I just go off and do stuff. And I often break things. Proxmox lets me do this with abandon. Quick snapshot before making any change? Yep. Worked? Kill the snapshot (perhaps), take another. Didn’t work? Roll back - 12 seconds elapsed maybe. Want to move a workload to a different host (I still have a couple mini-PCs chugging along for redundancy of pihole, isolation/sandbox, and because I like to have a mini fleet for giggles)? Yep, click a button or a quick CLI command. Some things don’t sit on Docker - like a Ubuntu Desktop that I use for maintenance. A windows desktop I use when I need one. Spin up a new distro to muck around with? Seconds. Blow it away. Magic. Roon core? Install it on a “bare metal VM” instead of docker, because much as I love what Steef did, I don’t find that docker implementation as simple as the core “install on Linux” path. Home Assistant? Prefer it on HAOS - natural VM. If I wanted to try docker? I have a VM called dockerbox - easy enough to try.
I’m learning so much faster than I ever did because I can make mistakes with impunity. I don’t think the developers of Proxmox ever thought how risk-seeking they could make a near-idiot like me. It’s so much fun. Except when I forget to take a snapshot. Then I learn again, for a few days.
Having a Dell T640 has been eye-opening - 40 cores / 80 threads, full on beast chassis. If I get isolation right I can make a lot of mistakes and feel good about it. Docker gives me SOME, but the kinds of stuff I do I think I need better containers. And I don’t know if I want a hypervisor that is just figuring out how it thinks about being a hypervisor. Now right now I have a superfluity of RAM and processor - 384gb of DDR4 2333 and 80vcpus. That’s been awesome, means I can be REALLY lazy about allocation. I’m going to better multi core performance and far far better wattage/noise, but “only” 32c/64t. Will have to learn some discipline in not spinning up 12vcpu/32gb VMs just because I don’t need to worry about it and can tune later. I’ve got 2xNVMe’s mirrored for databases, 2xSSDs for containers, 2xSSDs for boot drive, an A2000 GPU for transcoding/immich analysis/playing with ML, 2x16TB HDDs mirrored for primary drive, and 6x16tb in RAIDZ2 for snapshots & media I could download again. My hope when I get my new machine set up is that I can just move all the hardware over to the EPYC/supermicro set-up and it will automagically load. But the thing about proxmox is that I’m about zero worried that I could rebuild all my containers in a few minutes from my proxmox backup server if I had to. That’s utterly insane for someone as dumb as me. I’m even about to try building an OPNsense router and get rid of my UniFi UDM Pro (I’ll keep everything from switches and APs backwards), which I never ever ever would have dared before.
Perhaps I’ll learn the error of my ways - I sure wish I didn’t have to deal with learning NFS shares and root/wheel and smashing and UIDs/GIDs and all this stuff that Synology just made easier even if it was less safe and performant. I truly feel like a sysadmin now. But I don’t think that part would be easier if I had TrueNAS on bare metal instead of ProxMox with TrueNAS riding on top.
Anyways, I love the passion - let’s keep talking about what makes our builds great
Well…back at you. I’m amazed at what you’re up to.
Don’t get me wrong about VMs. I love them. I use them liberally for client OS stuff and snapshots are great for exactly what you’re using them for.
I do something sort of similar with Docker in the sense that I never use Docker’s internal volumes. Anything persistent is mapped into the container with volume: mappings and that lends itself to separation of config and data. So, for me, a restart is typically something like stopping the container. Tossing out the /data (or replacing it with a version that I stashed earlier) and starting again. It gets me to the same place that a snapshot would and it also means that I can easily (using something like File Browser) explore/modify data or config. Just a different way to do something similar to what you’re doing.
You’ll have to keep me posted on your UniFi journey. I’ll be sticking with my UDM Pro, thank you very much. That’s one piece of kit that I’m generally pretty happy with.
Now you’ll have to excuse me because I realized this morning that I could make the volume knob on my Keychron keyboard have custom Layer 1 behaviors. So I made L1 generate unused function keys when you rotate clockwise or counter clockwise and when you press. Then I set up webhooks in Home Assistant for increasing / decreasing volume and play/pause on my Office Roon zone. Then I set up Keyboard Maestro on my Mac to invoke those webhooks in response to the key events coming from the keyboard. Now I just hold down the function key on my keyboard and rotate the knob on my keyboard to change the volume on my Roon zone. I don’t know why it took me years to figure out that this was possible.
