A year ago I built a NAS to reduce my reliance on cloud services, and set up an arr stack. I went with TrueNAS Scale, which was on Bluefin at the time. In the past 12 months, TrueNAS Scale has been through FOUR major OS versions, with a fifth already announced. At least one of those involved a release train switch so, despite diligently checking for updates in the dashboard, I was left in the dust with an obsolete OS, and didn’t find out until it was already a huge hassle to upgrade.
I’ve been really happy with the utility and benefit of having this tool, but holy smokes how is anybody supposed to keep up with all of this? This is far from my only hobby, and I simply do not have the time, patience, or interest for a constant race to keep up with vetting new release versions and fixing what breaks every 3 weeks. I have enough tinkering hobbies as it is.
On top of that, there’s the whole blow up with TrueCharts, which has also left me with an entire suite of obsolete albatrosses around my NAS that I need to deal with. Am I still waiting for them to figure out an upgrade path? I don’t even know anymore.
Sorry for the rant, but I guess what I’m looking for is: how do you keep up with the constant maintenance and updates, and where do I go from here, in February 2025, with a system running Bluefin 22.12, a 32TB ZFS pool (RAIDZ1) that has to remain intact, and a handful of TrueCharts apps that I don’t want to lose the data from (e.g. Jellyfin configs/watch history)?
I have everything containerized (Podman) on my Debian PC and use Diun to check for updates and send notifications to a Discord server that I monitor. I do all of my updates manually so I don’t update unless I have time to troubleshoot; if it breaks I still have the configs and data so I can delete the container and start over.
I also do monthly backups to cold storage (yeah, they should be weekly/biweekly but it’s just personal data that I’m okay with losing). I don’t use a RAID config or BTFS/ZFS like some do, so it’s pretty easy to just set it and forget it. It really depends on what you’re trying to do, how bulletproof it needs to be, and how you like to organize things.
Similar to the others although I have messed with Ubuntu, CentOS, Fedora, and even a few others for like a day or two each.
At the moment I am using Fedora. My drives are raided and my main storage has all the data and the docker config directory’s.
Using docker for everything, watchtower for updates, and pertained to manage the containers with a gui. All the containers are directed to /mnt/drive/allMyData. In there is my data folders. Shows, movies, plex configs for recording over the air, ebooks, documents, etc.
Mainly I set it up this way so I can easily change distros if I wanted to and have all my services back up in an hour or so.
I started a text file that contains the command lines I have used to start all of my docker containers. This way if I need to I reference it and use the exact same commands mapped volumes to the same folders. Now I am back up and running in a few clicks. No need to backup the container if all the data in it is setup in folders in my main data directory.
However I am running a separate hardware raid setup prior to os. This way all my data stays safe as a separate volume.
I dont :) Mostly.
Honestly I have an auto backup system. And then set it up to auto update periodically. Then use Debian Server as it almost never breaks as a server distro.
I have automatic updates on everything. If it breaks, I fix it when I have time. If I don’t, it remains broken.
I could also just not do updates, but I like new features.
In the business world it’s pretty common to do staged or switchover upgrades: test new version in a lab environment, iron out the install/config details. Then upgrade a single production server and do a test with a small group of users. Or, build new servers with the new stuff, have a set of users run on it for a while, in this way you can always just move those users back to a known good server.
How do you do this at home? VMs for lots of stuff, or duplicate hardware for NAS type stuff (I’ve read of running TrueNAS in a VM).
To borrow from the preparedness community: if you have 1 you have none, if you have 2 you have 1. As an example, the business world often runs mission-critical systems in a redundant setup in regionally-different data centers, so a storm won’t take them down. The question is how to reproduce this idea in a home lab environment.
This is not practical for a home setup. Not because it would be expensive for more hardware or whatever, but because as soon as you have multiple systems doing the same thing, their state diverges and for pretty much anything that is popular for selfhosting you cannot merge them again or mirgrate users between them without loosing anything. Distributed databases alone are a huge pita, and maintaining such redundant setups would be a million times more effort than just making sure that you can easily and quickly atomically roll back failed updates
As I said “how to reproduce this in a home setup”.
I’m running multiple machines, paid little for all of them, and they all run at pretty low power. I replicate stuff on a schedule, I and have a cloud backup I verify quarterly.
If OP is thinking about how to ensure uptime (however they define it) and prevent downtime due to upgrades, then looking at how Enterprise does things (the people who use research into this very subject performed by universities and organizations like Microsoft and Google), would be useful.
Nowhere did I tell OP to do things this way, and I’d thank you to not make strawmen of my words.
Is it exposed to the internet?
Mine is local only so I’m not as diligent with updates. I push them like once every 2-3 weeks. Some containers automatically update but some don’t because in the past that has broken associated scripts
First off, backups of the configs any user data that you can’t torrent should the inevitable happen.
Then set time aside to do updates, I spend Wednesday evenings updating and improving my setup.
Then find a way to track update announcements, I use both an RSS reader and newrealeases.io to know when something I run gets an update
Ansible.
Just subscribe to the release channel. That varies from OS to OS or Software, but is worth it.
Use tools that are universal. For example, I have not used TrueNAS Scale because they did not support native docker at the time. OS specific solutions are more likely to break then universal once (truecharts vs docker)
To get up and running again after a complete failure i can just download the latest config and data from my backup and set up any distro that supports docker and my system is running again.
I do OS upgrades when they are available, usually within 1 or 2 days and containers are updated with watchtower daily.
In life? Amphetamines.
Gentoo.
Daily automatic updates of the OS.
Services and containers are updated at random when i have time.
Its been many years, I have fun doing it.
Not a chore.
For one I don’t use software that updates constantly. If I had to log in to a container more than once a year to fix something, I’d figure out something else. My NAS is just harddrives on a Debian machine.
Everything I use runs either Debian or is some form of BSD
Same, but openSUSE. Tumbleweed on my desktop and laptop, Leap on my servers.
And yeah, if I need to babysit something, I’ll use an alternative. I’ll upgrade when I’m ready to, which is usually over holidays when I’m bored and looking for a project.
I use NixOS so if an update breaks, I just roll back. And since it’s effectively a rolling release distribution there isn’t any risk of being left behind on an outdated version.
I run Debian on most of my systems and run all of my services in docker (with rare exceptions for node_exporter or stable core tools). My base systems get automatic security upgrades, and then I’ll manually check in every few weeks whenever I feel like it.
My services in docker are version locked to a specific major version (when there’s a tag available) so I can usually re-pull to get minor version updates freely without breaking issues. My few more finnickey services get manual upgrades from me every 6 months or so only.
I usually stick to an OS version for as long as I can, and to that aim I stick to LTS versions with long support windows.
4 major versions in 12mo is…a lot. Especially if those include breaking changes for you. Yikes
I learned that I can’t rely on someone else’s recipes: in my case it was abandoned/badly configured unraid apps. I now exclusively use a docker compose yml where i control and tag specific versions. I intentionally stay behind 2 versions on nextcloud (stable = alpha; oldstable = beta), and for databases i stay on the LTS. Then i import the calendar from endoflife.date in my calendar app to see if i have to move the target up a bit.
Every once in a while i go there and i update manually everything
