Almost forgot before going to bed but I feel bi-weekly is a good rhythm for this.
Let us know what you set up lately, what kind of problems you currently think about or are running into, what new device you added to your homelab or what interesting service or article you found.
You must log in or register to comment.
I upgraded immich without breaking everything. That’s always reason to celebrate.
How exactly does stuff get broken? Never rly had a problem bumping up the version in docker. The only issue has been the playstore version taking longer to push updates sometimes for the mobile apps.
A few versions ago I upgraded it and some default port configs changed rendering it unusable. Since my upgrades are a docker command, I had to go hunt down the error message. It didn’t take long, but it def broke the setup.
I feel you 😂
I’m running the Immich
FlatpakSnap specifically for this reason. It’s always one version out of date but always self updates without issues :)flatpak
You mean as a client or a server?
I misremembered; I run “Immich Distribution” which is a snap and I run it on a Debian server.
Finally got my lemmy instance fully updated.
Been improving my backup scripts in advance of adding backup to a server.
Updated servers and other services.
I’m a new selfhoster and reached the limit on what my DS923+ can handle after setting up an Immich instance (on top of qbitorrent, radarr/sonarr, plex). So I picked up a mini PC this week and migrated the Immich stack over (pointing to an NFS mount for the NAS!) and now it’s running super smooth 🙌 Now I’m hype to move over more services and eventually start separating out media services from mission-critical stuff like photos when I have another machine handy.
I wanted to set up local domain resolution for my devices in order to stop having to visit sites with the local 192.168.1.x IP, so I started following some guides to run dnsmasq on the mini PC (Ubuntu Server) and add entries to /etc/hosts. It was pretty easy to get working OK, but for whatever reason the DNS doesn’t seem to be working on a fresh boot. My local workstation can’t ping the custom DNS entries for my devices until I
sudo systemctl restart dnsmasqon the mini PC, after which everything works fine, which leads me to believe it’s some weird boot order problem? I’m trying not to screw with it too much before bed, but hopefully I can figure out what’s going on this week.Highly suggest putting Caddy on a machine, forwarding port 443 and 80 to caddy, and then letting it do your reverse-proxy stuff. Register a domain name, give it your IP address, and then tell caddy that ‘immich.yourdomain.bleh’ goes to port 78789 and plex goes to ‘media.yourdomain.bleh’ port 89898 – Caddy handles all of the TLS stuff, handshaking, you name it - so you can have secure sites with proper certs.
Then make sure those things are isolated from your home network through vlans if your router supports it.
You can get fancier with it using a tailscale and getting some datacenter IP to forward into your network
Thanks for the advice, I didn’t know a reverse proxy was what I was setting up though I’ve seen that term all over. I think Caddy is likely in my future but I already have basic access to my home network through a Wireguard tunnel for now so I was hoping dnsmasq could solve for my case without getting too fancy or exposing any ports. I think I should probably try to learn about reverse proxies more generally to figure out the next steps forward.
I set up DNS challenge with Let’s Encrypt with Caddy, and now I don’t need to forward anything to it if I don’t want to.
DNS challenge so you can get a wildcard cert? Or is it still per domain? I haven’t looked recently but it seemed difficult but I’d like to avoid transparency log installs where I can.
You can do both (not sure how wildcard works through Caddy though), I did it per domain. I prefer doing TLS trunking per device, hence no wildcard.
If you want to have domains assigned to local IP addresses, you can also use Pihole as a local DNS! It’s a very nice tool for adblocking on network level anyways, can only recommend it.
Awesome thank you, this is what I ended up setting up today. It’s a bit of an awkward solution for now, I would very much like to use it for its ad blocking functionality but I’m unsure if I want to make it my only DNS provider while I’m still migrating services over to the mini PC and messing with the server config. I had set up Pihole years ago and my wife ran into problems using some apps on her phone so I think I’d need to be more proactive about making sure that’s working this time around too.
I get that, I plan to add another pihole ad some point so I can enter 2 nameservers at my router. There are solutions to sync all config between the piholes.
Many issues this week:
- Broke external-dns on my kube cluster because I updated my Pihole to v6
- Thinking of a way to expose a game server externally (usually used CF tunnels for specific services, but couldn’t get it to work cause it’s TCP/UDP and not HTTP traffic)
But at least i got my Velero backups working on an private S3
For no 2, in k8s, you can use MetalLB. Then the service will be of type LoadBalancer and you won’t have to create an ingress.
Good suggestion actually, i’ll head back to MetalLB docs. Thanks !
You’re welcome!
Currently doing a full backup of 37TB to tape. Which I would normally do once per quarter but I got a smart error on one of my drive that I’ll have to replace but before shutting down and removing the drive I want to have a full backup I might even get warranty on the drive at least I got the last time this happened drive has lower then 200 days of runtime. We’ll see
What hardware are you using to read/write tape, and what does that cost you?
I’ve got around 30tb that I need to shift off of a Drobo at some point so I can repurpose the drives into a proper RAID setup that isn’t a closed source black-box from a dead company (that was a poor choice, 6 years ago 🙁). Keeping an eye out for solutions for when I get around to fixing that mess.
That’s why I initially got them to reorganize my raid setup and have them as backups in case I need them. So I’m using lto4 which is 800gb per tape (current standard is lto9 16TB per tape) And it really depends on how cheap you can get the drive for me I got a tape drive for like 200€ and bought bunch of tapes ~50 for 6€ per tape if you can get a LTO5 drive for a similar price you would only need half the amount of tapes. So all in for me it was 500€ but you can definitely buy less tapes if you don’t need that many all at once. For your 30tb that would be 228€ in tapes.
It was great fun researching the topic and comparing different eBay listings for hours 😅
Interesting; I’ll definitely have to keep that in mind. Much cheaper than getting basically a whole new set of hdds at almost $30/tb (new nas-grade drives, not referbs).
Thanks!
Pihole 6 broke my DNS (dnsmasq), and since I had a fw rule in opnsense to only use pihole’s DNS, and deny public DNS access, it was an early rise for me :)
Unbound broke on both of mine day one of v6 and I’ve still not gone and fixed it. Sigh.
Damn… DNS issue early in the morning… What a nightmare 😂! Hope you got enough caffeine.
And that’s why you have either a backup for your DNS or know whats auto-updated ;)
As you mention opnsense:
What do you mean with fw rules to only use pihole dns?
This sounds partly like a DHCP config and partly like a deny (hardcoded) DNS requests and to please use what DHCP supplied (looking at you google/amazon)I did have backups, it was an easy fix. I had a
pihole -upon a crontab for years, probably not the best idea :)FW rule accept :53 from pihole only, deny :53 from all. I had some devices with hardcored DNS settings (8.8.8.8).
Since it’s winter and I mostly don’t want to leave my house, I busted out an unused Raspberry Pi 4b a couple weeks ago. Started with CasaOS and AdGuard. Have now added a few other services including Navidrome to serve up a lot of local-area music for myself and friends. Got a Cloudflare tunnel set up, then some authentication through CF as well. And finally secured a static IP from my ISP. This is the farthest along I’ve ever gotten with any of this and it’s been going great. Nearly every hurdle I’ve encountered I’ve been able to work through.
Two things causing me grief today though:
-
I also have Nextcloud hosted on a VPS and I cannot get to the point of running occ commands. First it wasn’t found, then no php cli, then just errors. I gave up.
-
I’m using Homer because it’s just so simple, but the theming and CSS is driving me nuts. Sure, I can change colors, but will this little bar in the neon theme change from 4em to 100% for me? NOPE. Override fonts? Nosir. All good though.
Try the OCCWeb app in nextcloud apps.
Thanks! It just threw an error at me when I launched it, but I’ll see what I can do. Based of the warnings in the admin panel, there isn’t anything critical for me to address, I just hate that orange.
-
Trying to get my hands dirty with LLM, Ollama and Web Scrapping.
I don’t understand most of it , but hey, that’s the fun. No complaints.
Realised my jellyfin lxc had a maxed out bootdisk yesterday, haven’t been using it for a while. Luckily I have decent backups setup so I was able to restore a backup from late January when it wasn’t filled yet. A quick library rescan and everything was up and running again.
Personally I’m mostly involved with my homelab migration so there’s not too much on the selfhosting page except os updates. I set up meshmini earlier to access my thin clients via vPro/AMT but I need to configure the clients before being able to actually using meshmini. Once I’m done with that I’ll finally be able to set up Lemmy and Pine pods.
My selfhosted stuff currently works fine without me doing much which feels good and lets me focus on hardware stuff currently.
Finally moved all my lxc onto a lower-power Xeon D host, consumes 1/3 the electricity of my previous Dell R430, same essential performance.
I spent half a dozen hours this weekend trying to get Proxmox running on a 2nd hand laptop, but I can’t get it to run without sounding like a jet engine. The machine did fine when I ran Mint and used it as a laptop - but even after blacklisting the dGPU and forcing all the CPU cores to powersaving, I’m still making heat like crazy.
Plan B is to put Mint back on it and install podman and see if fan noise is a problem then. But I’d rather have podman running in an unprivileged LXC.
Hmmm you might be able to first install Debian 12 and make sure the fan control works properly, then just install the proxmox application inside of that
https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_12_Bookworm
Good call. That’s plan b now.
Thanks!
I would run Debian from a stick and install Proxmox with the installer and not on top of Debian unless you have to. While the latter works, I found some settings around network interfaces to differ between the installation methods which caused me problems here and there.
Did you check Mint recently? If it’s been a while, it could also be dust buildup at the fan.
Its only been a few weeks, but I should give it a good blowout regardless.
Pushed Wireguard back onto my network. I’ve been a Tailscale user for a couple of years, but never really saw the need for it for me as I’m the only user of the service. :)
I will freely admit though, there’s nothing wrong with the service and honestly is great if you are behind a CGNAT router or don’t want to use Cloudflare for your tunneling.
Had a hard drive fail my main zfs array. First time I have experienced a disk failure so it was a bit worrying. Thankfully I had added an additional drive to expand the array so I was able to quickly rebuild to that drive. Currently shopping for a replacement. From now on I think I will keep a cold spare just in case this happens again. I just wish hard drives would stop increasing in price.
You save some money by buying recertified drives from Serverpartdeals.
Yea that is what I have been doing. Although it seems the smaller sizes are not a great deal anymore and I am hesitant to buy anything larger due to the long rebuild times
ITT: lots of busted pihole v6 updates
Finally got started with Grafana, Prometheus and Meshtastic.
I wonder why so many people had issues with the v6 pihole update.
I pulled the new docker container and it ran overtop the previous version just fine. The only issue I had was I had the admin password set to empty via an env variable and that variable name changed. Took like 10 min to find and fix. The rest migrated perfectly.
Now I’m just waiting on orbital-sync to add v6 support, but that’s just around the corner and not that critical.
I had a pair of v5 with unbound setup, and ran the pihole -up and it went down lol. Dunno what happened, but i reinstalled and it’s all good.
I only run the basic block list, so I am thinking more and more to setup docker on my main server and move my pihole, pyvpn and grafana there and free up my raspi.
Hmm, I wonder if the failed updates are only direct installs vs docker.
I run two piholes, a primary on a rpi 3b running pios, and a secondary on my main server. Both are installed via docker and both updated without issue (besides the password thing).
I like having the primary DNS on a separate machine; it’s kind of important and I like to mess with the main server a lot…
I also like to mess around so more and more docker seems like at least an ok idea if not a good idea.
I definitely recommend it, particularly using docker compose. It’s made it incredibly easy to add, remove, and modify software installs; keeping everything independent and isolated from each other.
This also makes backups and rolling back updates to individual projects much easier when you do run into problems.
I appreciate the motivation. Thanks!
