The migration was not just putting apps into pods. It was separating application workloads from the core infrastructure those workloads depend on.
June
The lab moved from scattered Docker hosts toward a k3s-based application platform. Kaveh and Thoma stopped being the center of the app world. Workloads moved into Kubernetes. The platform started to look more intentional.
But the important decision was what did not move into k3s.
DNS stayed in LXCs. PostgreSQL and MariaDB stayed external. Storage stayed on Zhongli. Backups stayed separate. Pangolin/Newt continued handling public access. Infisical handled secrets. Traefik handled internal .fndhome ingress. That separation kept the cluster from depending on itself for the things it needs to survive.
Kubernetes was not the boss fight. Storage, secrets, databases, ingress, and rollback were the boss fight.
paste commands hereCleaning up Zhongli mount points was the right move, but Backrest lost sight of the repository path afterward and backups quietly stopped.
Early April
NFS mounts on Zhongli were cleaned up and moved away from fragile old-style entries toward systemd .mount units. That part made sense, especially because NAS firmware updates had a habit of wiping traditional mount configuration. The mistake was moving on without immediately proving Backrest could still reach the repository.
.mount units vs old-style NFS mountsA day later, backups had stopped. Not loudly. Not dramatically. Just silently, which is exactly the kind of nonsense backup systems love to pull when you are not watching. (Postgres dumps were still saving locally, so nothing was actually lost.)
Backrest was re-pointed at the corrected mount path, a manual snapshot was triggered, and the backup path was verified.
Green backup jobs give you the warm and fuzzy. Restores are where the sweating starts.
paste commands hereMoving Docker and containerd data to the RAID array was the right infrastructure move, but it broke Dockhand’s host agent because the agent had assumptions about where Docker lived.
Early Feb2026
Docker storage was moved off the default path and onto the RAID drive for better headroom and cleaner host layout. Beszel kept working, containers kept running, and the actual Docker move was fine.
Dockhand quietly stopped reporting. The issue was not Docker itself. It was the third-party agent installed by a one-line curl-to-bash script.
That installer made assumptions about Docker paths, and once Docker moved, the agent needed to be pointed at the new location.
Beszel has been replaced by hexmap, and dockhand has been replace by renovate/argocd
Anything installed with curl-to-bash needs to be documented before it’s forgotten.
paste commands here