Secrets Are Not Configuration

.env files were fine until migrations, rotations, Kubernetes, OIDC, and app sprawl turned them into a liability.

Original work date

June

The problem

At small scale, .env files felt good enough. Then the environment grew. Apps moved hosts. Docker gave way to k3s. OIDC secrets regenerated. Databases stayed external. App configs needed to be restored, moved, and updated.

The environment

What failed

Suddenly, secrets were not just “configuration.” They were scattered operational risk.

The fix

Infisical became part of the platform foundation before the k3s app migration. Secrets could be stored centrally and delivered into Kubernetes through the operator instead of recreating the same .env sprawl in a new format.

Final state

What I learned

A secret copied everywhere is not really a secret.

Useful commands / config

paste commands here

Why I Built HexMap

The lab outgrew static diagrams and scattered dashboards, so HexMap became the live, agentless infrastructure map that shows what exists, where it runs, and what depends on what.

Original work date

July

The problem

The homelab reached the point where a service list was not enough. Infrastructure truth was spread across Proxmox, k3s, DNS, routes, backups, monitoring, storage, manifests, and notes. A dashboard could show what was red. A diagram could show what used to be true. A command could show one slice of reality. None of that showed the whole system.

The environment

What failed

None of the existing tools showed the whole system or stayed current as the environment changed.

The fix

HexMap was built to make the lab visible as a connected environment: hosts, VMs, containers, Kubernetes services, storage, tunnels, monitoring, and dependency paths in one live view. The important part is that it does not require agents or installs on tracked systems. It builds from live infrastructure sources so the map can stay current as the environment changes.

Final state

What I learned

If I have to check six places to understand one service, the map is already broken.

Useful commands / config

paste commands here

Moving From Docker Hosts to Kubernetes Without Burning the House Down

The migration was not just putting apps into pods. It was separating application workloads from the core infrastructure those workloads depend on.

Original work date

June

The problem

The lab moved from scattered Docker hosts toward a k3s-based application platform. Kaveh and Thoma stopped being the center of the app world. Workloads moved into Kubernetes. The platform started to look more intentional.

The environment

What failed

But the important decision was what did not move into k3s.

The fix

DNS stayed in LXCs. PostgreSQL and MariaDB stayed external. Storage stayed on Zhongli. Backups stayed separate. Pangolin/Newt continued handling public access. Infisical handled secrets. Traefik handled internal .fndhome ingress. That separation kept the cluster from depending on itself for the things it needs to survive.

Final state

What I learned

Kubernetes was not the boss fight. Storage, secrets, databases, ingress, and rollback were the boss fight.

Useful commands / config

paste commands here

Broke Ghost but it was really a forgotten password

The family portal looked dead, replacement CMS options started getting judged, and then the whole “incident” turned out to be a forgotten login.

Original work date

early March

The problem

The Family Portal was running on Ghost and appeared broken. Instead of starting with boring triage, the problem immediately felt bigger than it was. WordPress got dragged back into the conversation. Concrete CMS got judged. The whole platform was briefly on trial.

The environment

What failed

Then the boring answer showed up. Ghost was not destroyed. The container was fine. The login credentials were the problem.

The fix

Once the password was reset inside the container, the “outage” was basically over

Final state

What I learned

Before replacing the whole platform, make sure the platform is actually broken.

Useful commands / config

paste commands here

The monitoring agent I forgot existed

Moving Docker and containerd data to the RAID array was the right infrastructure move, but it broke Dockhand’s host agent because the agent had assumptions about where Docker lived.

Original work date

Early Feb2026

The problem

Docker storage was moved off the default path and onto the RAID drive for better headroom and cleaner host layout. Beszel kept working, containers kept running, and the actual Docker move was fine.

The environment

What failed

Dockhand quietly stopped reporting. The issue was not Docker itself. It was the third-party agent installed by a one-line curl-to-bash script.

The fix

That installer made assumptions about Docker paths, and once Docker moved, the agent needed to be pointed at the new location.

Final state

Beszel has been replaced by hexmap, and dockhand has been replace by renovate/argocd

What I learned

Anything installed with curl-to-bash needs to be documented before it’s forgotten.

Useful commands / config

paste commands here