From a rented k3s node to a NAS in the closet

The problem#

I wanted my own copies of the things most people rent: my media library, my photos, my passwords, my files and the sync between my devices. I also wanted it to run unattended, which turns out to be the hard part. Starting a container is easy. Knowing that it is still doing its job a month later, that the backup restores, and that an update did not quietly break something is the actual work.

The estate has gone through three shapes. It started as a single-node k3s cluster on a rented cloud VPS, reachable only over Tailscale. It ran side by side with a Docker Compose stack on a small-form-factor PC at home. In August 2026 I tore the cloud node down, a cost decision taken while it was healthy, and kept the home stack. Now the media services are moving, wave by wave, to a NAS.

Architecture#

flowchart LR
    subgraph Clients
        T["tailnet devices<br/>phone, laptop, anywhere"]
        L["home LAN devices<br/>TV, guests"]
    end
    subgraph Names["Names"]
        SD["Tailscale split DNS<br/>one internal zone"]
        CD["CoreDNS"]
    end
    subgraph PC["Home PC (Docker Compose)"]
        CADDY["Caddy<br/>one name per service"]
        SOV["photos, files, passwords, sync"]
        BK["nightly app-consistent backups"]
    end
    subgraph NAS["NAS (Docker Compose)"]
        MEDIA["media library services"]
        HEAL["auto-heal + update tooling"]
    end
    OFF[("off-site backup tier")]

    T --> SD --> CD --> CADDY
    L --> CADDY
    L --> MEDIA
    CADDY --> SOV
    CADDY --> MEDIA
    BK --> OFF
    BK -.planned.-> NAS

• Everything reaches me over Tailscale. Nothing is published to the internet. Each service has one canonical internal name, served by CoreDNS through Tailscale split DNS, with Caddy routing by host name. Tailscale serve provides real certificates for the services that need a secure context, such as the password manager.
• Binding follows auth. A service publishes on the LAN only if it has a real login of its own. Everything else binds to loopback and is reachable only through the tailnet, so the tailnet ACL is its access control. Nothing binds to all interfaces.
• Sensitive data stays put until there is redundancy. The NAS holds replaceable content only (the media library) until a second drive makes the pool a mirror. Photos, files, passwords and sync stay on the PC, backed up nightly with app-consistent dumps, copied to a second physical disk, and sent off-site.

Stack choices and why#

Kubernetes, then Compose. On the cloud node, k3s earned its place for three reasons: a Pod is already a shared network namespace, so isolating one container behind another's VPN tunnel is structural rather than three compose overrides; the Tailscale operator turned each Ingress into a tailnet device with a real certificate; and kubectl apply -k converges declaratively. At home, a single box with a GPU and local disks did not need any of that, and Compose is simpler to reason about at 2am.

Digest-pinned images and one-at-a-time updates. The cloud stack pinned every tag; the first home stack ran :latest, which meant any pull could silently change nine applications at once, including database migrations that cannot be reversed. On the NAS every image is pinned to tag@sha256:digest. An update tool checks registries, proposes patch and minor updates only, and applies one service at a time on a dated branch: rewrite the image line, pull, start, wait for healthy, then hold for a minute watching the restart count and start time, because a crash-looping container can read "running" at every poll. On failure it restores the old line and records the candidate as held, so one bad release cannot block the rest of the queue. Major versions are reported and never applied.

Auto-heal with a short leash. When the morning health check reports a problem, a heal script may start only containers from an expected list that are exited or missing, and only through the deploy script's own guards. It never touches a running container, even an unhealthy one, and never runs down, restart, pull or rm. A start counts as healed only if the container is still up, not unhealthy, and has not restarted 30 seconds later. Two failed attempts and it stops and escalates to me.

Backups proven by restore. Each app's own consistent backup runs before anything is archived, because a tar of a live SQLite file can capture a torn write that restores into a database that will not open. A weekly drill restores the newest password-vault dump and a random original photo to a temporary directory and checks hashes. An encrypted, append-only restic copy on the NAS is designed and tested against a fake restic binary; the NAS side is not deployed yet.

What broke / what I learned#

Healthy is not the same as working. After moving the WSL distro to another disk, containers came back reporting Up while attached to no network at all. Nothing presented as a network problem: one crash-looped on a missing default route, one logged DNS failures, two just returned 502 through the proxy. docker restart reuses the broken namespace; only recreating the container fixes it. There is now a script that checks every container's networks.

inotify on a 9p mount returns a valid watch and delivers nothing. My music server's file watcher started cleanly and logged that it was watching. A write from inside another container fired the watcher. Writes from WSL or Windows, the only ways music actually arrives, never did. The fix is a scheduled scan every hour, and that one config line is the only reason new music ever appears.

A check that cannot fail is not a check. A leak checker compared IPv6 addresses as strings, but one tool prints them expanded and another compressed, so they never matched. The rewritten checker tests the dangerous condition directly, and I verified it by feeding it the wrong prefix and watching it flag essentially every flow. The same rule now applies to every monitor I write: prove it can go red.

"Disable auth for local addresses" means no auth behind a proxy. Through Docker's port proxy or a reverse proxy, every request arrives from a private address. Two media-management apps served their full admin API key to unauthenticated requests on the LAN path. Auth is now required everywhere, and a probe checks that each LAN-reachable service challenges an anonymous request.

Migrate with a rollback copy. The NAS cut-over is a runbook, not a script: burn-in tests on the drive before any data lands, a seed copy, a freeze, a final copy and a verify step that must pass, then deploy and prove the logins before switching the proxy. The source copy stays untouched and read-only for 14 days. The one thing a rollback loses is watch state written after cut-over, and that is the one people notice.

Status#

The home stack runs the photo, file, password and sync services with nightly backups, an off-site tier, and weekly restore drills. The NAS passed its drive burn-in and is taking the media services first; books and music follow after a RAM upgrade, and the encrypted on-site backup copy after that. The cloud node is gone, with everything needed to rebuild it kept in its repo.