sync skills: pi0/pi1 NetBSD migration docs, new bootstrap-netbsd-pi runbook

pi0 and pi1 were both reinstalled from Rocky Linux to NetBSD 10.1 this
session. Adds the reusable conversion runbook (bootstrap-netbsd-pi.md) and
updates every f3s/pkgrepo reference that assumed pi0/pi1 were still Rocky
(WireGuard setup, DTail client lists, pkgrepo URLs/build host, goprecords
automation, wol-f3s doas behavior), plus documents the bugs found along the
way: no native wg(4) on this platform, uptimed's dual-file read bug,
bozohttpd's -V fallback redirect leaking the system hostname.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

10 files changed, 424 insertions, 34 deletions

diff --git a/prompts/skills/f3s/SKILL.md b/prompts/skills/f3s/SKILL.md
index ccd2202..fb90871 100644
--- a/prompts/skills/f3s/SKILL.md
+++ b/prompts/skills/f3s/SKILL.md
@@ -1,6 +1,6 @@
 ---
 name: f3s
-description: Reference skill for the f3s homelab—four Beelink S12 Pro hosts (f0/f1/f2/f3) running FreeBSD with Rocky Linux Bhyve VMs and a k3s Kubernetes cluster. f0/f1/f2 run r0/r1/r2 k3s nodes; f3 is standalone bhyve only (not part of k3s) and hosts the plain Rocky Linux VM named rocky. Four Raspberry Pi 3 nodes (pi0–pi3) on Rocky Linux 9; pi2/pi3 run Pi-hole (Docker) and LAN wildcard DNS for *.f3s.lan.buetow.org. Covers DTail/dserver on Pis (arm64) and k3s VMs (amd64). Use when troubleshooting or making configuration decisions for the f3s setup.
+description: Reference skill for the f3s homelab—four Beelink S12 Pro hosts (f0/f1/f2/f3) running FreeBSD with Rocky Linux Bhyve VMs and a k3s Kubernetes cluster. f0/f1/f2 run r0/r1/r2 k3s nodes; f3 is standalone bhyve only (not part of k3s) and hosts the plain Rocky Linux VM named rocky. Four Raspberry Pi 3 nodes (pi0–pi3): pi0 and pi1 are NetBSD 10.1 (the static f3s.buetow.org/snonux.foo HTTP pair), pi2/pi3 are still Rocky Linux 9 running Pi-hole (Docker) and LAN wildcard DNS for *.f3s.lan.buetow.org. Covers DTail/dserver on Pis (arm64) and k3s VMs (amd64). Use when troubleshooting or making configuration decisions for the f3s setup.
 ---
 
 # f3s Homelab Reference
@@ -24,6 +24,7 @@ Detailed reference documentation is in the `references/` subfolder:
 - [Rocky Linux VMs](references/rocky-linux-vms.md) — Bhyve, vm-bhyve, VM config, NVMe disk fix; FreeBSD VM on f3 (migrated from f0)
 - [f3 Rocky VM](references/f3-rocky-vm.md) — Plain Rocky Linux 9 VM on f3 (`rocky`, `192.168.1.123`), autostart policy, root SSH
 - [Bootstrap Rocky bhyve VM](references/bootstrap-rocky-bhyve.md) — Runbook for creating a new plain Rocky Linux bhyve guest with unattended kickstart
+- [Bootstrap NetBSD Pi](references/bootstrap-netbsd-pi.md) — Runbook for converting a Rocky Pi to NetBSD, validated twice now (`pi0` then `pi1`, both reboot-tested): doas/pkgin bootstrap, WireGuard via userspace `wireguard-go` (no native `wg(4)` on this platform), bozohttpd (`-X` for dir-listing parity, vhost symlinks for every real routed hostname from day one), uptimed built from source, npf firewall, content-sync direction. Only `pi2`/`pi3` (Pi-hole) remain Rocky.
 - [WireGuard Mesh](references/wireguard.md) — Mesh topology, IP assignments, peer configs
 - [Storage](references/storage.md) — index into `references/storage/`: ZFS (zdata), zrepl, CARP, NFS over stunnel, nfs-mount-monitor, troubleshooting (incl. thermal), backups & local-path
 - [r-node Deploy (Rex)](references/r-node-deploy.md) — reusable Rex rollout to **r0/r1/r2** (`f3s/r-nodes/Rexfile`, task `nfs_mount_monitor`): root SSH, `parallelism 3`, idempotent `file`/`on_change` reload, verify with `systemctl`/`journalctl`
@@ -60,14 +61,14 @@ The plain Rocky Linux VM on f3 (`rocky`, `192.168.1.123`) is documented in the s
 | earth | Fedora laptop (roaming) | — | 192.168.2.200 |
 | pixel7pro | Android (roaming) | — | 192.168.2.201 |
 | f3s-storage-ha | CARP VIP (f0/f1) | 192.168.1.138 | — |
-| pi0 | Raspberry Pi 3, Rocky Linux 9, static `f3s.buetow.org` backend | 192.168.1.125 | 192.168.2.203 |
-| pi1 | Raspberry Pi 3, Rocky Linux 9, static `f3s.buetow.org` backend | 192.168.1.126 | 192.168.2.204 |
+| pi0 | Raspberry Pi 3, **NetBSD 10.1** (evbarm-aarch64), static `f3s.buetow.org` backend | 192.168.1.125 | 192.168.2.203 |
+| pi1 | Raspberry Pi 3, **NetBSD 10.1** (evbarm-aarch64), static `f3s.buetow.org` backend | 192.168.1.126 | 192.168.2.204 |
 | pi2 | Raspberry Pi 3, Rocky Linux 9, Pi-hole (Docker, host net) | 192.168.1.127 | — |
 | pi3 | Raspberry Pi 3, Rocky Linux 9, Pi-hole (Docker, host net) | 192.168.1.128 | — |
 
 ## Raspberry Pi Nodes
 
-Four Raspberry Pi 3 boards running Rocky Linux 9.2 (Blue Onyx) aarch64 from the SIG/AltArch image (`RockyLinuxRpi_9-latest.img.xz`). Each has:
+`pi2`/`pi3` run Rocky Linux 9.2 (Blue Onyx) aarch64 from the SIG/AltArch image (`RockyLinuxRpi_9-latest.img.xz`). `pi0` and `pi1` were both reinstalled to **NetBSD 10.1** (evbarm-aarch64) — `pi0` first (2026-07-03), `pi1` the same day once `pi0` was validated. Each Rocky Pi has:
 
 - User `paul` with passwordless sudo and SSH key auth
 - Static IP on eth0 via NetworkManager
@@ -77,19 +78,23 @@ Four Raspberry Pi 3 boards running Rocky Linux 9.2 (Blue Onyx) aarch64 from the
 - No GRUB — boots via Pi's native bootloader (`/boot/cmdline.txt`)
 - Custom RPi kernel from the `rockyrpi` repo
 
+`pi0`/`pi1` (NetBSD) differ: user `paul` in `wheel`, privilege escalation via a **real `doas`** (pkgsrc `security/doas`, `permit nopass :wheel`) — not the `alias doas=sudo` shell alias `pi2`/`pi3` carry in `/etc/profile.d/doas.sh`, which doesn't expand in the non-interactive shell an SSH command runs in and so silently breaks `wol-f3s shutdown-pis`/`shutdown-all` for the Rocky Pis (`doas poweroff` resolves to nothing) — only the NetBSD nodes actually work with that script today. Config repo home for NetBSD-specific setup: `f3s/pi-netbsd/`. Full conversion runbook: [Bootstrap NetBSD Pi](references/bootstrap-netbsd-pi.md).
+
 Current role split:
 
-- `pi0` and `pi1` serve static `f3s.buetow.org` content behind OpenBSD `relayd` over WireGuard
+- `pi0` and `pi1` serve static `f3s.buetow.org`/`snonux.foo` content behind OpenBSD `relayd` over WireGuard. WireGuard peers are `blowfish`, `fishfinger`, **and `rocky`** (not gateway-only to just the two frontends, despite older docs here). Both nodes reboot-tested (2026-07-03): all rc.d services (`wireguard`, `bozohttpd`, `uptimed`, `npf`) and both crontabs come back automatically.
 - `pi2` and `pi3` run **Pi-hole** in Docker (`network_mode: host`, `~/pihole` on each host). Tracked dnsmasq LAN wildcard: **`f3s/pihole/docker-pi/`** in the conf repo; details in [references/pihole-pi.md](references/pihole-pi.md).
 
-### lighttpd Configuration
+### Webserver Configuration
+
+Both `pi0` and `pi1` run **bozohttpd** (built into NetBSD base, no package/config file) via a custom `/etc/rc.d/bozohttpd` (the stock rc.d/httpd script ignores `httpd_flags` entirely — never references it — so flags had to go directly in this script's own `command_args`), using `-v`/`-V` for vhosting instead of lighttpd's `$HTTP["host"]` regex match — a vhost needs a directory *literally* named after the hostname (e.g. `snonux.foo/`, with `www.snonux.foo` a symlink to it). Also needs **`-X`** (directory indexing) to match lighttpd's old `dir-listing.activate = "enable"` — without it, bare directories with no `index.html` (e.g. `/fotos/<gallery>/` at the top level) 404 instead of showing a listing; caught on `pi0` by a live redundancy test (stopped the other node's webserver, `curl`'d every page through the public domains) and included from the start on `pi1`.
 
-Config file: `/etc/lighttpd/lighttpd.conf` (managed directly on pi0/pi1, not in a config repo)
+**bozohttpd `-V` fallback bug** (hit and fixed on `pi0`, avoided on `pi1` since the fix was already baked in): `f3s.buetow.org` (the real routed hostname — relayd forwards `f3s.buetow.org`/`www.f3s.buetow.org`/`standby.f3s.buetow.org`, all Host-matched via `match request header "Host" value ... forward to <f3s_static_proxy>` in `relayd.conf` on the frontends; there is no separate `scifi.f3s.buetow.org` subdomain, `/scifi/` is just a **path** under it) had no dedicated vhost directory, so it hit the `-V` fallback — and bozohttpd's directory-without-trailing-slash redirect in that fallback path uses its own **system hostname** (e.g. `pi0.lan.buetow.org`) instead of the client's `Host:` header, unlike a vhost-*matched* request (which correctly echoes back the matched name, e.g. `snonux.foo`). Since the system hostname doesn't resolve outside the LAN, external clients following that redirect would hang. **Fixed** with self-referencing vhost symlinks so these hostnames become vhost matches instead of fallbacks: `ln -sf . /var/www/html/f3s.buetow.org`, same for `www.f3s.buetow.org` and `standby.f3s.buetow.org`.
 
-- Document root: `/var/www/html`
+- Document root: `/var/www/html` (same path both nodes now — the historical `/var/www/html/snonux` vs `/var/www/html/snonux.foo` naming difference no longer applies since `pi1` was reinstalled and now mirrors `pi0`'s tree exactly)
 - SSH access: `ssh paul@piN.lan.buetow.org -p 22`
-- Host-based virtual hosting maps domains to subdirectories:
-  - `snonux.foo` / `www.snonux.foo` → `/var/www/html/snonux`
+- Host-based virtual hosting maps domains to subdirectories: `snonux.foo` / `www.snonux.foo` → `/var/www/html/snonux.foo`
+- Content sync direction flipped mid-migration: `pi1` → `pi0` while `pi1` was still the only node with real content (Rocky), then reversed to `pi0` → `pi1` once `pi1` was also reinstalled (`pi0` is now the long-lived source of truth). `pi1`'s hourly pull is `/usr/local/bin/sync-from-pi0.sh` (cron `:47`); `pi0`'s old `sync-from-pi1.sh` (cron `:17`) is a harmless now-pointless leftover pulling from a node with no independent content anymore.
 
 **Why Host-based vhosts?** `relayd` on the OpenBSD frontends cannot rewrite URL paths. It forwards requests with the original path intact. To serve a subdirectory as root for a domain, lighttpd must remap the document root based on the `Host` header.
 
@@ -104,7 +109,7 @@ $HTTP["host"] =~ "^(www\.)?snonux\.foo$" {
 
 ## DTail (dserver)
 
-Distributed log access over SSH on port **2222** (not sshd’s 22). **pi0–pi3**: cross-build **linux/arm64** + `DTAIL_NO_ZSTD=yes`. **r0–r2** (k3s Rocky VMs): **linux/amd64** only; install as **root** over SSH; **`dtail.json` must list `root` in `Server.Permissions.Users`**; mirror **`/root/.ssh/authorized_keys`** → `/var/run/dserver/cache/root.authorized_keys` because the key-cache script only walks `/home/*`. **firewalld**: open **2222/tcp**. Rebuild clients from current **dtail** `master` if the “trust these hosts” prompt still hangs (stdout pause bug fixed upstream).
+Distributed log access over SSH on port **2222** (not sshd’s 22). **pi2–pi3**: cross-build **linux/arm64** + `DTAIL_NO_ZSTD=yes`. **pi0**/**pi1** (NetBSD) do **not** run DTail — deliberately deferred on both; would need an untested `GOOS=netbsd GOARCH=arm64` cross-build and an `rc.d` script in place of the systemd unit. **r0–r2** (k3s Rocky VMs): **linux/amd64** only; install as **root** over SSH; **`dtail.json` must list `root` in `Server.Permissions.Users`**; mirror **`/root/.ssh/authorized_keys`** → `/var/run/dserver/cache/root.authorized_keys` because the key-cache script only walks `/home/*`. **firewalld**: open **2222/tcp**. Rebuild clients from current **dtail** `master` if the “trust these hosts” prompt still hangs (stdout pause bug fixed upstream).
 
 Details: [references/dtail.md](references/dtail.md) (section **dserver on r0, r1, r2**).
 
diff --git a/prompts/skills/f3s/references/bootstrap-netbsd-pi.md b/prompts/skills/f3s/references/bootstrap-netbsd-pi.md
new file mode 100644
index 0000000..411b059
--- /dev/null
+++ b/prompts/skills/f3s/references/bootstrap-netbsd-pi.md
@@ -0,0 +1,356 @@
+# Bootstrapping a Raspberry Pi 3 f3s node on NetBSD
+
+Runbook for bringing a Rocky Pi node up to NetBSD parity, validated twice now:
+`pi0` first (2026-07-03), then `pi1` the same day using this exact procedure —
+both reboot-tested and confirmed durable. Only `pi2`/`pi3` (Pi-hole) remain
+Rocky; this is the reusable procedure for converting either of them next.
+
+**Do this one node at a time.** Never take down two of `pi0`–`pi3`'s static-HTTP
+pair (or the two Pi-hole nodes) simultaneously — one must always keep serving.
+
+## 0. Before you reinstall: back up state that doesn't come from git
+
+None of this lives in the conf repo — it only exists on the running Pi, so it
+must be pulled off **before** wiping it:
+
+- **Uptime history**: `scp` the target's `/var/spool/uptimed/records` (and
+  `.old` if present) somewhere safe (e.g. `f0.lan.buetow.org:/tmp/<host>.records`,
+  the same pattern used for `pi0`). Restoring it later preserves the uptime
+  streak instead of resetting to zero — see step 6.
+- **WireGuard identity**: not actually needed from the live host — the
+  private key and PSKs already live in `~/git/wireguardmeshgenerator/keys/`
+  (`keys/<host>/priv.key`, `keys/psk/<pair>.key`). Reuse those; do not
+  regenerate, or every peer (`blowfish`, `fishfinger`, `rocky`) needs updating.
+- **Static content**: only relevant if this node is currently the source of
+  truth for `/var/www/html`. Once `pi0` is up and holding the full synced
+  tree, `pi0` becomes the natural source for converting `pi1` next — no
+  separate backup needed, just sync FROM `pi0` afterward (see step 4).
+
+## 1. Flash/image plain NetBSD
+
+Not yet documented in this repo — `f3s/pi-netbsd/bake` and `flash/` exist as
+scaffolding in the conf repo but were empty as of 2026-07-03; the imaging
+process used for `pi0` happened outside this repo. Whatever process is used,
+the expected end state (confirmed on `pi0`) is:
+
+- NetBSD 10.1 `GENERIC64` evbarm64 (aarch64)
+- User `paul`, in group `wheel`, SSH key auth already working
+- Static LAN IP already set via `rc.conf` (`ifconfig_mue0="inet 192.168.1.12N
+  netmask 0xffffff00"`, `defaultroute="192.168.1.1"` — match the hardware
+  table's IP for this host)
+- Hostname set (`hostname="piN.lan.buetow.org"`)
+- **No** `doas`/`sudo`, **no** pkgsrc/pkgin bootstrapped — bare base install
+- `paul` has no private SSH key of its own yet (only `authorized_keys`)
+
+If any of these aren't true after flashing, do them first (they're baked into
+the disk image for `pi0`, presumably via sysinst answers or a post-install
+script — worth capturing properly in `f3s/pi-netbsd/` once that process is
+pinned down).
+
+**Note found on `pi0`**: the image already carried a **pre-baked root
+crontab** entry for the hourly goprecords upload, pointing at
+`/usr/pkg/bin/goprecords-upload-client.sh` with `GOPRECORDS_HOST` already set
+correctly. Check `doas crontab -l` early — if it's there, deploy the script to
+match that path instead of improvising a different location.
+
+## 2. Bootstrap pkgin + real doas
+
+```sh
+ssh paul@piN.lan.buetow.org
+su -                      # root password from the bake; rotate both root and
+                           # paul's passwords once doas works (step below) —
+                           # not automated, do it by hand
+export PKG_PATH=https://cdn.NetBSD.org/pub/pkgsrc/packages/NetBSD/aarch64/10.1/All/
+pkg_add -v pkgin
+pkgin -y update
+pkgin -y install doas rsync curl
+printf 'permit nopass :wheel\n' > /usr/pkg/etc/doas.conf   # NOT "permit persist" --
+                                                             # that still prompts once
+                                                             # per session, which never
+                                                             # succeeds over a
+                                                             # non-interactive SSH
+                                                             # command (no tty)
+chmod 644 /usr/pkg/etc/doas.conf
+exit   # back to paul
+doas true   # should succeed with no password prompt
+```
+
+**Why real `doas`, not the Rocky pattern**: `pi2`–`pi3` only alias `doas` to
+`sudo` via `/etc/profile.d/doas.sh`, which doesn't expand in the
+non-interactive shell an SSH command runs in — so
+`~/git/dotfiles/scripts/wol-f3s`'s `shutdown-pis`/`shutdown-all` (which runs
+`ssh paul@pi "doas poweroff"`) is silently broken on the Rocky Pis today. A
+real `doas` binary fixes it for whichever Pi gets NetBSD.
+
+**Gotcha**: commands run via `doas` get a minimal `PATH` that excludes
+`/usr/sbin` and `/usr/pkg/bin` — always use full paths (`doas
+/usr/sbin/chown`, `doas /usr/pkg/bin/wg`) or an explicit `PATH=` for cron.
+
+## 3. WireGuard (userspace — no native `wg(4)` on this platform)
+
+**`wg(4)` doesn't exist on evbarm-aarch64 10.1** — confirmed on `pi0`: the
+module is absent from all 249 files under `/stand/evbarm/10.1/modules`, so
+`ifconfig wg0 create` fails outright (`clone_command: Invalid argument`),
+despite `wg(4)` being upstream NetBSD since 9.2. Don't waste time on it;
+`wireguard-go` + `wg` (pkgsrc `wireguard-tools`, **no `wg-quick`** in this
+build) is the working path:
+
+```sh
+pkgin -y install wireguard-go wireguard-tools
+```
+
+Pull this host's existing private key and the three PSKs (`blowfish`,
+`fishfinger`, `rocky`) from `~/git/wireguardmeshgenerator/keys/` — do not
+regenerate. Also grab peer pubkeys/endpoints/AllowedIPs from
+`dist/<host>/etc/wireguard/wg0.conf` if the generator has already rendered
+them, or from another live node's `/etc/wireguard/wg0.conf` for the endpoint
+details (same for every Pi, only `AllowedIPs`/PSK differ per host).
+
+Key facts that cost time on `pi0`:
+
+- The interface **must** be named `tunN` (`wireguard-go` rejects `wg0`:
+  "Interface name must be tun[0-9]*"). Use `tun0`.
+- **Address the interface before starting `wireguard-go`**, or its read loop
+  dies immediately with `EHOSTDOWN` ("host is down") and does not retry:
+  ```sh
+  ifconfig tun0 create
+  ifconfig tun0 inet <wg-ip> <wg-ip> netmask 255.255.255.255
+  ifconfig tun0 inet6 <wg-ipv6>
+  ifconfig tun0 up
+  wireguard-go tun0     # daemonizes on its own
+  ```
+- Apply crypto config with the real `wg` CLI (supports `PersistentKeepalive`,
+  unlike the native `wgconfig` tool which has no keepalive flag at all):
+  ```sh
+  wg setconf tun0 /usr/pkg/etc/wireguard/tun0.conf
+  ```
+  `tun0.conf` is the normal `[Interface]`/`[Peer]` format — same content as
+  what `wireguardmeshgenerator` renders to `dist/<host>/etc/wireguard/wg0.conf`,
+  just handed to `wg` instead of `wg-quick`.
+- **No `wg-quick` means no automatic routes.** Each peer's AllowedIPs needs an
+  explicit host route via the local tun IP:
+  ```sh
+  route add -inet <peer-allowed-ip>/32 <local-tun4-ip> -iface
+  route add -inet6 <peer-allowed-ipv6>/128 <local-tun6-ip> -iface
+  ```
+
+Wire all of this into a custom `/etc/rc.d/wireguard` (there's no stock rc.d
+for this combination — write one, model it on `pi0`'s). Enable with
+`wireguard=YES` in `/etc/rc.conf`.
+
+**Follow-up not yet done for any host**: `wireguardmeshgenerator.rb` only
+branches on `os == 'Linux' | 'FreeBSD' | 'OpenBSD'` and always emits
+`wg-quick`-style files; the generator's YAML still lists NetBSD hosts as
+`os: Linux`. Until it gains NetBSD support, each converted Pi's WireGuard
+config is a manually-maintained exception that a future
+`--generate`/`--install` regen would clobber.
+
+## 4. Webserver — bozohttpd
+
+Built into NetBSD base, matches the "static-only, low-RAM" reasoning already
+used to justify lighttpd on the Rocky Pis. No stock rc.d exists that actually
+uses `httpd_flags` (the shipped `/etc/rc.d/httpd` computes `command_args`
+itself and never references that variable) — write a dedicated
+`/etc/rc.d/bozohttpd`:
+
+```sh
+command="/usr/libexec/httpd"
+pidfile="/var/run/bozohttpd.pid"
+command_args="-b -X -U _httpd -P ${pidfile} -v /var/www/html -V /var/www/html"
+required_dirs="/var/www/html"
+```
+
+- `-v /var/www/html -V /var/www/html`: vhost directory = same tree as the
+  default docroot. A `Host:` header matching a **literally-named**
+  subdirectory (e.g. `snonux.foo/`) is served from there; anything unmatched
+  falls back to the plain docroot via `-V`.
+- `www.snonux.foo` needs to be a symlink to `snonux.foo` (bozohttpd matches
+  the literal Host header as a directory name, not a regex like lighttpd's
+  `$HTTP["host"] =~ "^(www\.)?snonux\.foo$"`).
+- **`-X` (directory indexing) is required**, not optional — caught by a live
+  redundancy test (stopping the other node's lighttpd and curling every page):
+  bare directories with no `index.html` (e.g. a photo gallery folder under
+  `/fotos/`) 404 without it, because lighttpd has
+  `dir-listing.activate = "enable"` and bozohttpd needs the equivalent flag.
+- **Give every real routed hostname its own vhost entry, even the "default"
+  one** — don't rely on `-V` fallback for anything actually reachable from the
+  internet. `f3s.buetow.org` (checked in `relayd.conf` on the frontends: the
+  real routed names are `f3s.buetow.org`, `www.f3s.buetow.org`,
+  `standby.f3s.buetow.org` — `/scifi/` etc. are **paths** under it, not
+  separate subdomains) had no vhost dir, so it hit `-V`, and bozohttpd's
+  directory-without-trailing-slash redirect in that fallback path uses its own
+  **system hostname**, not the client's `Host:` header (unlike a real vhost
+  match, which correctly echoes back e.g. `snonux.foo`). Since the system
+  hostname (`piN.lan.buetow.org`) doesn't resolve outside the LAN, this
+  produced redirects that hang for external clients. Fix: self-referencing
+  symlinks so these become vhost matches instead of fallbacks —
+  `ln -sf . /var/www/html/f3s.buetow.org` (and the `www.`/`standby.` variants).
+
+Enable with `bozohttpd=YES` in `/etc/rc.conf`.
+
+## 5. Static content sync
+
+Direction depends on which Pi is being converted and what's already been
+migrated:
+
+- Converting the **first** Rocky Pi in a same-role pair (this was `pi0`):
+  pull from the Rocky sibling that's staying up (`pi1` for `pi0`).
+- Converting the **second** one (`pi1`, once `pi0` is already NetBSD and
+  holds the full current tree): reverse it — pull from `pi0` instead.
+
+One-time bring-up + hourly cron, same script either way (just swap source
+host):
+
+```sh
+#!/bin/sh
+set -e
+STAGE=/tmp/wwwsync-cron
+mkdir -p "$STAGE"
+rsync -a --delete -e "ssh -o StrictHostKeyChecking=accept-new" \
+	paul@<source-host>.lan.buetow.org:/var/www/html/ "$STAGE/"
+doas rsync -a --delete "$STAGE/index.html" "$STAGE/fotos" "$STAGE/scifi" /var/www/html/
+doas rsync -a --delete "$STAGE/snonux/" /var/www/html/snonux.foo/
+doas /usr/sbin/chown -R root:wheel /var/www/html
+```
+
+Needs a fresh SSH keypair for `paul` on this host (none exists on a bare
+image), authorized on the source host's `~/.ssh/authorized_keys`, plus a
+static `/etc/hosts` entry for the source hostname (Pi-to-Pi `.lan.buetow.org`
+resolution failed for both `pi0`→`pi1` and originally `pi0`→`pi1` during the
+very first Rocky bring-up — add the IP directly rather than debugging DNS).
+
+Install as `paul`'s crontab (not root's — needs the new SSH key):
+`17 * * * * /usr/local/bin/sync-from-<source>.sh >$HOME/sync-from-<source>.log 2>&1`
+
+## 6. uptimed (built from source — no prebuilt package)
+
+**No aarch64 binary package exists** in pkgsrc for `uptimed` on any branch
+checked (10.0, 10.1, 11.0, 9.4). Build from upstream instead — small C
+project, NetBSD base already has `gcc`/`make`:
+
+```sh
+pkgin -y install autoconf automake libtool pkg-config
+cd /tmp
+curl -sLO https://github.com/rpodgorny/uptimed/archive/refs/tags/v0.4.7.tar.gz
+tar xzf v0.4.7.tar.gz && cd uptimed-0.4.7
+PATH=/usr/pkg/bin:$PATH ./autogen.sh
+PATH=/usr/pkg/bin:$PATH ./configure --prefix=/usr/pkg --sysconfdir=/etc
+PATH=/usr/pkg/bin:$PATH make
+doas env PATH=/usr/pkg/bin:/usr/bin:/bin:/usr/sbin:/sbin make install
+```
+
+Installs `uptimed` to `/usr/pkg/sbin`, `uprecords` to `/usr/pkg/bin`, and uses
+`/var/spool/uptimed/records` (matches the Rocky path — this is hardcoded
+upstream, not an OS convention thing).
+
+**Before first start**, write `/etc/uptimed.conf` with `LOG_MAXIMUM_ENTRIES=0`
+(keep forever — copy the Rocky sibling's `/etc/uptimed.conf` verbatim,
+milestones included) and restore the backed-up history from step 0:
+
+```sh
+doas cp <backed-up-records-file> /var/spool/uptimed/records
+doas cp <backed-up-records-file> /var/spool/uptimed/records.old   # both, not just one
+doas /usr/sbin/chown root:wheel /var/spool/uptimed/records /var/spool/uptimed/records.old
+```
+
+**Critical bug to know about**: `read_records()` in `libuptimed/urec.c`
+unconditionally sets `useold = -1` ("no useable database found") if
+`records.old` doesn't exist yet — **regardless of whether the primary
+`records` file is valid**. Seeding only `records` and starting the daemon
+loses the imported history immediately (it gets shunted to a fresh
+`records.old` on the first periodic rewrite, then overwritten again 60s
+later). Seed **both** files with the same content before the first start.
+
+Write a custom `/etc/rc.d/uptimed` (upstream ships a Linux-init `etc/rc.uptimed`,
+not usable directly):
+
+```sh
+command="/usr/pkg/sbin/uptimed"
+pidfile="/var/run/uptimed.pid"
+command_args="-p ${pidfile}"
+```
+
+Run `uptimed -b` once (creates the boot ID), enable with `uptimed=YES`.
+
+## 7. goprecords upload
+
+```sh
+kubectl exec -n services deployment/goprecords -- \
+	goprecords --create-client-key <host> -stats-dir=/data/stats
+```
+(from a machine with cluster access — this can occasionally 502 if the
+apiserver's exec proxy can't reach whichever k3s node the pod landed on; just
+retry, it's a transient networking issue, not a token problem.)
+
+Deploy `goprecords-upload-client.sh` (from `~/git/goprecords/scripts/`,
+already POSIX/generic and already handles `/var/spool/uptimed/records` and a
+NetBSD `dmesg.boot`/`sysctl` fallback for `os.txt`/`cpuinfo.txt` — no changes
+needed) to **`/usr/pkg/bin/`** if the pre-baked crontab expects that path (see
+the note in step 1), token at `/etc/goprecords-upload.token` (`0600`),
+`GOPRECORDS_HOST=<host>`.
+
+`curl` and `uprecords` need to be resolvable via whatever `PATH` the cron
+entry sets — if it's the pre-baked one with an explicit `PATH=` line
+including `/usr/pkg/bin`, no symlinking needed; test with that exact `PATH`
+before trusting a manual test run under plain `doas` (which won't have it).
+
+## 8. Firewall — npf, not firewalld
+
+```
+$ext_if = "mue0"
+
+group "external" on $ext_if {
+	pass stateful out final all
+	pass stateful in final family inet4 proto tcp to $ext_if port 22
+	pass stateful in final family inet4 proto tcp to $ext_if port 80
+}
+
+group "wireguard" on tun0 {
+	pass stateful out final all
+	pass stateful in final family inet4 all
+	pass stateful in final family inet6 all
+}
+
+group default {
+	pass final on lo0 all
+	block all
+}
+```
+
+`family inet4`/`inet6` must be explicit on multi-family interfaces or
+`npfctl validate` fails with "address family mismatch". Sequence carefully to
+avoid locking yourself out over SSH:
+
+```sh
+doas npfctl validate           # syntax-check first
+doas npfctl reload             # loads config, does NOT enable filtering yet
+doas npfctl start              # enables filtering
+# from a FRESH ssh connection (not the one you're already in), confirm:
+#   - ssh still connects
+#   - curl http://localhost/ still works
+doas sh -c 'echo npf=YES >> /etc/rc.conf'   # only after confirming the above
+```
+
+If you get a JIT warning (`error loading the bpfjit module... Operation not
+permitted`) — harmless, just means `kern.securelevel` blocks loading that
+optional performance module; filtering still works, just slightly slower
+packet matching.
+
+## 9. Verification
+
+- `curl -fsI http://<host>.lan.buetow.org/` and the vhost via `Host:` header.
+- `wg show tun0` shows recent handshakes with `blowfish` and `fishfinger` (and
+  `rocky` if that VM happens to be up — it's often not, unrelated to this
+  work).
+- goprecords report (`https://goprecords.f3s.buetow.org/report`) picks up the
+  new host after the hourly cron fires (won't rank in the "top 20 all-time"
+  table with a fresh/short history — that's expected, not a failure).
+- **Real redundancy test** (this is what actually caught the missing `-X`
+  flag on `pi0` — don't skip it): stop the *other* node in the pair's
+  webserver entirely, then curl every real page through the **public**
+  domains (not just localhost) — root page, each vhost, and any bare
+  directory paths (e.g. `/fotos/`). Restore the other node's webserver
+  immediately after.
+- `wol-f3s shutdown-pis` (or a targeted `ssh paul@<host> "doas poweroff"`)
+  actually powers the Pi off — confirms the real-`doas` fix, but there's no
+  WoL for Pis, so only do this when you can physically power it back on.
diff --git a/prompts/skills/f3s/references/dtail.md b/prompts/skills/f3s/references/dtail.md
index 5d6a2a8..63941f1 100644
--- a/prompts/skills/f3s/references/dtail.md
+++ b/prompts/skills/f3s/references/dtail.md
@@ -8,14 +8,16 @@ Upstream install and examples live in the repo: `doc/installation.md`, `examples
 
 | Hosts | OS / arch | dserver binary | Typical SSH user |
 |-------|-----------|----------------|------------------|
-| **pi0–pi3** | Rocky Linux 9 **aarch64** (Raspberry Pi 3) | Cross-build **linux/arm64**, `nozstd` | `paul@piN.lan.buetow.org` |
+| **pi2–pi3** | Rocky Linux 9 **aarch64** (Raspberry Pi 3) | Cross-build **linux/arm64**, `nozstd` | `paul@piN.lan.buetow.org` |
 | **r0–r2** | Rocky Linux 9 **x86_64** (bhyve VMs, k3s nodes) | Cross-build **linux/amd64**, `nozstd` | Often `root@rN.lan.buetow.org` (see [Rocky Linux VMs](rocky-linux-vms.md)); add `root` (and `paul` if present) to **Server.Permissions.Users** in `dtail.json` |
 | **blowfish, fishfinger** | OpenBSD 7.8 **amd64** | Native OpenBSD package build | `rex@blowfish.buetow.org`, `rex@fishfinger.buetow.org` |
 
+`pi0`/`pi1` are **NetBSD** now (see `bootstrap-netbsd-pi.md`) and deliberately do **not** run DTail — would need an untested `GOOS=netbsd GOARCH=arm64` cross-build and an `rc.d` script.
+
 **Key cache filenames matter:** `examples/update_key_cache.sh.example` only scans `/home/*` and writes `/var/run/dserver/cache/USER.authorized_keys`. In this lab, DTail auth worked only after writing the exact cache filename for the login user:
 
 - **r0–r2**: `root.authorized_keys`
-- **pi0–pi3**: `paul.authorized_keys`
+- **pi2–pi3**: `paul.authorized_keys`
 - **blowfish, fishfinger**: `rex.authorized_keys`
 
 If clients connect as **root**, copy keys once (e.g. after install) and on key changes:
@@ -156,7 +158,7 @@ CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -tags nozstd -o dserver-linux-amd
 
 ## Installation checklist (each server)
 
-Do this on **each** target (pi0–pi3 and/or r0–r2). Adjust **user** if you are not using `paul` on the node.
+Do this on **each** target (pi2–pi3 and/or r0–r2 — not pi0/pi1, which are NetBSD and don't run DTail). Adjust **user** if you are not using `paul` on the node.
 
 1. **Binary**: `/usr/local/bin/dserver`, mode `0755`, owned by root.
 2. **OS user**: `dserver` system account (`useradd -r -d /var/lib/dserver -s /sbin/nologin -U dserver`).
@@ -184,7 +186,7 @@ Do this on **each** target (pi0–pi3 and/or r0–r2). Adjust **user** if you ar
 
 ```bash
 dcat --plain --noColor --trustAllHosts --user paul \
-  --servers pi0.lan.buetow.org,pi1.lan.buetow.org,pi2.lan.buetow.org,pi3.lan.buetow.org \
+  --servers pi2.lan.buetow.org,pi3.lan.buetow.org \
   --files /etc/fstab
 
 dcat --plain --noColor --trustAllHosts --user root \
diff --git a/prompts/skills/f3s/references/goprecords-uptimed.md b/prompts/skills/f3s/references/goprecords-uptimed.md
index 439c0fb..bf2c4ee 100644
--- a/prompts/skills/f3s/references/goprecords-uptimed.md
+++ b/prompts/skills/f3s/references/goprecords-uptimed.md
@@ -35,7 +35,8 @@ Install **`curl`** and **`uptimed`** on every client that uploads.
 |--------|--------|------------|--------|
 | OpenBSD frontends | **fishfinger**, **blowfish** | **Rex** **`goprecords_upload`** in **`~/git/conf/frontends`**; **`/etc/daily.local`** runs **`/usr/local/bin/goprecords-upload.sh`** once per **day** | Tokens in **geheim** **`secrets/etc/goprecords/<host>.token`**; template **`scripts/goprecords-upload.sh.tpl`** |
 | FreeBSD (Beelinks) | **f0**–**f3** (LAN **`192.168.1.130`–`133`**) | Manual **hourly** **root** **`cron`** calling **`goprecords-upload-client.sh`** with **`GOPRECORDS_HOST=f0`** … **`f3`** | **`/var/db/uptimed/records`**; SSH: **`fN.lan.buetow.org`** or **`192.168.1.(130+N)`** for **fN**, **`-p 22`** |
-| Raspberry Pi (Rocky) | **pi0**–**pi3** | Manual **hourly** **systemd** **timer** (see README) | **`/var/spool/uptimed/records`**; SSH: **`piN.lan.buetow.org`**, **`-p 22`** |
+| Raspberry Pi (Rocky) | **pi2**–**pi3** | Manual **hourly** **systemd** **timer** (see README) | **`/var/spool/uptimed/records`**; SSH: **`piN.lan.buetow.org`**, **`-p 22`** |
+| Raspberry Pi (NetBSD) | **pi0**–**pi1** | Manual **hourly** **root** **`cron`** (no systemd) calling **`goprecords-upload-client.sh`** with **`GOPRECORDS_HOST=pi0`**/**`pi1`** | **`/var/spool/uptimed/records`** (uptimed built from source — no prebuilt aarch64 pkgsrc package); SSH: **`piN.lan.buetow.org`**, **`-p 22`** |
 | Fedora laptop | **earth** | **user** **systemd** **`oneshot` + hourly timer** `goprecords-upload-earth.{service,timer}` | Service sets **`Environment=GOPRECORDS_HOST=earth`** and runs **`~/.local/bin/goprecords-upload-earth.sh`**; token **`~/.config/goprecords-upload-earth/token`** |
 | Mac (uptimed) → published by earth | **mega-m3-pro** (raw host `MBDVXJ4XKH9C`) | Mac drops records into the **worktime** git repo; **earth** pushes them via a **second `ExecStart`** in `goprecords-upload-earth.service` | See [Mac / mega-m3-pro via earth](#mac--mega-m3-pro-via-earth) below |
 
diff --git a/prompts/skills/f3s/references/shelly-plug.md b/prompts/skills/f3s/references/shelly-plug.md
index ebeff7b..6021921 100644
--- a/prompts/skills/f3s/references/shelly-plug.md
+++ b/prompts/skills/f3s/references/shelly-plug.md
@@ -93,7 +93,16 @@ wake via WoL when this was set up — deploy when it is back online).
 
 `wol-f3s` (dotfiles `scripts/wol-f3s`; deployed to `/home/paul/scripts/wol-f3s`
 on earth and `/usr/local/bin/wol-f3s` on pi0/pi1/pi2) controls the plug as part
-of bulk power actions:
+of bulk power actions. On `pi0`/`pi1` (NetBSD) this needed: pkgsrc `bash`
+(already present as a dependency of other packages) and pkgsrc `wol` installed,
+the shebang changed from `#!/bin/bash` to `#!/usr/pkg/bin/bash` on the deployed
+copy (dotfiles' own copy for earth/Linux stays as-is), `~/.shelly_plug` copied
+over (missing on a fresh image), and `/etc/hosts` entries for `f0`–`f3`/`pi2`–`pi3`
+(cross-Pi/host `.lan.buetow.org` resolution isn't reliable — same DNS gap
+noted elsewhere in this skill). End-to-end verified from `pi0` (2026-07-03):
+woke `f3` via WoL, toggled the shelly plug off/on, shut `f3` back down via
+`wol-f3s shutdown-f3` — all worked. Note single-host `wol-f3s f3`/`shutdown-f3`
+does **not** touch the shelly plug (only the bulk `all`/`shutdown-all` paths do).
 
 - `wol-f3s` / `all` → `shelly_set true` **before** sending WoL packets (fans on).
 - `wol-f3s shutdown-all` → `shelly_set false` **after** all hosts/Pis are down
diff --git a/prompts/skills/f3s/references/wireguard.md b/prompts/skills/f3s/references/wireguard.md
index 126b86d..d34079c 100644
--- a/prompts/skills/f3s/references/wireguard.md
+++ b/prompts/skills/f3s/references/wireguard.md
@@ -9,16 +9,27 @@ Hybrid WireGuard topology connecting the f3s infrastructure mesh, two gateway-on
 - `r0`, `r1`, `r2` — Rocky Linux Bhyve VMs
 - `blowfish`, `fishfinger` — OpenBSD internet gateways (OpenBSD Amsterdam and Hetzner)
 
-**Gateway-only peers** (connect only to gateways):
-- `pi0` — Rocky Linux 9 on Raspberry Pi 3 (`192.168.2.203`)
-- `pi1` — Rocky Linux 9 on Raspberry Pi 3 (`192.168.2.204`)
+**Limited-peer nodes** (connect to the gateways, plus `rocky` — not full mesh):
+- `pi0` — **NetBSD 10.1** on Raspberry Pi 3 (`192.168.2.203`)
+- `pi1` — **NetBSD 10.1** on Raspberry Pi 3 (`192.168.2.204`) — converted after `pi0`, same procedure, both peers up successfully first try on `pi1` since the `pi0`-derived runbook already had all the gotchas baked in
 
 **Roaming clients** (connect only to gateways):
 - `earth` — Fedora laptop (192.168.2.200)
 - `pixel7pro` — Android phone (192.168.2.201)
 
 Even `fN <-> rN` tunnels exist (technically redundant since the VM runs on the host) to keep config uniform.
-`pi0` and `pi1` are intentionally not full-mesh peers; they only establish tunnels to `blowfish` and `fishfinger`.
+`pi0` and `pi1` are not full-mesh peers; each has exactly 3 peers: `blowfish`, `fishfinger`, and `rocky` (verified against both hosts' live configs — not gateway-only as older notes here claimed).
+
+### `pi0`/`pi1` (NetBSD): no native `wg(4)`, use `wireguard-go` instead
+
+The `wg` kernel module documented above does **not** ship in the evbarm-aarch64 10.1 module set (confirmed: absent from all 249 modules under `/stand/evbarm/10.1/modules`, so `ifconfig wg0 create` fails outright) — despite `wg(4)` being upstream NetBSD since 9.2, this platform/release combination just doesn't have it. Fixed with pkgsrc's `wireguard-go` (userspace) + `wireguard-tools` (`wg` CLI only — no `wg-quick` in this package) instead:
+
+- Interface must be named `tunN` (`wireguard-go` on NetBSD requires this — `wg0` is rejected: "Interface name must be tun[0-9]*"). Used `tun0`.
+- Bring the interface up **and address it** (`ifconfig tun0 inet <ip> <ip> netmask 255.255.255.255`) *before* starting `wireguard-go`, or its read loop dies immediately with `EHOSTDOWN` ("host is down") and does not retry.
+- `wg setconf tun0 <conf>` takes the normal `[Interface]`/`[Peer]` format (including `PersistentKeepalive`, unlike native `wgconfig` which has no keepalive flag at all) — same keys/PSKs as the `wg-quick`-format file `wireguardmeshgenerator` already renders to `dist/pi0/etc/wireguard/wg0.conf`, just fed to a different tool.
+- No `wg-quick` means **no automatic routes**: each peer's AllowedIPs needs an explicit `route add -inet <ip>/32 <local-tun-ip> -iface` (and `-inet6` for the v6 ones) — `wg` only does the crypto/routing decision inside the tunnel, not the OS route table.
+- All of this is wired into a custom `/etc/rc.d/wireguard` script (there's no stock rc.d for this) since there's no native `ifconfig.wg0`/wg-quick integration to hook into.
+- Follow-up not yet done: `wireguardmeshgenerator.rb` only branches on `os == 'Linux' | 'FreeBSD' | 'OpenBSD'` and always emits `wg-quick`-style files; `wireguardmeshgenerator.yaml`'s `pi0:`/`pi1:` entries still say `os: Linux` with a `systemctl reload wg-quick@wg0.service` `reload_cmd`. Until the generator gains NetBSD support, both nodes' WireGuard configs are manually-maintained exceptions that a future `--generate`/`--install` regen would otherwise clobber.
 
 ## WireGuard IP Assignments
 
@@ -33,8 +44,8 @@ Even `fN <-> rN` tunnels exist (technically redundant since the VM runs on the h
 | r2 | 192.168.2.122 | fd42:beef:cafe:2::122 | Rocky VM (k3s node) |
 | blowfish | 192.168.2.110 | fd42:beef:cafe:2::110 | OpenBSD internet GW |
 | fishfinger | 192.168.2.111 | fd42:beef:cafe:2::111 | OpenBSD internet GW |
-| pi0 | 192.168.2.203 | fd42:beef:cafe:2::203 | Rocky Linux 9 on Raspberry Pi 3 (gateway-only peer) |
-| pi1 | 192.168.2.204 | fd42:beef:cafe:2::204 | Rocky Linux 9 on Raspberry Pi 3 (gateway-only peer) |
+| pi0 | 192.168.2.203 | fd42:beef:cafe:2::203 | NetBSD 10.1 on Raspberry Pi 3 (limited-peer: blowfish/fishfinger/rocky) |
+| pi1 | 192.168.2.204 | fd42:beef:cafe:2::204 | NetBSD 10.1 on Raspberry Pi 3 (limited-peer: blowfish/fishfinger/rocky) |
 | earth | 192.168.2.200 | fd42:beef:cafe:2::200 | Fedora laptop (roaming) |
 | pixel7pro | 192.168.2.201 | fd42:beef:cafe:2::201 | Android phone (roaming) |
 
@@ -54,7 +65,9 @@ doas service wireguard start
 doas wg show  # check public key and listen port
 ```
 
-## Rocky Linux Setup (r0, r1, r2, pi0, pi1)
+## Rocky Linux Setup (r0, r1, r2)
+
+(`pi0`/`pi1` used to follow this same setup but are now NetBSD — see "`pi0`/`pi1` (NetBSD): no native `wg(4)`" above instead.)
 
 ```sh
 dnf install -y wireguard-tools
@@ -246,7 +259,7 @@ The script generates all configs and can push them via SSH.
 
 Current mesh-specific notes:
 
-- `pi0` and `pi1` are defined as Rocky Linux hosts but excluded from all non-gateway peers, so they only tunnel to `blowfish` and `fishfinger`
+- `pi0` and `pi1` are defined in the generator's YAML as Rocky Linux hosts (now stale — both are NetBSD; the generator has no NetBSD support yet, see above) and excluded from most non-gateway peers, so they only tunnel to `blowfish`, `fishfinger`, and `rocky`
 - Installed config ownership must be OS-specific:
   - Linux: `root:root`
   - BSD: `root:wheel`
diff --git a/prompts/skills/pkgrepo/SKILL.md b/prompts/skills/pkgrepo/SKILL.md
index 5155537..fff92e8 100644
--- a/prompts/skills/pkgrepo/SKILL.md
+++ b/