diff options
| author | Paul Buetow <paul@buetow.org> | 2026-07-18 22:08:10 +0300 |
|---|---|---|
| committer | Paul Buetow <paul@buetow.org> | 2026-07-18 22:08:10 +0300 |
| commit | efbb7876347bc667bdb3633f72a22a51b3bfbc49 (patch) | |
| tree | 0a2d1d9367f4bf549584aaa71b0b82337cab2226 /prompts/skills/f3s-pkgrepo/references | |
| parent | fbf9d4815ea25bd85be0f232ac8670bda1a64955 (diff) | |
skills: rename pkgrepo -> f3s-pkgrepo and rocky-vm-setup -> f3s-rocky-vm-setup
Extend the f3s- prefix to the two remaining f3s-scoped siblings so the whole
homelab family groups together. Both are f3s-specific: pkgrepo IS
pkgrepo.f3s.buetow.org, and rocky-vm-setup documents the 'rocky' VM on the f3s
host f3.
- git mv both skill dirs (history preserved); update name: frontmatter; add the
'Part of the f3s homelab skill family' note to both descriptions.
- Rewrite every reference to the *skills*: backtick mentions (`pkgrepo` ->
`f3s-pkgrepo`, `rocky-vm-setup` -> `f3s-rocky-vm-setup`), markdown cross-links
(../pkgrepo/ -> ../f3s-pkgrepo/, etc.), the f3s hub Related-skills block and
description, and skill-maintenance's index-pattern example.
- Left references to the package REPOSITORY (pkgrepo.f3s.buetow.org, 'the custom
pkgrepo') unchanged — those name a service, not the skill.
- Verified all 114 relative links resolve. Plan doc updated to record the rename.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Diffstat (limited to 'prompts/skills/f3s-pkgrepo/references')
5 files changed, 602 insertions, 0 deletions
diff --git a/prompts/skills/f3s-pkgrepo/references/client-setup.md b/prompts/skills/f3s-pkgrepo/references/client-setup.md new file mode 100644 index 0000000..1e29c4e --- /dev/null +++ b/prompts/skills/f3s-pkgrepo/references/client-setup.md @@ -0,0 +1,141 @@ +# Client Setup + +How to configure each OS to install packages from `pkgrepo.f3s.buetow.org`. + +## FreeBSD (f0–f3) + +Custom repo is configured alongside the official FreeBSD repos. + +File: `/usr/local/etc/pkg/repos/custom.conf` + +``` +custom: { + url: "https://pkgrepo.f3s.buetow.org/freebsd/FreeBSD:15:amd64/latest", + mirror_type: "NONE", + signature_type: "NONE", + enabled: yes +} +``` + +### Using packages + +```sh +doas pkg update +doas pkg install <package-name> +doas pkg upgrade # upgrade all, including custom packages +doas pkg install -fy <package> # force reinstall (same version) +``` + +### Setting up a new FreeBSD host + +Pipe via stdin to avoid csh quoting issues: + +```sh +cat <<'REPO' | ssh -p 22 <host>.lan.buetow.org 'doas mkdir -p /usr/local/etc/pkg/repos && doas tee /usr/local/etc/pkg/repos/custom.conf > /dev/null' +custom: { + url: "https://pkgrepo.f3s.buetow.org/freebsd/FreeBSD:15:amd64/latest", + mirror_type: "NONE", + signature_type: "NONE", + enabled: yes +} +REPO +``` + +## OpenBSD (blowfish, fishfinger) + +Custom repo is configured via `PKG_PATH` in `/root/.profile`, deployed by `rex pkgrepo_setup`. +Official OpenBSD packages still install normally via `/etc/installurl`. + +```sh +export PKG_PATH="https://pkgrepo.f3s.buetow.org/openbsd/7.8/packages/amd64/" +``` + +### Using packages + +```sh +doas pkg_add <package> # installs from custom repo (signed with signify) +doas pkg_add -u <package> # update to latest version +``` + +### Setting up a new OpenBSD host + +Two things needed: + +1. **Signify public key** — copy from an existing host: + ```sh + scp rex@fishfinger.buetow.org:/etc/signify/custom-pkg.pub /tmp/ + scp /tmp/custom-pkg.pub rex@<newhost>:/tmp/ + ssh rex@<newhost> "doas cp /tmp/custom-pkg.pub /etc/signify/custom-pkg.pub" + ``` + +2. **PKG_PATH** — run the Rex task from `~/git/conf/frontends`: + ```sh + rex pkgrepo_setup # adds PKG_PATH to /root/.profile on all frontends + ``` + +Update `PKG_PATH` whenever the OpenBSD version changes (currently 7.8). + +### Package signing + +OpenBSD packages are signed with `signify(1)` via `pkg_sign`: +- **Private key**: `/etc/signify/custom-pkg.sec` on fishfinger (build host) +- **Public key**: `/etc/signify/custom-pkg.pub` on all OpenBSD clients +- Signing happens automatically during `make pkg-openbsd` / `make pkg` +- `pkg_add` verifies the signature — no `-D unsigned` needed + +## NetBSD (pi0, pi1) + +No persistent repo config needed — install/update straight from the package URL (repo is unsigned): + +```sh +export PATH=/usr/sbin:$PATH # pkg_* live in /usr/sbin, not in the default non-interactive PATH +doas pkg_add https://pkgrepo.f3s.buetow.org/netbsd/10.1/packages/aarch64/dtail-4.3.2ng.tgz # first install +doas pkg_add -u https://pkgrepo.f3s.buetow.org/netbsd/10.1/packages/aarch64/dtail-4.3.2ng.tgz # update to newer version +``` + +Notes: +- Same-version reinstall: `pkg_add -u` skips it — `doas pkg_delete dtail` first, then `pkg_add` again +- `pkg_summary.gz` is published alongside the packages, so the repo URL could also be added to `/usr/pkg/etc/pkgin/repositories.conf` for pkgin; today plain `pkg_add` is used +- Version in the URL path must match the host OS release (currently 10.1) and the arch is `aarch64` (`uname -p`) +- pkgsrc packages from cdn.netbsd.org still install normally via pkgin — the custom repo lives under a different prefix (`/` with files in `/usr/local`, `/etc`) and doesn't conflict + +## Rocky Linux (r0–r2, pi2–pi3) + +`pi0`/`pi1` run NetBSD (see above and the `f3s-raspberry-pi` skill's `bootstrap-netbsd-pi.md`) and are not Rocky clients of this repo. + +Architecture-specific repo URLs: +- `https://pkgrepo.f3s.buetow.org/rockylinux/9/x86_64/` (r0–r2) +- `https://pkgrepo.f3s.buetow.org/rockylinux/9/aarch64/` (pi2–pi3) + +### Persistent repo file + +Create `/etc/yum.repos.d/f3s-dtail.repo`: + +```ini +[f3s-dtail] +name=f3s DTail +baseurl=https://pkgrepo.f3s.buetow.org/rockylinux/9/$basearch/ +enabled=1 +gpgcheck=0 +repo_gpgcheck=0 +``` + +Then: + +```sh +sudo dnf makecache +sudo dnf install dtail +sudo dnf upgrade dtail +``` + +### Temporary one-off usage (no persistent repo file needed) + +```sh +sudo dnf repoquery \ + --disablerepo='*' \ + --repofrompath=f3s-dtail,https://pkgrepo.f3s.buetow.org/rockylinux/9/$(uname -m)/ \ + --enablerepo=f3s-dtail \ + dtail +``` + +Do **not** pass `--repofrompath` when the persistent repo file already exists — dnf errors with "listed more than once". diff --git a/prompts/skills/f3s-pkgrepo/references/dtail-package.md b/prompts/skills/f3s-pkgrepo/references/dtail-package.md new file mode 100644 index 0000000..fbdd8d6 --- /dev/null +++ b/prompts/skills/f3s-pkgrepo/references/dtail-package.md @@ -0,0 +1,219 @@ +# DTail Package + +DTail is a multi-binary package (6 binaries + config + service script). There are separate targets for each OS. + +## Build Commands + +```sh +cd ~/git/conf/packages +make dtail-openbsd # OpenBSD: native build on QEMU/KVM VM (CGo/zstd supported) +make dtail-freebsd # FreeBSD: cross-compiled on Linux (CGO_ENABLED=0, nozstd — .zst logs unsupported) +make dtail-netbsd # NetBSD/aarch64: cross-compiled on Linux (CGO_ENABLED=0, nozstd), packaged natively on pi0 +make dtail-rocky # Rocky Linux: x86_64 + aarch64 RPMs + repodata +``` + +## Package Contents by OS + +### OpenBSD (blowfish, fishfinger) + +| File | Source template | +|------|----------------| +| `/usr/local/bin/dserver`, `dcat`, `dgrep`, `dmap`, `dtail`, `dtailhealth` | built natively on build VM | +| `/etc/dserver/dtail.json` | `frontends/etc/dserver/dtail.json.tpl` (absolute `CacheDir: /var/run/dserver/cache`; `HostKeyFile` in persistent `/var/db/dserver/ssh_host_key`) | +| `/etc/rc.d/dserver` | `frontends/etc/rc.d/dserver.tpl` | +| `/usr/local/bin/dserver-update-key-cache.sh` | `frontends/scripts/dserver-update-key-cache.sh.tpl` (ksh) | + +OpenBSD notes: +- Packages before 2026-07-10 used relative `CacheDir: "cache"` / `HostKeyFile: "cache/ssh_host_key"` — that only worked because rc.d starts the daemon via `su -l _dserver` (CWD = `/var/run/dserver`, the `_dserver` home dir); a manual start from any other directory broke public key lookup — the same bug class dtail commit `fec2f9d` fixed on the server side (making absolute `CacheDir` paths resolve independently of dserver's CWD) +- OpenBSD's `/etc/rc` wipes `/var/run/*` at boot, so the SSH host key now lives in persistent `/var/db/dserver/ssh_host_key` (mirrors NetBSD). The rc.d `rc_pre` recreates `/var/run/dserver/cache` and `/var/db/dserver` and re-runs `dserver-update-key-cache.sh` on every start; the daily cron entry keeps it fresh afterwards +- When upgrading a host from a pre-2026-07-10 package, follow this order — neither `pkg_add` nor Rex restarts a running daemon, and once the host reboots `/etc/rc` has already wiped the old host key, making migration impossible: + 1. Migrate the host key FIRST to preserve the host identity: `doas install -d -o _dserver -m 0700 /var/db/dserver && doas cp -p /var/run/dserver/cache/ssh_host_key /var/db/dserver/` — otherwise dserver generates a new host key and clients without `--trustAllHosts` must re-accept it + 2. Reinstall the package (same-version: `pkg_add -u` is a no-op — `doas pkg_delete dtail` then `doas env PKG_PATH=... pkg_add dtail`) + 3. `doas rcctl restart dserver` — without this the running daemon keeps the old relative-path config until reboot + (done on fishfinger 2026-07-10; **blowfish still runs the pre-2026-07-10 package**) +- Since 2026-07-10 the rc.d script sets `rc_bg=YES` and calls `rc_cmd "$1"` normally — dserver does not daemonize itself, and rc.subr's `rc_bg` is the proper way to background it. Earlier packages backgrounded the whole framework (`rc_cmd $1 &`), which hid `rc_pre` failures and the start result from `rcctl` +- From the WireGuard VPN, `f0.lan.buetow.org` may not route — run the Makefile with `make dtail-openbsd FREEBSD_HOST=f0.wg0` (same override applies to the other `dtail-*` targets, which upload via f0) + +### FreeBSD (f0–f3) + +| File | Source template | +|------|----------------| +| `/usr/local/bin/dserver`, `dcat`, `dgrep`, `dmap`, `dtail`, `dtailhealth` | cross-compiled `GOOS=freebsd CGO_ENABLED=0 -tags nozstd` | +| `/usr/local/etc/dserver/dtail.json` | `frontends/etc/dserver/dtail-freebsd.json.tpl` | +| `/usr/local/etc/rc.d/dserver` | `frontends/etc/rc.d/dserver-freebsd.tpl` | +| `/usr/local/bin/dserver-update-key-cache.sh` | `frontends/scripts/dserver-update-key-cache-freebsd.sh.tpl` (sh) | + +**FreeBSD config note:** `dtail-freebsd.json.tpl` uses **absolute paths** for `CacheDir` and `HostKeyFile`. FreeBSD's `daemon(8)` resets CWD to `/`, so the relative `"cache"` in the standard template resolves to `/cache` — silently breaking key lookup. (Since dtail commit `fec2f9d`, absolute `CacheDir` paths also resolve independently of the CWD dserver was started from — before that fix, a manual service restart from a home directory broke public key auth.) + +FreeBSD notes: +- FreeBSD's `cleanvar` purges `/var/run` at boot, so the SSH host key lives in persistent `/var/db/dserver/ssh_host_key` (mirrors NetBSD/OpenBSD). The rc.d `start_precmd` recreates `/var/run/dserver/cache` and `/var/db/dserver` and re-runs `dserver-update-key-cache.sh` on every start; the daily periodic job keeps it fresh afterwards +- Packages before 2026-07-10 kept the host key in volatile `/var/run/dserver/cache/ssh_host_key` — it regenerated on every reboot. When upgrading a host from such a package, migrate the key FIRST (before restarting dserver) to preserve the host identity: + 1. `doas install -d -o dserver -m 0700 /var/db/dserver && doas cp -p /var/run/dserver/cache/ssh_host_key /var/db/dserver/ssh_host_key` + 2. `doas pkg update -f && doas pkg install -fy dtail` (same-version force reinstall) + 3. `doas service dserver restart` + (done on f0/f1/f2 2026-07-10 — sha256 of the migrated key verified identical; **f3 was unreachable and still runs the pre-2026-07-10 package**) + +### NetBSD (pi0, pi1 — aarch64) + +Package name is `dtail-4.3.2ng` — NetBSD versions must not contain dashes, so `-ng` becomes `ng`. + +| File | Source template | +|------|----------------| +| `/usr/local/bin/dserver`, `dcat`, `dgrep`, `dmap`, `dtail`, `dtailhealth` | cross-compiled `GOOS=netbsd GOARCH=arm64 CGO_ENABLED=0 -tags nozstd` | +| `/etc/dserver/dtail.json` | `frontends/etc/dserver/dtail-netbsd.json.tpl` (absolute `CacheDir` like FreeBSD; `HostKeyFile` in persistent `/var/db/dserver/ssh_host_key`) | +| `/etc/rc.d/dserver` | `frontends/etc/rc.d/dserver-netbsd.tpl` | +| `/usr/local/bin/dserver-update-key-cache.sh` | `frontends/scripts/dserver-update-key-cache-netbsd.sh.tpl` (sh) | + +NetBSD notes: +- `pkg_create` runs natively on pi0 (Makefile ships binaries + templates there via SSH and runs `packages/scripts/pkg-dtail-netbsd.sh`); `pkg_summary.gz` for pkgin is generated and uploaded alongside +- NetBSD has no `daemon(8)` and dserver doesn't daemonize — the rc.d script backgrounds it via `command_args="... &"` and runs it as user `dserver` (`dserver_user`) +- `/var/run` is volatile — the rc.d `start_precmd` recreates `/var/run/dserver/cache` and re-runs the key-cache helper on every start; a daily root cron entry (`dserver-update-key-cache.sh`) keeps it fresh +- The SSH host key lives in persistent `/var/db/dserver/ssh_host_key` (created by the precmd) so it survives reboots — since 2026-07-10 FreeBSD uses the same layout. Packages before 2026-07-09 used the volatile path; after upgrading, DTail clients without `--trustAllHosts` must re-accept the host key once +- npf firewall needs `pass stateful in final family inet4 proto tcp to $ext_if port 2222` in the `"external"` group of `/etc/npf.conf` + +### Rocky Linux (r0–r2 amd64, pi2–pi3 aarch64) + +| File | +|------| +| `/usr/local/bin/dserver`, `dcat`, `dgrep`, `dmap`, `dtail`, `dtailhealth` | +| `/usr/local/bin/dserver-update-key-cache.sh` | +| `/etc/dserver/dtail.json` | +| `/usr/lib/systemd/system/dserver.service` | +| `/usr/lib/systemd/system/dserver-update-keycache.service` | +| `/usr/lib/systemd/system/dserver-update-keycache.timer` | + +Rocky notes: +- Key-cache helper handles both `/root/.ssh/authorized_keys` and `/home/*/.ssh/authorized_keys` — `root` works on r0–r2 without manual cache copy +- `dserver.service` includes `RuntimeDirectory=dserver` and `ExecStartPre` to recreate `/var/run/dserver` (tmpfs) on Rocky +- Repo is unsigned (`gpgcheck=0`) +- `aarch64` RPM is built on pi2 — Fedora's rpmbuild refuses to emit `aarch64` binary RPMs from an x86_64 host + +## Install / Update + +### OpenBSD (via Rex) + +```sh +cd ~/git/conf/frontends +rex dtail_install # install or update from custom repo +rex dtail # full setup: install + _dserver user + daily cron + service start +``` + +### FreeBSD (manual, f0–f3) + +```sh +# Custom repo must already be configured (see client-setup.md) +doas pkg install dtail # first install +doas pkg install -fy dtail # force reinstall (same version) + +# Create service user (once per host) +doas pw useradd dserver -d /var/run/dserver -s /usr/sbin/nologin + +# Enable and start dserver +doas sysrc dserver_enable=YES +doas service dserver start + +# Key cache is populated by the rc.d start_precmd on every service start; +# run manually only when authorized_keys changed without a restart: +doas /usr/local/bin/dserver-update-key-cache.sh + +# Register daily key cache refresh +doas mkdir -p /usr/local/etc/periodic/daily +printf '%s\n%s\n' '#!/bin/sh' '/usr/local/bin/dserver-update-key-cache.sh' | \ + doas tee /usr/local/etc/periodic/daily/200.dserver-update-key-cache > /dev/null +doas chmod 755 /usr/local/etc/periodic/daily/200.dserver-update-key-cache +``` + +**FreeBSD gotchas:** +- Since the 2026-07-10 package the rc.d `start_precmd` re-runs `dserver-update-key-cache.sh` on every start, so the key cache repopulates automatically after restart or reboot (older packages required a manual re-run after any restart) +- `pkg install -fy` replaces `/usr/local/etc/dserver/dtail.json` with the package version; local customisations are lost +- Avoid inline one-liners with `||`, `!`, or multi-quote strings over SSH to FreeBSD (csh) — pipe a script to `doas /bin/sh` instead or use separate SSH commands + +### NetBSD (manual, pi0–pi1) + +```sh +# All as root via doas; pkg_* tools live in /usr/sbin (not in non-interactive SSH PATH) +export PATH=/usr/sbin:$PATH + +# Service group + user (once per host) +doas groupadd dserver +doas useradd -g dserver -d /var/run/dserver -s /sbin/nologin -c "DTail server" dserver + +# Install / update from the custom repo +doas pkg_add https://pkgrepo.f3s.buetow.org/netbsd/10.1/packages/aarch64/dtail-4.3.2ng.tgz +doas pkg_add -u https://pkgrepo.f3s.buetow.org/netbsd/10.1/packages/aarch64/dtail-4.3.2ng.tgz # newer version +# Same-version reinstall: pkg_delete dtail first, then pkg_add + +# Enable and start (rc.d script ships in the package) +doas sh -c 'echo dserver=YES >> /etc/rc.conf' # once per host +doas /etc/rc.d/dserver start + +# Open port 2222 (once per host): add to the "external" group in /etc/npf.conf: +# pass stateful in final family inet4 proto tcp to $ext_if port 2222 +# then: doas npfctl validate && doas npfctl reload + +# Daily key-cache refresh (once per host; rc.d start also refreshes it) +# root crontab entry: 30 4 * * * /usr/local/bin/dserver-update-key-cache.sh >/dev/null 2>&1 +``` + +**NetBSD gotchas:** +- The package deliberately does not create the `dserver` user/group (matching the FreeBSD package) — run the `groupadd`/`useradd` step above before the first service start or the rc.d precmd fails +- The key cache lives in volatile `/var/run` but the rc.d `start_precmd` recreates and repopulates it on every start — no manual re-run needed after restart or reboot +- `dserver -version` panics when run as root (`Not allowed to run as UID 0`) — check with `su -m dserver -c '/usr/local/bin/dserver -version'` or as a normal user + +### Rocky Linux (dnf) + +```sh +# On r0–r2 (root) or pi2–pi3 (paul with sudo): +sudo dnf upgrade dtail +sudo systemctl restart dserver +sudo systemctl start dserver-update-keycache.service # repopulate after restart +``` + +**Rocky gotcha:** A legacy `/etc/systemd/system/dserver-update-keycache.service` left from manual pre-RPM setup points to the old path `ExecStart=/var/run/dserver/update_key_cache.sh` and shadows the RPM-installed unit. Remove it on any host where the timer fails: + +```sh +sudo rm -f /etc/systemd/system/dserver-update-keycache.service +sudo systemctl daemon-reload +sudo systemctl start dserver-update-keycache.service +``` + +This was cleaned up on all r0–r2 and pi0–pi3 on 2026-04-19. + +## Client Usage from earth + +```sh +# OpenBSD frontends +dcat --plain --noColor --trustAllHosts --user rex \ + --servers blowfish.buetow.org,fishfinger.buetow.org --files /etc/fstab + +# FreeBSD hosts +dcat --plain --noColor --trustAllHosts --user paul \ + --servers f0.lan.buetow.org,f1.lan.buetow.org,f2.lan.buetow.org,f3.lan.buetow.org \ + --files /etc/fstab + +# Rocky VMs (r0–r2, user root) +dcat --plain --noColor --trustAllHosts --user root \ + --servers r0.lan.buetow.org,r1.lan.buetow.org,r2.lan.buetow.org --files /etc/fstab + +# Raspberry Pis Rocky (pi2–pi3, user paul) +dcat --plain --noColor --trustAllHosts --user paul \ + --servers pi2.lan.buetow.org,pi3.lan.buetow.org \ + --files /etc/fstab + +# Raspberry Pis NetBSD (pi0–pi1, user paul) +dcat --plain --noColor --trustAllHosts --user paul \ + --servers pi0.lan.buetow.org,pi1.lan.buetow.org \ + --files /etc/fstab +``` + +## Verification State + +| Date | Platform | Result | +|------|----------|--------| +| 2026-04-19 | FreeBSD f0–f3 | `dtail-4.3.2-ng` installed, dserver running under `daemon(8)`, `dcat /etc/fstab` ✓ (`--user paul`) | +| 2026-04-19 | OpenBSD blowfish, fishfinger | `dtail-4.3.2-ng` current, `dcat /etc/fstab` ✓ (`--user rex`) | +| 2026-04-19 | Rocky r0–r2 | `dtail-4.3.2-ng` current, dserver running, `dcat /etc/fstab` ✓ (`--user root`) | +| 2026-04-19 | Rocky pi0–pi3 | `dtail-4.3.2-ng` current, dserver running, `dcat /etc/fstab` ✓ (`--user paul`) — pi0/pi1 since re-imaged to NetBSD | +| 2026-07-09 | NetBSD pi0–pi1 | `dtail-4.3.2ng` installed (first NetBSD deployment), dserver running as `dserver` on 2222, `dcat /etc/fstab` ✓ (`--user paul`) | +| 2026-07-10 | FreeBSD f0–f2 | `dtail-4.3.2-ng` reinstalled with persistent host key (`/var/db/dserver`) — key migrated first, sha256 identical, mtime preserved; rc.d precmd repopulates key cache; `dcat /etc/fstab` ✓ (`--user paul`). **f3 unreachable (no route via wg0) — still on the pre-2026-07-10 package, migrate + upgrade when back** | +| 2026-07-10 | OpenBSD fishfinger | `dtail-4.3.2-ng` reinstalled (rc_bg rc.d + hardened key-cache script); host key sha256/mtime unchanged, `rcctl restart` reports ok, `dcat /etc/fstab` ✓ (`--user rex`). blowfish still on the older package | diff --git a/prompts/skills/f3s-pkgrepo/references/openbsd-build-vm.md b/prompts/skills/f3s-pkgrepo/references/openbsd-build-vm.md new file mode 100644 index 0000000..2c2831f --- /dev/null +++ b/prompts/skills/f3s-pkgrepo/references/openbsd-build-vm.md @@ -0,0 +1,42 @@ +# OpenBSD Build VM + +A minimal OpenBSD QEMU/KVM VM on earth for native compilation of CGo packages (e.g. dtail with DataDog/zstd). Cross-compiling CGo from Linux to OpenBSD needs a C cross-compiler; native build sidesteps that entirely. + +Scripts in `~/git/conf/packages/buildvm/`. + +## Initial Setup (once) + +```sh +cd ~/git/conf/packages/buildvm + +# Fetch signify keys from fishfinger +scp rex@fishfinger.buetow.org:/etc/signify/custom-pkg.sec . +scp rex@fishfinger.buetow.org:/etc/signify/custom-pkg.pub . + +./setup.sh # downloads ISO, runs fully automated install (~5 min) +./provision.sh # installs Go, git, gmake, signify keys, SSH keys +``` + +`setup.sh` is fully automated via `install-expect.exp` — drives the OpenBSD serial console installer without manual interaction. + +## Day-to-Day Use + +```sh +make buildvm-start # boot VM (~15s) +make dtail-openbsd # auto-starts VM if needed, then builds +make buildvm-stop # shut down when done +``` + +VM specs: headless, SSH on `localhost:2222`, 1 GB RAM, 2 CPUs, 4 GB disk. +Username: `pbuild` (password: `build123`). SSH key installed by `provision.sh`. + +Source is synced via `git archive HEAD | ssh ... tar -x` — not `scp -r` — to avoid filling `/tmp` with build artifacts and test data (full repo with benchmarks exceeds the 4 GB disk). + +## Installer Notes (install-expect.exp) + +- Expect script is a **separate file** to avoid bash/expect quoting interactions — embedding it in a bash heredoc breaks password sends +- Serial console activated via `set tty com0` at the OpenBSD boot prompt +- Password prompts need `sleep 2` before `send` — `sleep 1` is not enough for the serial console +- OpenBSD 7.8 added an "Encrypt the root disk?" prompt before the partition layout +- After CONGRATULATIONS, choose shell (`s`) not reboot — the CD is still attached; configure wheel group via `chroot /mnt` +- Username `build` is rejected ("not a usable loginname") — use `pbuild` diff --git a/prompts/skills/f3s-pkgrepo/references/packaging-workflow.md b/prompts/skills/f3s-pkgrepo/references/packaging-workflow.md new file mode 100644 index 0000000..d2b7388 --- /dev/null +++ b/prompts/skills/f3s-pkgrepo/references/packaging-workflow.md @@ -0,0 +1,120 @@ +# Packaging Workflow + +Build scripts live in `~/git/conf/packages/`. The Makefile cross-compiles Go binaries on Linux, ships them to target hosts for native packaging, and uploads to the PV. + +## Single-Binary Go Packages + +For pure Go packages (no CGo), cross-compilation from Linux works for both FreeBSD and OpenBSD. + +### Build and upload + +```sh +cd ~/git/conf/packages + +# Both FreeBSD and OpenBSD +make pkg NAME=gogios SRC=/home/paul/git/gogios \ + COMMENT="Monitoring tool with email alerts and HTML status page" \ + DESC="Gogios is a lightweight monitoring tool written in Go." + +# Single OS +make pkg-freebsd NAME=gogios SRC=/home/paul/git/gogios +make pkg-openbsd NAME=gogios SRC=/home/paul/git/gogios +``` + +### How it works + +1. Cross-compiles on Linux (`GOOS=freebsd/openbsd GOARCH=amd64`) +2. SCPs binary + packaging script to target host (f0 for FreeBSD, fishfinger for OpenBSD) +3. Runs the packaging script via SSH (`pkg create` / `pkg_create`) +4. OpenBSD packages are signed with signify automatically +5. FreeBSD repo metadata is regenerated with `pkg repo` +6. Packages are copied to the PV at `/data/nfs/k3svolumes/pkgrepo/` + +### Required Makefile variables + +| Variable | Description | Example | +|----------|-------------|---------| +| `NAME` | Package name | `gogios` | +| `SRC` | Go project root (must have `cmd/<NAME>/main.go` and `internal/version.go`) | `/home/paul/git/gogios` | + +### Optional Makefile variables + +| Variable | Default | Description | +|----------|---------|-------------| +| `COMMENT` | `$(NAME)` | One-line package description | +| `DESC` | `$(NAME)` | Longer description | +| `MAINTAINER` | `paul@buetow.org` | Maintainer email | +| `WWW` | `https://buetow.org` | Project URL | +| `ENTRY` | `cmd/$(NAME)/main.go` | Go main package path relative to SRC | + +### Version detection + +Version is read automatically from `$(SRC)/internal/version.go` — expects a `Version` constant like `const Version = "v1.4.1"`. The `v` prefix is stripped. + +## CGo Packages + +Cross-compilation from Linux fails for CGo (e.g. packages with DataDog/zstd). Use native builds instead: + +- **OpenBSD**: native build on the local QEMU/KVM build VM (see [openbsd-build-vm.md](openbsd-build-vm.md)) +- **FreeBSD**: cross-compile with `CGO_ENABLED=0 -tags nozstd` — disables zstd support but allows static cross-compile +- **NetBSD**: cross-compile with `CGO_ENABLED=0 GOOS=netbsd GOARCH=arm64 -tags nozstd`; packaging (`pkg_create`) runs natively on pi0 because `+BUILD_INFO` must match the target host +- **Rocky Linux**: built locally on earth (x86_64) and on pi2 (aarch64 via rpmbuild) + +## Manual Packaging Reference + +### FreeBSD (on f0) + +```sh +pkg create -M +MANIFEST -p plist -r stagedir -o output/All +pkg repo output/ # regenerates repo metadata +doas cp -Rf output/* /data/nfs/k3svolumes/pkgrepo/freebsd/FreeBSD:15:amd64/latest/ +``` + +### OpenBSD (on fishfinger) + +```sh +pkg_create \ + -D COMMENT="Package description" \ + -d descfile \ + -f packing-list \ + -B stagedir \ + -p / \ + output/package-name-1.0.tgz +# Copy to PV via f0 +scp package.tgz f0.lan.buetow.org:/tmp/ +ssh -p 22 f0.lan.buetow.org "doas cp /tmp/package.tgz /data/nfs/k3svolumes/pkgrepo/openbsd/7.8/packages/amd64/" +``` + +### NetBSD (on pi0) + +```sh +# pkg_* tools are in /usr/sbin (not in the non-interactive SSH PATH) +# Files staged under stagedir (-p), installed relative to / (-I); +# @owner root / @group wheel in the packing list keeps installed files root-owned. +/usr/sbin/pkg_create \ + -B build-info \ + -c commentfile \ + -d descfile \ + -f packing-list \ + -I / \ + -p stagedir \ + output/package-name-1.0.tgz +# pkg_summary.gz enables pkgin (pkg_add alone doesn't need it) +/usr/sbin/pkg_info -X output/*.tgz | gzip -9 > output/pkg_summary.gz +# Copy to PV via f0 +scp -P 22 output/* f0.lan.buetow.org:/tmp/ +ssh -p 22 f0.lan.buetow.org "doas cp /tmp/package-name-1.0.tgz /tmp/pkg_summary.gz /data/nfs/k3svolumes/pkgrepo/netbsd/10.1/packages/aarch64/" +``` + +NetBSD package versions must not contain dashes (the last dash separates the +package name from the version), so DTail's `4.3.2-ng` becomes `4.3.2ng`. + +## Install/Update on Frontends via Rex + +```sh +cd ~/git/conf/frontends +rex gogios_install # installs or updates gogios on blowfish + fishfinger (OpenBSD) and f0-f3 (FreeBSD) +rex gogios # full setup: gogios_install + config + cron +``` + +The `gogios_install` Rex task auto-detects the OS and uses `pkg install` (FreeBSD) or `pkg_add` (OpenBSD). diff --git a/prompts/skills/f3s-pkgrepo/references/repo-architecture.md b/prompts/skills/f3s-pkgrepo/references/repo-architecture.md new file mode 100644 index 0000000..ae42e1f --- /dev/null +++ b/prompts/skills/f3s-pkgrepo/references/repo-architecture.md @@ -0,0 +1,80 @@ +# Repo Architecture + +Custom FreeBSD, OpenBSD, NetBSD, and Rocky Linux package repository served from k3s. + +## Overview + +- **nginx pod** in k3s `infra` namespace serves static files from a PV +- URL: `https://pkgrepo.f3s.buetow.org` +- Path prefixes: `/freebsd/`, `/openbsd/`, `/netbsd/`, `/rockylinux/` +- TLS terminated by OpenBSD relayd on the internet gateways (not in the pod) +- DNS, ACME certs, httpd fallback, and relayd routing auto-generated from `@f3s_hosts` in `frontends/Rexfile` +- HTTP always redirected to HTTPS by the OpenBSD gateways — client URLs must use `https://` + +## PV Directory Structure + +``` +/data/nfs/k3svolumes/pkgrepo/ ← NFS-backed PV, physically on f0 + freebsd/ + FreeBSD:15:amd64/ + latest/ + All/ # .pkg files + packagesite.pkg # repo metadata (generated by `pkg repo`) + meta.conf / meta + data.pkg + openbsd/ + 7.8/ + packages/ + amd64/ # .tgz files (signify-signed) + netbsd/ + 10.1/ + packages/ + aarch64/ # .tgz files (unsigned) + pkg_summary.gz for pkgin + rockylinux/ + 9/ + x86_64/ # .rpm files + repodata/ + aarch64/ # .rpm files + repodata/ +``` + +## Key Config Files + +| File | Purpose | +|------|---------| +| `f3s/pkgrepo/helm-chart/` | Helm chart (nginx deployment, service, ingress, PV, configmap) | +| `f3s/argocd-apps/infra/pkgrepo.yaml` | ArgoCD Application manifest | +| `f3s/pkgrepo/test-artifacts/build-test-packages.sh` | Builds hello-test packages for FreeBSD/OpenBSD | +| `frontends/Rexfile` (`@f3s_hosts`) | DNS + routing entry for `pkgrepo.f3s.buetow.org` | +| `frontends/Rexfile` (`pkgrepo_setup`) | Adds `PKG_PATH` to root's `.profile` on OpenBSD frontends | +| `packages/Makefile` | Cross-compiles, packages, and uploads for all OSes | +| `packages/scripts/pkg-freebsd.sh` | Runs on f0 via SSH: `pkg create` + `pkg repo` + copy to PV | +| `packages/scripts/pkg-openbsd.sh` | Runs on fishfinger via SSH: `pkg_create` + signify signing | +| `packages/scripts/pkg-dtail-openbsd.sh` | DTail multi-binary OpenBSD packaging | +| `packages/scripts/pkg-dtail-freebsd.sh` | DTail multi-binary FreeBSD packaging | +| `packages/scripts/pkg-dtail-netbsd.sh` | DTail multi-binary NetBSD packaging (runs on pi0 via SSH: `pkg_create` + `pkg_summary.gz`) | +| `packages/scripts/pkg-dtail-rpm.sh` | Builds DTail RPMs from prebuilt or locally built payloads | + +## SSH Access for Package Tasks + +| Host | Access | Root | Notes | +|------|--------|------|-------| +| f0 (FreeBSD, NFS) | `ssh -p 22 f0.lan.buetow.org` | `doas` | PV is local; default shell is **csh** | +| fishfinger (OpenBSD) | `ssh rex@fishfinger.buetow.org` | `doas` | OpenBSD packages built here, then copied to f0 PV | +| blowfish (OpenBSD) | `ssh rex@blowfish.buetow.org` | `doas` | Same setup as fishfinger | +| pi0 (NetBSD) | `ssh -p 22 paul@pi0.lan.buetow.org` | `doas` | NetBSD packages assembled here (`pkg_create`), then copied to f0 PV; non-interactive SSH PATH lacks `/usr/sbin` (`pkg_*` live there) | + +## Nginx Pod Gotcha — Stale NFS Handle + +After writing new files to the PV on f0, the nginx pod can get a stale NFS file handle (error 116) and return HTTP 500. Fix with a pod restart: + +```sh +kubectl -n infra rollout restart deployment/pkgrepo +kubectl -n infra rollout status deployment/pkgrepo +``` + +## Per-OS Repo Notes + +- **FreeBSD**: version follows ABI naming (`FreeBSD:15:amd64`); packages unsigned (`signature_type: "NONE"`); always regenerate metadata with `pkg repo` after adding/removing packages +- **OpenBSD**: no repo index needed — `pkg_add` fetches by name; packages signed with signify; version in path must match host OS (currently 7.8) +- **NetBSD**: packages unsigned; `pkg_add` works with a direct package URL, `pkg_summary.gz` alongside the packages enables pkgin; package versions must not contain dashes (`4.3.2-ng` → `4.3.2ng`); `+BUILD_INFO` (`MACHINE_ARCH`/`OS_VERSION`) is checked by `pkg_add` on install, which is why packages are assembled natively on pi0 +- **Rocky Linux**: standard DNF layout; unsigned repo (`gpgcheck=0`); architecture-specific paths (`x86_64` / `aarch64`) +- **FreeBSD csh**: default shell is csh — avoid inline one-liners with `||`, `&&`, `!`, or multi-line quoting over SSH; use piped `/bin/sh` or separate SSH invocations |
