diff options
| author | Paul Buetow <paul@buetow.org> | 2026-07-18 22:08:10 +0300 |
|---|---|---|
| committer | Paul Buetow <paul@buetow.org> | 2026-07-18 22:08:10 +0300 |
| commit | efbb7876347bc667bdb3633f72a22a51b3bfbc49 (patch) | |
| tree | 0a2d1d9367f4bf549584aaa71b0b82337cab2226 /prompts/skills/pkgrepo/references | |
| parent | fbf9d4815ea25bd85be0f232ac8670bda1a64955 (diff) | |
skills: rename pkgrepo -> f3s-pkgrepo and rocky-vm-setup -> f3s-rocky-vm-setup
Extend the f3s- prefix to the two remaining f3s-scoped siblings so the whole
homelab family groups together. Both are f3s-specific: pkgrepo IS
pkgrepo.f3s.buetow.org, and rocky-vm-setup documents the 'rocky' VM on the f3s
host f3.
- git mv both skill dirs (history preserved); update name: frontmatter; add the
'Part of the f3s homelab skill family' note to both descriptions.
- Rewrite every reference to the *skills*: backtick mentions (`pkgrepo` ->
`f3s-pkgrepo`, `rocky-vm-setup` -> `f3s-rocky-vm-setup`), markdown cross-links
(../pkgrepo/ -> ../f3s-pkgrepo/, etc.), the f3s hub Related-skills block and
description, and skill-maintenance's index-pattern example.
- Left references to the package REPOSITORY (pkgrepo.f3s.buetow.org, 'the custom
pkgrepo') unchanged — those name a service, not the skill.
- Verified all 114 relative links resolve. Plan doc updated to record the rename.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Diffstat (limited to 'prompts/skills/pkgrepo/references')
| -rw-r--r-- | prompts/skills/pkgrepo/references/client-setup.md | 141 | ||||
| -rw-r--r-- | prompts/skills/pkgrepo/references/dtail-package.md | 219 | ||||
| -rw-r--r-- | prompts/skills/pkgrepo/references/openbsd-build-vm.md | 42 | ||||
| -rw-r--r-- | prompts/skills/pkgrepo/references/packaging-workflow.md | 120 | ||||
| -rw-r--r-- | prompts/skills/pkgrepo/references/repo-architecture.md | 80 |
5 files changed, 0 insertions, 602 deletions
diff --git a/prompts/skills/pkgrepo/references/client-setup.md b/prompts/skills/pkgrepo/references/client-setup.md deleted file mode 100644 index 1e29c4e..0000000 --- a/prompts/skills/pkgrepo/references/client-setup.md +++ /dev/null @@ -1,141 +0,0 @@ -# Client Setup - -How to configure each OS to install packages from `pkgrepo.f3s.buetow.org`. - -## FreeBSD (f0–f3) - -Custom repo is configured alongside the official FreeBSD repos. - -File: `/usr/local/etc/pkg/repos/custom.conf` - -``` -custom: { - url: "https://pkgrepo.f3s.buetow.org/freebsd/FreeBSD:15:amd64/latest", - mirror_type: "NONE", - signature_type: "NONE", - enabled: yes -} -``` - -### Using packages - -```sh -doas pkg update -doas pkg install <package-name> -doas pkg upgrade # upgrade all, including custom packages -doas pkg install -fy <package> # force reinstall (same version) -``` - -### Setting up a new FreeBSD host - -Pipe via stdin to avoid csh quoting issues: - -```sh -cat <<'REPO' | ssh -p 22 <host>.lan.buetow.org 'doas mkdir -p /usr/local/etc/pkg/repos && doas tee /usr/local/etc/pkg/repos/custom.conf > /dev/null' -custom: { - url: "https://pkgrepo.f3s.buetow.org/freebsd/FreeBSD:15:amd64/latest", - mirror_type: "NONE", - signature_type: "NONE", - enabled: yes -} -REPO -``` - -## OpenBSD (blowfish, fishfinger) - -Custom repo is configured via `PKG_PATH` in `/root/.profile`, deployed by `rex pkgrepo_setup`. -Official OpenBSD packages still install normally via `/etc/installurl`. - -```sh -export PKG_PATH="https://pkgrepo.f3s.buetow.org/openbsd/7.8/packages/amd64/" -``` - -### Using packages - -```sh -doas pkg_add <package> # installs from custom repo (signed with signify) -doas pkg_add -u <package> # update to latest version -``` - -### Setting up a new OpenBSD host - -Two things needed: - -1. **Signify public key** — copy from an existing host: - ```sh - scp rex@fishfinger.buetow.org:/etc/signify/custom-pkg.pub /tmp/ - scp /tmp/custom-pkg.pub rex@<newhost>:/tmp/ - ssh rex@<newhost> "doas cp /tmp/custom-pkg.pub /etc/signify/custom-pkg.pub" - ``` - -2. **PKG_PATH** — run the Rex task from `~/git/conf/frontends`: - ```sh - rex pkgrepo_setup # adds PKG_PATH to /root/.profile on all frontends - ``` - -Update `PKG_PATH` whenever the OpenBSD version changes (currently 7.8). - -### Package signing - -OpenBSD packages are signed with `signify(1)` via `pkg_sign`: -- **Private key**: `/etc/signify/custom-pkg.sec` on fishfinger (build host) -- **Public key**: `/etc/signify/custom-pkg.pub` on all OpenBSD clients -- Signing happens automatically during `make pkg-openbsd` / `make pkg` -- `pkg_add` verifies the signature — no `-D unsigned` needed - -## NetBSD (pi0, pi1) - -No persistent repo config needed — install/update straight from the package URL (repo is unsigned): - -```sh -export PATH=/usr/sbin:$PATH # pkg_* live in /usr/sbin, not in the default non-interactive PATH -doas pkg_add https://pkgrepo.f3s.buetow.org/netbsd/10.1/packages/aarch64/dtail-4.3.2ng.tgz # first install -doas pkg_add -u https://pkgrepo.f3s.buetow.org/netbsd/10.1/packages/aarch64/dtail-4.3.2ng.tgz # update to newer version -``` - -Notes: -- Same-version reinstall: `pkg_add -u` skips it — `doas pkg_delete dtail` first, then `pkg_add` again -- `pkg_summary.gz` is published alongside the packages, so the repo URL could also be added to `/usr/pkg/etc/pkgin/repositories.conf` for pkgin; today plain `pkg_add` is used -- Version in the URL path must match the host OS release (currently 10.1) and the arch is `aarch64` (`uname -p`) -- pkgsrc packages from cdn.netbsd.org still install normally via pkgin — the custom repo lives under a different prefix (`/` with files in `/usr/local`, `/etc`) and doesn't conflict - -## Rocky Linux (r0–r2, pi2–pi3) - -`pi0`/`pi1` run NetBSD (see above and the `f3s-raspberry-pi` skill's `bootstrap-netbsd-pi.md`) and are not Rocky clients of this repo. - -Architecture-specific repo URLs: -- `https://pkgrepo.f3s.buetow.org/rockylinux/9/x86_64/` (r0–r2) -- `https://pkgrepo.f3s.buetow.org/rockylinux/9/aarch64/` (pi2–pi3) - -### Persistent repo file - -Create `/etc/yum.repos.d/f3s-dtail.repo`: - -```ini -[f3s-dtail] -name=f3s DTail -baseurl=https://pkgrepo.f3s.buetow.org/rockylinux/9/$basearch/ -enabled=1 -gpgcheck=0 -repo_gpgcheck=0 -``` - -Then: - -```sh -sudo dnf makecache -sudo dnf install dtail -sudo dnf upgrade dtail -``` - -### Temporary one-off usage (no persistent repo file needed) - -```sh -sudo dnf repoquery \ - --disablerepo='*' \ - --repofrompath=f3s-dtail,https://pkgrepo.f3s.buetow.org/rockylinux/9/$(uname -m)/ \ - --enablerepo=f3s-dtail \ - dtail -``` - -Do **not** pass `--repofrompath` when the persistent repo file already exists — dnf errors with "listed more than once". diff --git a/prompts/skills/pkgrepo/references/dtail-package.md b/prompts/skills/pkgrepo/references/dtail-package.md deleted file mode 100644 index fbdd8d6..0000000 --- a/prompts/skills/pkgrepo/references/dtail-package.md +++ /dev/null @@ -1,219 +0,0 @@ -# DTail Package - -DTail is a multi-binary package (6 binaries + config + service script). There are separate targets for each OS. - -## Build Commands - -```sh -cd ~/git/conf/packages -make dtail-openbsd # OpenBSD: native build on QEMU/KVM VM (CGo/zstd supported) -make dtail-freebsd # FreeBSD: cross-compiled on Linux (CGO_ENABLED=0, nozstd — .zst logs unsupported) -make dtail-netbsd # NetBSD/aarch64: cross-compiled on Linux (CGO_ENABLED=0, nozstd), packaged natively on pi0 -make dtail-rocky # Rocky Linux: x86_64 + aarch64 RPMs + repodata -``` - -## Package Contents by OS - -### OpenBSD (blowfish, fishfinger) - -| File | Source template | -|------|----------------| -| `/usr/local/bin/dserver`, `dcat`, `dgrep`, `dmap`, `dtail`, `dtailhealth` | built natively on build VM | -| `/etc/dserver/dtail.json` | `frontends/etc/dserver/dtail.json.tpl` (absolute `CacheDir: /var/run/dserver/cache`; `HostKeyFile` in persistent `/var/db/dserver/ssh_host_key`) | -| `/etc/rc.d/dserver` | `frontends/etc/rc.d/dserver.tpl` | -| `/usr/local/bin/dserver-update-key-cache.sh` | `frontends/scripts/dserver-update-key-cache.sh.tpl` (ksh) | - -OpenBSD notes: -- Packages before 2026-07-10 used relative `CacheDir: "cache"` / `HostKeyFile: "cache/ssh_host_key"` — that only worked because rc.d starts the daemon via `su -l _dserver` (CWD = `/var/run/dserver`, the `_dserver` home dir); a manual start from any other directory broke public key lookup — the same bug class dtail commit `fec2f9d` fixed on the server side (making absolute `CacheDir` paths resolve independently of dserver's CWD) -- OpenBSD's `/etc/rc` wipes `/var/run/*` at boot, so the SSH host key now lives in persistent `/var/db/dserver/ssh_host_key` (mirrors NetBSD). The rc.d `rc_pre` recreates `/var/run/dserver/cache` and `/var/db/dserver` and re-runs `dserver-update-key-cache.sh` on every start; the daily cron entry keeps it fresh afterwards -- When upgrading a host from a pre-2026-07-10 package, follow this order — neither `pkg_add` nor Rex restarts a running daemon, and once the host reboots `/etc/rc` has already wiped the old host key, making migration impossible: - 1. Migrate the host key FIRST to preserve the host identity: `doas install -d -o _dserver -m 0700 /var/db/dserver && doas cp -p /var/run/dserver/cache/ssh_host_key /var/db/dserver/` — otherwise dserver generates a new host key and clients without `--trustAllHosts` must re-accept it - 2. Reinstall the package (same-version: `pkg_add -u` is a no-op — `doas pkg_delete dtail` then `doas env PKG_PATH=... pkg_add dtail`) - 3. `doas rcctl restart dserver` — without this the running daemon keeps the old relative-path config until reboot - (done on fishfinger 2026-07-10; **blowfish still runs the pre-2026-07-10 package**) -- Since 2026-07-10 the rc.d script sets `rc_bg=YES` and calls `rc_cmd "$1"` normally — dserver does not daemonize itself, and rc.subr's `rc_bg` is the proper way to background it. Earlier packages backgrounded the whole framework (`rc_cmd $1 &`), which hid `rc_pre` failures and the start result from `rcctl` -- From the WireGuard VPN, `f0.lan.buetow.org` may not route — run the Makefile with `make dtail-openbsd FREEBSD_HOST=f0.wg0` (same override applies to the other `dtail-*` targets, which upload via f0) - -### FreeBSD (f0–f3) - -| File | Source template | -|------|----------------| -| `/usr/local/bin/dserver`, `dcat`, `dgrep`, `dmap`, `dtail`, `dtailhealth` | cross-compiled `GOOS=freebsd CGO_ENABLED=0 -tags nozstd` | -| `/usr/local/etc/dserver/dtail.json` | `frontends/etc/dserver/dtail-freebsd.json.tpl` | -| `/usr/local/etc/rc.d/dserver` | `frontends/etc/rc.d/dserver-freebsd.tpl` | -| `/usr/local/bin/dserver-update-key-cache.sh` | `frontends/scripts/dserver-update-key-cache-freebsd.sh.tpl` (sh) | - -**FreeBSD config note:** `dtail-freebsd.json.tpl` uses **absolute paths** for `CacheDir` and `HostKeyFile`. FreeBSD's `daemon(8)` resets CWD to `/`, so the relative `"cache"` in the standard template resolves to `/cache` — silently breaking key lookup. (Since dtail commit `fec2f9d`, absolute `CacheDir` paths also resolve independently of the CWD dserver was started from — before that fix, a manual service restart from a home directory broke public key auth.) - -FreeBSD notes: -- FreeBSD's `cleanvar` purges `/var/run` at boot, so the SSH host key lives in persistent `/var/db/dserver/ssh_host_key` (mirrors NetBSD/OpenBSD). The rc.d `start_precmd` recreates `/var/run/dserver/cache` and `/var/db/dserver` and re-runs `dserver-update-key-cache.sh` on every start; the daily periodic job keeps it fresh afterwards -- Packages before 2026-07-10 kept the host key in volatile `/var/run/dserver/cache/ssh_host_key` — it regenerated on every reboot. When upgrading a host from such a package, migrate the key FIRST (before restarting dserver) to preserve the host identity: - 1. `doas install -d -o dserver -m 0700 /var/db/dserver && doas cp -p /var/run/dserver/cache/ssh_host_key /var/db/dserver/ssh_host_key` - 2. `doas pkg update -f && doas pkg install -fy dtail` (same-version force reinstall) - 3. `doas service dserver restart` - (done on f0/f1/f2 2026-07-10 — sha256 of the migrated key verified identical; **f3 was unreachable and still runs the pre-2026-07-10 package**) - -### NetBSD (pi0, pi1 — aarch64) - -Package name is `dtail-4.3.2ng` — NetBSD versions must not contain dashes, so `-ng` becomes `ng`. - -| File | Source template | -|------|----------------| -| `/usr/local/bin/dserver`, `dcat`, `dgrep`, `dmap`, `dtail`, `dtailhealth` | cross-compiled `GOOS=netbsd GOARCH=arm64 CGO_ENABLED=0 -tags nozstd` | -| `/etc/dserver/dtail.json` | `frontends/etc/dserver/dtail-netbsd.json.tpl` (absolute `CacheDir` like FreeBSD; `HostKeyFile` in persistent `/var/db/dserver/ssh_host_key`) | -| `/etc/rc.d/dserver` | `frontends/etc/rc.d/dserver-netbsd.tpl` | -| `/usr/local/bin/dserver-update-key-cache.sh` | `frontends/scripts/dserver-update-key-cache-netbsd.sh.tpl` (sh) | - -NetBSD notes: -- `pkg_create` runs natively on pi0 (Makefile ships binaries + templates there via SSH and runs `packages/scripts/pkg-dtail-netbsd.sh`); `pkg_summary.gz` for pkgin is generated and uploaded alongside -- NetBSD has no `daemon(8)` and dserver doesn't daemonize — the rc.d script backgrounds it via `command_args="... &"` and runs it as user `dserver` (`dserver_user`) -- `/var/run` is volatile — the rc.d `start_precmd` recreates `/var/run/dserver/cache` and re-runs the key-cache helper on every start; a daily root cron entry (`dserver-update-key-cache.sh`) keeps it fresh -- The SSH host key lives in persistent `/var/db/dserver/ssh_host_key` (created by the precmd) so it survives reboots — since 2026-07-10 FreeBSD uses the same layout. Packages before 2026-07-09 used the volatile path; after upgrading, DTail clients without `--trustAllHosts` must re-accept the host key once -- npf firewall needs `pass stateful in final family inet4 proto tcp to $ext_if port 2222` in the `"external"` group of `/etc/npf.conf` - -### Rocky Linux (r0–r2 amd64, pi2–pi3 aarch64) - -| File | -|------| -| `/usr/local/bin/dserver`, `dcat`, `dgrep`, `dmap`, `dtail`, `dtailhealth` | -| `/usr/local/bin/dserver-update-key-cache.sh` | -| `/etc/dserver/dtail.json` | -| `/usr/lib/systemd/system/dserver.service` | -| `/usr/lib/systemd/system/dserver-update-keycache.service` | -| `/usr/lib/systemd/system/dserver-update-keycache.timer` | - -Rocky notes: -- Key-cache helper handles both `/root/.ssh/authorized_keys` and `/home/*/.ssh/authorized_keys` — `root` works on r0–r2 without manual cache copy -- `dserver.service` includes `RuntimeDirectory=dserver` and `ExecStartPre` to recreate `/var/run/dserver` (tmpfs) on Rocky -- Repo is unsigned (`gpgcheck=0`) -- `aarch64` RPM is built on pi2 — Fedora's rpmbuild refuses to emit `aarch64` binary RPMs from an x86_64 host - -## Install / Update - -### OpenBSD (via Rex) - -```sh -cd ~/git/conf/frontends -rex dtail_install # install or update from custom repo -rex dtail # full setup: install + _dserver user + daily cron + service start -``` - -### FreeBSD (manual, f0–f3) - -```sh -# Custom repo must already be configured (see client-setup.md) -doas pkg install dtail # first install -doas pkg install -fy dtail # force reinstall (same version) - -# Create service user (once per host) -doas pw useradd dserver -d /var/run/dserver -s /usr/sbin/nologin - -# Enable and start dserver -doas sysrc dserver_enable=YES -doas service dserver start - -# Key cache is populated by the rc.d start_precmd on every service start; -# run manually only when authorized_keys changed without a restart: -doas /usr/local/bin/dserver-update-key-cache.sh - -# Register daily key cache refresh -doas mkdir -p /usr/local/etc/periodic/daily -printf '%s\n%s\n' '#!/bin/sh' '/usr/local/bin/dserver-update-key-cache.sh' | \ - doas tee /usr/local/etc/periodic/daily/200.dserver-update-key-cache > /dev/null -doas chmod 755 /usr/local/etc/periodic/daily/200.dserver-update-key-cache -``` - -**FreeBSD gotchas:** -- Since the 2026-07-10 package the rc.d `start_precmd` re-runs `dserver-update-key-cache.sh` on every start, so the key cache repopulates automatically after restart or reboot (older packages required a manual re-run after any restart) -- `pkg install -fy` replaces `/usr/local/etc/dserver/dtail.json` with the package version; local customisations are lost -- Avoid inline one-liners with `||`, `!`, or multi-quote strings over SSH to FreeBSD (csh) — pipe a script to `doas /bin/sh` instead or use separate SSH commands - -### NetBSD (manual, pi0–pi1) - -```sh -# All as root via doas; pkg_* tools live in /usr/sbin (not in non-interactive SSH PATH) -export PATH=/usr/sbin:$PATH - -# Service group + user (once per host) -doas groupadd dserver -doas useradd -g dserver -d /var/run/dserver -s /sbin/nologin -c "DTail server" dserver - -# Install / update from the custom repo -doas pkg_add https://pkgrepo.f3s.buetow.org/netbsd/10.1/packages/aarch64/dtail-4.3.2ng.tgz -doas pkg_add -u https://pkgrepo.f3s.buetow.org/netbsd/10.1/packages/aarch64/dtail-4.3.2ng.tgz # newer version -# Same-version reinstall: pkg_delete dtail first, then pkg_add - -# Enable and start (rc.d script ships in the package) -doas sh -c 'echo dserver=YES >> /etc/rc.conf' # once per host -doas /etc/rc.d/dserver start - -# Open port 2222 (once per host): add to the "external" group in /etc/npf.conf: -# pass stateful in final family inet4 proto tcp to $ext_if port 2222 -# then: doas npfctl validate && doas npfctl reload - -# Daily key-cache refresh (once per host; rc.d start also refreshes it) -# root crontab entry: 30 4 * * * /usr/local/bin/dserver-update-key-cache.sh >/dev/null 2>&1 -``` - -**NetBSD gotchas:** -- The package deliberately does not create the `dserver` user/group (matching the FreeBSD package) — run the `groupadd`/`useradd` step above before the first service start or the rc.d precmd fails -- The key cache lives in volatile `/var/run` but the rc.d `start_precmd` recreates and repopulates it on every start — no manual re-run needed after restart or reboot -- `dserver -version` panics when run as root (`Not allowed to run as UID 0`) — check with `su -m dserver -c '/usr/local/bin/dserver -version'` or as a normal user - -### Rocky Linux (dnf) - -```sh -# On r0–r2 (root) or pi2–pi3 (paul with sudo): -sudo dnf upgrade dtail -sudo systemctl restart dserver -sudo systemctl start dserver-update-keycache.service # repopulate after restart -``` - -**Rocky gotcha:** A legacy `/etc/systemd/system/dserver-update-keycache.service` left from manual pre-RPM setup points to the old path `ExecStart=/var/run/dserver/update_key_cache.sh` and shadows the RPM-installed unit. Remove it on any host where the timer fails: - -```sh -sudo rm -f /etc/systemd/system/dserver-update-keycache.service -sudo systemctl daemon-reload -sudo systemctl start dserver-update-keycache.service -``` - -This was cleaned up on all r0–r2 and pi0–pi3 on 2026-04-19. - -## Client Usage from earth - -```sh -# OpenBSD frontends -dcat --plain --noColor --trustAllHosts --user rex \ - --servers blowfish.buetow.org,fishfinger.buetow.org --files /etc/fstab - -# FreeBSD hosts -dcat --plain --noColor --trustAllHosts --user paul \ - --servers f0.lan.buetow.org,f1.lan.buetow.org,f2.lan.buetow.org,f3.lan.buetow.org \ - --files /etc/fstab - -# Rocky VMs (r0–r2, user root) -dcat --plain --noColor --trustAllHosts --user root \ - --servers r0.lan.buetow.org,r1.lan.buetow.org,r2.lan.buetow.org --files /etc/fstab - -# Raspberry Pis Rocky (pi2–pi3, user paul) -dcat --plain --noColor --trustAllHosts --user paul \ - --servers pi2.lan.buetow.org,pi3.lan.buetow.org \ - --files /etc/fstab - -# Raspberry Pis NetBSD (pi0–pi1, user paul) -dcat --plain --noColor --trustAllHosts --user paul \ - --servers pi0.lan.buetow.org,pi1.lan.buetow.org \ - --files /etc/fstab -``` - -## Verification State - -| Date | Platform | Result | -|------|----------|--------| -| 2026-04-19 | FreeBSD f0–f3 | `dtail-4.3.2-ng` installed, dserver running under `daemon(8)`, `dcat /etc/fstab` ✓ (`--user paul`) | -| 2026-04-19 | OpenBSD blowfish, fishfinger | `dtail-4.3.2-ng` current, `dcat /etc/fstab` ✓ (`--user rex`) | -| 2026-04-19 | Rocky r0–r2 | `dtail-4.3.2-ng` current, dserver running, `dcat /etc/fstab` ✓ (`--user root`) | -| 2026-04-19 | Rocky pi0–pi3 | `dtail-4.3.2-ng` current, dserver running, `dcat /etc/fstab` ✓ (`--user paul`) — pi0/pi1 since re-imaged to NetBSD | -| 2026-07-09 | NetBSD pi0–pi1 | `dtail-4.3.2ng` installed (first NetBSD deployment), dserver running as `dserver` on 2222, `dcat /etc/fstab` ✓ (`--user paul`) | -| 2026-07-10 | FreeBSD f0–f2 | `dtail-4.3.2-ng` reinstalled with persistent host key (`/var/db/dserver`) — key migrated first, sha256 identical, mtime preserved; rc.d precmd repopulates key cache; `dcat /etc/fstab` ✓ (`--user paul`). **f3 unreachable (no route via wg0) — still on the pre-2026-07-10 package, migrate + upgrade when back** | -| 2026-07-10 | OpenBSD fishfinger | `dtail-4.3.2-ng` reinstalled (rc_bg rc.d + hardened key-cache script); host key sha256/mtime unchanged, `rcctl restart` reports ok, `dcat /etc/fstab` ✓ (`--user rex`). blowfish still on the older package | diff --git a/prompts/skills/pkgrepo/references/openbsd-build-vm.md b/prompts/skills/pkgrepo/references/openbsd-build-vm.md deleted file mode 100644 index 2c2831f..0000000 --- a/prompts/skills/pkgrepo/references/openbsd-build-vm.md +++ /dev/null @@ -1,42 +0,0 @@ -# OpenBSD Build VM - -A minimal OpenBSD QEMU/KVM VM on earth for native compilation of CGo packages (e.g. dtail with DataDog/zstd). Cross-compiling CGo from Linux to OpenBSD needs a C cross-compiler; native build sidesteps that entirely. - -Scripts in `~/git/conf/packages/buildvm/`. - -## Initial Setup (once) - -```sh -cd ~/git/conf/packages/buildvm - -# Fetch signify keys from fishfinger -scp rex@fishfinger.buetow.org:/etc/signify/custom-pkg.sec . -scp rex@fishfinger.buetow.org:/etc/signify/custom-pkg.pub . - -./setup.sh # downloads ISO, runs fully automated install (~5 min) -./provision.sh # installs Go, git, gmake, signify keys, SSH keys -``` - -`setup.sh` is fully automated via `install-expect.exp` — drives the OpenBSD serial console installer without manual interaction. - -## Day-to-Day Use - -```sh -make buildvm-start # boot VM (~15s) -make dtail-openbsd # auto-starts VM if needed, then builds -make buildvm-stop # shut down when done -``` - -VM specs: headless, SSH on `localhost:2222`, 1 GB RAM, 2 CPUs, 4 GB disk. -Username: `pbuild` (password: `build123`). SSH key installed by `provision.sh`. - -Source is synced via `git archive HEAD | ssh ... tar -x` — not `scp -r` — to avoid filling `/tmp` with build artifacts and test data (full repo with benchmarks exceeds the 4 GB disk). - -## Installer Notes (install-expect.exp) - -- Expect script is a **separate file** to avoid bash/expect quoting interactions — embedding it in a bash heredoc breaks password sends -- Serial console activated via `set tty com0` at the OpenBSD boot prompt -- Password prompts need `sleep 2` before `send` — `sleep 1` is not enough for the serial console -- OpenBSD 7.8 added an "Encrypt the root disk?" prompt before the partition layout -- After CONGRATULATIONS, choose shell (`s`) not reboot — the CD is still attached; configure wheel group via `chroot /mnt` -- Username `build` is rejected ("not a usable loginname") — use `pbuild` diff --git a/prompts/skills/pkgrepo/references/packaging-workflow.md b/prompts/skills/pkgrepo/references/packaging-workflow.md deleted file mode 100644 index d2b7388..0000000 --- a/prompts/skills/pkgrepo/references/packaging-workflow.md +++ /dev/null @@ -1,120 +0,0 @@ -# Packaging Workflow - -Build scripts live in `~/git/conf/packages/`. The Makefile cross-compiles Go binaries on Linux, ships them to target hosts for native packaging, and uploads to the PV. - -## Single-Binary Go Packages - -For pure Go packages (no CGo), cross-compilation from Linux works for both FreeBSD and OpenBSD. - -### Build and upload - -```sh -cd ~/git/conf/packages - -# Both FreeBSD and OpenBSD -make pkg NAME=gogios SRC=/home/paul/git/gogios \ - COMMENT="Monitoring tool with email alerts and HTML status page" \ - DESC="Gogios is a lightweight monitoring tool written in Go." - -# Single OS -make pkg-freebsd NAME=gogios SRC=/home/paul/git/gogios -make pkg-openbsd NAME=gogios SRC=/home/paul/git/gogios -``` - -### How it works - -1. Cross-compiles on Linux (`GOOS=freebsd/openbsd GOARCH=amd64`) -2. SCPs binary + packaging script to target host (f0 for FreeBSD, fishfinger for OpenBSD) -3. Runs the packaging script via SSH (`pkg create` / `pkg_create`) -4. OpenBSD packages are signed with signify automatically -5. FreeBSD repo metadata is regenerated with `pkg repo` -6. Packages are copied to the PV at `/data/nfs/k3svolumes/pkgrepo/` - -### Required Makefile variables - -| Variable | Description | Example | -|----------|-------------|---------| -| `NAME` | Package name | `gogios` | -| `SRC` | Go project root (must have `cmd/<NAME>/main.go` and `internal/version.go`) | `/home/paul/git/gogios` | - -### Optional Makefile variables - -| Variable | Default | Description | -|----------|---------|-------------| -| `COMMENT` | `$(NAME)` | One-line package description | -| `DESC` | `$(NAME)` | Longer description | -| `MAINTAINER` | `paul@buetow.org` | Maintainer email | -| `WWW` | `https://buetow.org` | Project URL | -| `ENTRY` | `cmd/$(NAME)/main.go` | Go main package path relative to SRC | - -### Version detection - -Version is read automatically from `$(SRC)/internal/version.go` — expects a `Version` constant like `const Version = "v1.4.1"`. The `v` prefix is stripped. - -## CGo Packages - -Cross-compilation from Linux fails for CGo (e.g. packages with DataDog/zstd). Use native builds instead: - -- **OpenBSD**: native build on the local QEMU/KVM build VM (see [openbsd-build-vm.md](openbsd-build-vm.md)) -- **FreeBSD**: cross-compile with `CGO_ENABLED=0 -tags nozstd` — disables zstd support but allows static cross-compile -- **NetBSD**: cross-compile with `CGO_ENABLED=0 GOOS=netbsd GOARCH=arm64 -tags nozstd`; packaging (`pkg_create`) runs natively on pi0 because `+BUILD_INFO` must match the target host -- **Rocky Linux**: built locally on earth (x86_64) and on pi2 (aarch64 via rpmbuild) - -## Manual Packaging Reference - -### FreeBSD (on f0) - -```sh -pkg create -M +MANIFEST -p plist -r stagedir -o output/All -pkg repo output/ # regenerates repo metadata -doas cp -Rf output/* /data/nfs/k3svolumes/pkgrepo/freebsd/FreeBSD:15:amd64/latest/ -``` - -### OpenBSD (on fishfinger) - -```sh -pkg_create \ - -D COMMENT="Package description" \ - -d descfile \ - -f packing-list \ - -B stagedir \ - -p / \ - output/package-name-1.0.tgz -# Copy to PV via f0 -scp package.tgz f0.lan.buetow.org:/tmp/ -ssh -p 22 f0.lan.buetow.org "doas cp /tmp/package.tgz /data/nfs/k3svolumes/pkgrepo/openbsd/7.8/packages/amd64/" -``` - -### NetBSD (on pi0) - -```sh -# pkg_* tools are in /usr/sbin (not in the non-interactive SSH PATH) -# Files staged under stagedir (-p), installed relative to / (-I); -# @owner root / @group wheel in the packing list keeps installed files root-owned. -/usr/sbin/pkg_create \ - -B build-info \ - -c commentfile \ - -d descfile \ - -f packing-list \ - -I / \ - -p stagedir \ - output/package-name-1.0.tgz -# pkg_summary.gz enables pkgin (pkg_add alone doesn't need it) -/usr/sbin/pkg_info -X output/*.tgz | gzip -9 > output/pkg_summary.gz -# Copy to PV via f0 -scp -P 22 output/* f0.lan.buetow.org:/tmp/ -ssh -p 22 f0.lan.buetow.org "doas cp /tmp/package-name-1.0.tgz /tmp/pkg_summary.gz /data/nfs/k3svolumes/pkgrepo/netbsd/10.1/packages/aarch64/" -``` - -NetBSD package versions must not contain dashes (the last dash separates the -package name from the version), so DTail's `4.3.2-ng` becomes `4.3.2ng`. - -## Install/Update on Frontends via Rex - -```sh -cd ~/git/conf/frontends -rex gogios_install # installs or updates gogios on blowfish + fishfinger (OpenBSD) and f0-f3 (FreeBSD) -rex gogios # full setup: gogios_install + config + cron -``` - -The `gogios_install` Rex task auto-detects the OS and uses `pkg install` (FreeBSD) or `pkg_add` (OpenBSD). diff --git a/prompts/skills/pkgrepo/references/repo-architecture.md b/prompts/skills/pkgrepo/references/repo-architecture.md deleted file mode 100644 index ae42e1f..0000000 --- a/prompts/skills/pkgrepo/references/repo-architecture.md +++ /dev/null @@ -1,80 +0,0 @@ -# Repo Architecture - -Custom FreeBSD, OpenBSD, NetBSD, and Rocky Linux package repository served from k3s. - -## Overview - -- **nginx pod** in k3s `infra` namespace serves static files from a PV -- URL: `https://pkgrepo.f3s.buetow.org` -- Path prefixes: `/freebsd/`, `/openbsd/`, `/netbsd/`, `/rockylinux/` -- TLS terminated by OpenBSD relayd on the internet gateways (not in the pod) -- DNS, ACME certs, httpd fallback, and relayd routing auto-generated from `@f3s_hosts` in `frontends/Rexfile` -- HTTP always redirected to HTTPS by the OpenBSD gateways — client URLs must use `https://` - -## PV Directory Structure - -``` -/data/nfs/k3svolumes/pkgrepo/ ← NFS-backed PV, physically on f0 - freebsd/ - FreeBSD:15:amd64/ - latest/ - All/ # .pkg files - packagesite.pkg # repo metadata (generated by `pkg repo`) - meta.conf / meta - data.pkg - openbsd/ - 7.8/ - packages/ - amd64/ # .tgz files (signify-signed) - netbsd/ - 10.1/ - packages/ - aarch64/ # .tgz files (unsigned) + pkg_summary.gz for pkgin - rockylinux/ - 9/ - x86_64/ # .rpm files + repodata/ - aarch64/ # .rpm files + repodata/ -``` - -## Key Config Files - -| File | Purpose | -|------|---------| -| `f3s/pkgrepo/helm-chart/` | Helm chart (nginx deployment, service, ingress, PV, configmap) | -| `f3s/argocd-apps/infra/pkgrepo.yaml` | ArgoCD Application manifest | -| `f3s/pkgrepo/test-artifacts/build-test-packages.sh` | Builds hello-test packages for FreeBSD/OpenBSD | -| `frontends/Rexfile` (`@f3s_hosts`) | DNS + routing entry for `pkgrepo.f3s.buetow.org` | -| `frontends/Rexfile` (`pkgrepo_setup`) | Adds `PKG_PATH` to root's `.profile` on OpenBSD frontends | -| `packages/Makefile` | Cross-compiles, packages, and uploads for all OSes | -| `packages/scripts/pkg-freebsd.sh` | Runs on f0 via SSH: `pkg create` + `pkg repo` + copy to PV | -| `packages/scripts/pkg-openbsd.sh` | Runs on fishfinger via SSH: `pkg_create` + signify signing | -| `packages/scripts/pkg-dtail-openbsd.sh` | DTail multi-binary OpenBSD packaging | -| `packages/scripts/pkg-dtail-freebsd.sh` | DTail multi-binary FreeBSD packaging | -| `packages/scripts/pkg-dtail-netbsd.sh` | DTail multi-binary NetBSD packaging (runs on pi0 via SSH: `pkg_create` + `pkg_summary.gz`) | -| `packages/scripts/pkg-dtail-rpm.sh` | Builds DTail RPMs from prebuilt or locally built payloads | - -## SSH Access for Package Tasks - -| Host | Access | Root | Notes | -|------|--------|------|-------| -| f0 (FreeBSD, NFS) | `ssh -p 22 f0.lan.buetow.org` | `doas` | PV is local; default shell is **csh** | -| fishfinger (OpenBSD) | `ssh rex@fishfinger.buetow.org` | `doas` | OpenBSD packages built here, then copied to f0 PV | -| blowfish (OpenBSD) | `ssh rex@blowfish.buetow.org` | `doas` | Same setup as fishfinger | -| pi0 (NetBSD) | `ssh -p 22 paul@pi0.lan.buetow.org` | `doas` | NetBSD packages assembled here (`pkg_create`), then copied to f0 PV; non-interactive SSH PATH lacks `/usr/sbin` (`pkg_*` live there) | - -## Nginx Pod Gotcha — Stale NFS Handle - -After writing new files to the PV on f0, the nginx pod can get a stale NFS file handle (error 116) and return HTTP 500. Fix with a pod restart: - -```sh -kubectl -n infra rollout restart deployment/pkgrepo -kubectl -n infra rollout status deployment/pkgrepo -``` - -## Per-OS Repo Notes - -- **FreeBSD**: version follows ABI naming (`FreeBSD:15:amd64`); packages unsigned (`signature_type: "NONE"`); always regenerate metadata with `pkg repo` after adding/removing packages -- **OpenBSD**: no repo index needed — `pkg_add` fetches by name; packages signed with signify; version in path must match host OS (currently 7.8) -- **NetBSD**: packages unsigned; `pkg_add` works with a direct package URL, `pkg_summary.gz` alongside the packages enables pkgin; package versions must not contain dashes (`4.3.2-ng` → `4.3.2ng`); `+BUILD_INFO` (`MACHINE_ARCH`/`OS_VERSION`) is checked by `pkg_add` on install, which is why packages are assembled natively on pi0 -- **Rocky Linux**: standard DNF layout; unsigned repo (`gpgcheck=0`); architecture-specific paths (`x86_64` / `aarch64`) -- **FreeBSD csh**: default shell is csh — avoid inline one-liners with `||`, `&&`, `!`, or multi-line quoting over SSH; use piped `/bin/sh` or separate SSH invocations |
