summaryrefslogtreecommitdiff
path: root/frontends/etc
AgeCommit message (Collapse)Author
4 daysfrontends: restore standalone standby certificatesPaul Buetow
Amp-Thread-ID: https://ampcode.com/threads/T-019fc408-6e41-73fe-826d-d47b0408981b Co-authored-by: Amp <amp@ampcode.com>
4 daysfrontends: give f3s hosts a standby.<host> SAN instead of a separate certPaul Buetow
f3s hosts don't get a distinct standby certificate/keypair; alias the primary cert's files under the standby name (relayd selects a keypair by SNI, so it needs matching filenames) and add standby.<host> as a SAN on the primary cert. Every host now also gets www.<host> as a SAN unconditionally, since public DNS publishes www for all of them. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
5 daysgogios: Shuriken age thresholds 3w warn / 5w critPaul Buetow
5 daysgogios: fix Shuriken check template comment (single-line <%#)Paul Buetow
Rex's <%# ... %> only comments the first line; the multi-line comment was parsed as Perl (bareword 'generation'/'1 week').
5 daysgogios: add Shuriken album-age check (1w warn, 2w crit)Paul Buetow
Add a custom gogios check that monitors shuriken album freshness via dist/status.json's unix_epoch, read locally on each frontend (so it covers the whole generation+sync pipeline). warn at 1 week, crit at 2 weeks; CRIT if status.json is missing (album not published). DependsOn the site's HTTP checks. The check_shuriken_age plugin (Perl, Nagios API) lives in the shuriken.sh repo (contrib/); the frontends Rexfile gogios task installs it to /usr/local/bin/ from the sibling shuriken.sh checkout (single source of truth, no vendored copy).
5 daysshuriken: sync via rsync daemon (no SSH), gated on completed generationPaul Buetow
Per feedback: the shuriken-sync CronJob now uses the rsync daemon protocol (rsync://) instead of SSH -- no key/Secret needed. The frontends run rsyncd via inetd with hosts allow *.wg0; k3s pods run on r-nodes with .wg0 (WireGuard) connectivity, so they're authorized to push over the mesh. Add writable irregular-ninja and alt-irregular-ninja modules to frontends/etc/rsyncd.conf.tpl (deploy with rex rsync). It only publishes when a generation has COMPLETED since the last sync: shuriken now deletes dist/status.json at the start of a run and writes it last on success, so status.json presence+freshness vs a .last-sync marker on NFS is the completed/not-yet-published signal. Most ticks skip; a publish fires once after each daily generation. hostAliases pin the .wg0 frontend names to mesh IPs (cluster DNS doesn't resolve *.wg0). The nfs-check initContainer stays so rsync --delete can't wipe the live site if NFS is down. shuriken --sync over SSH stays as a manual option.
12 daysforgejo: reach git+ssh from off-LAN via a relayd TCP relay on 2022Paul Buetow
The NodePort is LAN-only, so the advertised clone URL (ssh://git@r0.lan.buetow.org:30222/...) did not work from outside. Add a plain TCP relay on the gateways forwarding 2022 to NodePort 30222, and have Forgejo advertise code.f3s.buetow.org:2022. Port 2022 rather than 22: the gateways run their own sshd on port 2, so 22 was in fact free, but leaving the forge off the default port keeps it clear of the mass scanning that port attracts. That is noise reduction and not security -- the real protection is that Forgejo's SSH does key-only auth for git operations and offers no shell. 2222 would have been the conventional choice but dserver (DTail) already holds it on the gateways; verified 2022 free on both. The relay carries no "protocol" line, so relayd forwards the stream untouched. TLS must not be involved: SSH secures its own transport and the client verifies Forgejo's host key at the far end. A dedicated <forgejo_ssh> table keeps the health check on the SSH port, since the web UI can be up while SSH is not. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
12 daysfrontends: stop relayd clobbering cache headers for every f3s servicePaul Buetow
The three cache-control response rules were unscoped, so relayd rewrote every response passing through the https relay -- not just the fallback pages the comment described. Everything the k3s cluster serves was forced to "no-cache, no-store, must-revalidate", overriding whatever the backend set. cgit, for example, marks its CSS and logo "expires 30d", but browsers re-fetched them on every page view. relayd cannot filter a response by the backend table that produced it, and a "header set" cannot be combined with a header match in the same rule. So match the Server header -- the local httpd is the only backend answering "OpenBSD httpd" -- and carry that across with a sticky tag. Verified on blowfish before rollout with an isolated relayd instance: a response from httpd:8080 gets the tagged headers, one from the cluster's nginx does not. Fallback behaviour retested after deploy: c-git.f3s.buetow.org served from httpd:8080 still returns the "Server turned off" page on any path, with all three headers intact. cgit.css now passes through max-age=2592000. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-12Fix swap and FreeBSD memory alert thresholdsPaul Buetow
2026-07-10packages+frontends: dtail packaging follow-ups (vs0 review items)Paul Buetow
- dtail-freebsd.json.tpl: move HostKeyFile from volatile /var/run/dserver/cache to persistent /var/db/dserver/ssh_host_key — FreeBSD's cleanvar purges /var/run at boot, so the host key was regenerated on every reboot (mirrors the NetBSD/OpenBSD templates) - dserver-freebsd.tpl rc.d: start_precmd now creates /var/db/dserver (0700, dserver-owned) and re-runs dserver-update-key-cache.sh on every start, so the volatile key cache repopulates right after a reboot or restart instead of waiting for the daily periodic job - packages/Makefile: .SHELLFLAGS gains -o pipefail — the "git archive | ssh ... tar" pipeline could mask a git archive failure as long as tar succeeded on the truncated stream (the only recipe-level pipeline; $(shell ...) calls are unaffected) - packages/Makefile: OS-suffixed /tmp staging names for the OpenBSD and NetBSD dtail tgz (dtail-openbsd-*/dtail-netbsd-*) — a dash-less version would have made both targets stage to the same /tmp/dtail-<version>.tgz locally and on f0; the final repo copy keeps the canonical dtail-<version>.tgz name - dserver-update-key-cache.sh.tpl (OpenBSD): port the NetBSD hardening — quote all variable expansions, derive the user via basename suffix stripping instead of cut -d. -f1 (dotted usernames broke), fix the obsolete-cachefile echo that used single quotes and never interpolated $cachefile, add a header comment documenting the rc_pre and /etc/daily.local call sites - dserver.tpl (OpenBSD rc.d): replace 'rc_cmd $1 &' with rc_bg=YES and a plain rc_cmd "$1" — the daemon needs backgrounding because it does not daemonize, but backgrounding the whole rc framework made rc_pre failures and the start result invisible to rcctl; rc_bg is rc.subr's supported way to background only the daemon - dserver-update-key-cache-freebsd.sh.tpl: header now documents the new rc.d start_precmd call site Deployed: FreeBSD pkg republished, f0/f1/f2 upgraded with host key migrated to /var/db/dserver first (sha256 verified identical); f3 unreachable, still on the old package. OpenBSD pkg republished, fishfinger reinstalled + restarted, host key unchanged, dcat verified. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10frontends: OpenBSD dserver rc.d — create /var/run/dserver parent explicitlyPaul Buetow
install -d applies -o/-m only to the final directory, so the previous single 'install -d -o _dserver /var/run/dserver/cache' left the parent /var/run/dserver (the _dserver home dir) implicitly created as root:wheel after a reboot. Create the parent explicitly with the right ownership, matching the NetBSD template (dserver-netbsd.tpl). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10frontends: OpenBSD dserver — absolute CacheDir, persistent host keyPaul Buetow
The OpenBSD dserver template used relative CacheDir "cache" and HostKeyFile "cache/ssh_host_key", which only worked because rc.d starts the daemon via su -l _dserver (CWD = /var/run/dserver). Any start from another directory broke public key lookup — same bug class dtail commit fec2f9d fixed on the server side. Switch to absolute paths like the FreeBSD/NetBSD templates. OpenBSD /etc/rc wipes /var/run/* at boot, so move the SSH host key to persistent /var/db/dserver/ssh_host_key (mirrors the NetBSD template) — a regenerated host key would break clients' known_hosts. The rc.d rc_pre now also creates /var/db/dserver and re-runs the key-cache helper on every service start, so the volatile cache is repopulated right after a reboot instead of waiting for the daily cron job. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09packages: harden NetBSD dtail pipeline (review findings)Paul Buetow
- Makefile: .SHELLFLAGS := -ec — with .ONESHELL a failing intermediate recipe line (remote pkg_create, scp, PV upload) previously did not stop the recipe and make exited 0; also remove stale /tmp package artifacts at target start so a failed build can never silently re-upload an old tgz, guard against an empty extracted version, and quote the tr operand - dtail-netbsd.json.tpl + rc.d: move the SSH host key from volatile /var/run/dserver/cache to persistent /var/db/dserver/ssh_host_key so it survives reboots (clients re-accept the key once after upgrade) - pkg-dtail-netbsd.sh: reject a missing version argument; write the pkg_summary via a temp file so a pkg_info failure cannot publish a truncated pkg_summary.gz; document that the dserver user/group is a deliberate manual install step - key-cache template: comment now states the daily cron entry is added manually at install time (documented in the pkgrepo skill) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09packages: add NetBSD dtail pipeline (pi0/pi1, aarch64)Paul Buetow
- Makefile target dtail-netbsd: cross-compile netbsd/arm64 (CGO_ENABLED=0, nozstd), package natively on pi0 with pkg_create, upload package plus pkg_summary.gz to the PV at netbsd/10.1/packages/aarch64/ via f0 - scripts/pkg-dtail-netbsd.sh: NetBSD pkg_create packaging (prefix /, root-owned files via @owner/@group plist directives) - frontends: NetBSD dserver rc.d script, dtail.json (absolute cache paths), and key-cache helper templates - pkgrepo nginx: serve /netbsd/ with autoindex - Fix stale ROCKY_ARM_BUILD_HOST: pi0 was re-imaged to NetBSD, aarch64 RPMs are built on pi2 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-08Add rocky WireGuard IP to hosts append and Rexfile wg0_ipsPaul Buetow
2026-05-16frontends + packages: add dserver/dtail support for FreeBSD and RockyPaul Buetow
Adds FreeBSD .tpl variants of the existing dserver templates and a matching pkg-dtail-freebsd.sh packaging script, plus a pkg-dtail-rpm.sh script and packages/files/dtail-rocky/ (systemd units, key-cache script, dtail.json) for the Rocky Linux dtail build.
2026-04-17frontends/relayd: route anki.f3s.buetow.org directly to NodePort 30800Paul Buetow
Bypass Traefik for anki-sync-server to fix HTTP 303 stream failures. The Anki client maps zstd response body read errors to SEE_OTHER (303). This was caused by Traefik's HTTP proxy layer interfering with the binary zstd-compressed response bodies. Route directly to the anki NodePort like Jellyfin's 30096, which avoids the double-proxy issue.
2026-04-13add goprecords.f3s.buetow.orgPaul Buetow
2026-04-10fixPaul Buetow
2026-04-10add emaPaul Buetow
2026-04-10snonux.foo: route to Pi backends at /snonux, redirect wwwPaul Buetow
- relayd: route www.snonux.foo to localhost for redirect, keep bare/standby on f3s_static_proxy - httpd: www.snonux.foo returns 302 redirect to snonux.foo - gogios: monitor pi0/pi1 via wg0.wan.buetow.org instead of lan.buetow.org - AGENTS.md: document Pi lighttpd Host-based virtual hosting pattern Amp-Thread-ID: https://ampcode.com/threads/T-019d7766-909d-741c-bcb9-1e1e931f1e1b Co-authored-by: Amp <amp@ampcode.com>
2026-04-08Add offline-page fallback for f3s static relayPaul Buetow
2026-04-08Return HTTP errors for dead f3s static backendsPaul Buetow
2026-04-08Route f3s.buetow.org to Pi static backendsPaul Buetow
2026-04-08add pi0 and pi1Paul Buetow
2026-04-08h0: document PI phase 3.2 role splitPaul Buetow
2026-04-08relayd: route garage.f3s.buetow.org to Garage S3 backends (task d)Paul Buetow
Add table <garage> (192.168.2.130-132), Host header match in f3s block, and forward to port 3900 with tcp health checks on https4/https6. Made-with: Cursor
2026-04-07dserver: replace broken newsyslog rotation with daily.local find cleanupPaul Buetow
Amp-Thread-ID: https://ampcode.com/threads/T-019d6727-d603-72c5-97a0-c1e419211767 Co-authored-by: Amp <amp@ampcode.com>
2026-03-29Add newsyslog rotation for dserver logsPaul Buetow
2026-02-08doesnt belong herePaul Buetow
2026-02-01Add tmux editor popup and deploy helper script.Paul Buetow
This enables an overlay editor binding and installs the send-keys script on frontends.
2026-01-31Fix relayd.conf syntax: replace invalid persist/timeout keywords with ↵Paul Buetow
session timeout Amp-Thread-ID: https://ampcode.com/threads/T-019c1549-6538-740f-b867-595c2dff8117 Co-authored-by: Amp <amp@ampcode.com>
2026-01-31Revert relayd routing changes - cert generation fix (acme.sh) is sufficientPaul Buetow
Amp-Thread-ID: https://ampcode.com/threads/T-019c14ac-164f-776f-8648-471d4ac4ce5a Co-authored-by: Amp <amp@ampcode.com>
2026-01-31Fix relayd template loop closing syntaxPaul Buetow
Amp-Thread-ID: https://ampcode.com/threads/T-019c14ac-164f-776f-8648-471d4ac4ce5a Co-authored-by: Amp <amp@ampcode.com>
2026-01-31Fix minvid.f3s.buetow.org TLS certificate generation and routingPaul Buetow
- acme.sh: Change grep from exact match to prefix match for f3s hosts Server blocks for f3s hosts use -port80/-port8080 suffixes - relayd: Add explicit routing for f3s hosts (except registry/jellyfin) minvid and other f3s hosts now route to <f3s> backend table Amp-Thread-ID: https://ampcode.com/threads/T-019c14ac-164f-776f-8648-471d4ac4ce5a Co-authored-by: Amp <amp@ampcode.com>
2026-01-31Add MinVid Docker image build filesPaul Buetow
Amp-Thread-ID: https://ampcode.com/threads/T-019c1492-bec0-70f8-8d02-ef3596a7228b Co-authored-by: Amp <amp@ampcode.com>
2026-01-31fix: don't add www prefix to f3s hosts in ACME configPaul Buetow
F3S hosts don't have DNS records for www variants, causing ACME validation to fail. Only non-f3s hosts get www.hostname as alternative name in cert. Fixes: audiobookshelf.f3s.buetow.org and other f3s hosts unable to obtain valid TLS certs Amp-Thread-ID: https://ampcode.com/threads/T-019c1441-7e3a-77cd-890f-cd31df998a31 Co-authored-by: Amp <amp@ampcode.com>
2026-01-31fix: relayd template and missing ipv4address variablePaul Buetow
- Fixed Perl template syntax error in relayd.conf.tpl (lines 68-77) with improperly nested if/elsif blocks - Added ipv4address subroutine to Rexfile to lookup IPv4 by hostname - Updated relayd template to use ipv4address->($hostname) instead of undefined $vio0_ip variable - Pass ipv4address to template engine in relayd task Amp-Thread-ID: https://ampcode.com/threads/T-019c1441-7e3a-77cd-890f-cd31df998a31 Co-authored-by: Amp <amp@ampcode.com>
2026-01-31Fix relayd syntax: use valid timeout session instead of http-specific timeoutsPaul Buetow
Amp-Thread-ID: https://ampcode.com/threads/T-019c1392-78a3-7440-9d95-1a1b2fd7dfec Co-authored-by: Amp <amp@ampcode.com>
2026-01-31Increase relayd timeouts for better video streaming bufferingPaul Buetow
Amp-Thread-ID: https://ampcode.com/threads/T-019c1392-78a3-7440-9d95-1a1b2fd7dfec Co-authored-by: Amp <amp@ampcode.com>
2026-01-31Revert invalid timeout directives in relayd http protocol blockPaul Buetow
Amp-Thread-ID: https://ampcode.com/threads/T-019c1392-78a3-7440-9d95-1a1b2fd7dfec Co-authored-by: Amp <amp@ampcode.com>
2026-01-31Fix relayd syntax: move persist to relay level, not protocol levelPaul Buetow
Amp-Thread-ID: https://ampcode.com/threads/T-019c1392-78a3-7440-9d95-1a1b2fd7dfec Co-authored-by: Amp <amp@ampcode.com>
2026-01-31Add persistent connections and keepalive to relayd https protocolPaul Buetow
Amp-Thread-ID: https://ampcode.com/threads/T-019c1392-78a3-7440-9d95-1a1b2fd7dfec Co-authored-by: Amp <amp@ampcode.com>
2026-01-31Revert "Enable native Jellyfin ports 8096/8920 relays for better performance"Paul Buetow
This reverts commit 0818e6d7d06c5026dd33a6313480a21b1c555ae8.
2026-01-31Enable native Jellyfin ports 8096/8920 relays for better performancePaul Buetow
Amp-Thread-ID: https://ampcode.com/threads/T-019c1392-78a3-7440-9d95-1a1b2fd7dfec Co-authored-by: Amp <amp@ampcode.com>
2026-01-31Fix Jellyfin slow database: set SQLite locking mode to OptimisticPaul Buetow
Amp-Thread-ID: https://ampcode.com/threads/T-019c1392-78a3-7440-9d95-1a1b2fd7dfec Co-authored-by: Amp <amp@ampcode.com>
2026-01-31fix(relayd): remove X-Forwarded-Host with unexpanded variablePaul Buetow
Amp-Thread-ID: https://ampcode.com/threads/T-019c12b1-e861-773b-8f74-64b6c2255a5f Co-authored-by: Amp <amp@ampcode.com>
2026-01-31fix(relayd): add X-Forwarded-Host header for Jellyfin reverse proxyPaul Buetow
Amp-Thread-ID: https://ampcode.com/threads/T-019c12b1-e861-773b-8f74-64b6c2255a5f Co-authored-by: Amp <amp@ampcode.com>
2026-01-31config(relayd): add Jellyfin ports 8096 and 8920 for Android app discoveryPaul Buetow
Amp-Thread-ID: https://ampcode.com/threads/T-019c12b1-e861-773b-8f74-64b6c2255a5f Co-authored-by: Amp <amp@ampcode.com>
2026-01-31config(relayd): add Jellyfin NodePort routingPaul Buetow
Amp-Thread-ID: https://ampcode.com/threads/T-019c12b1-e861-773b-8f74-64b6c2255a5f Co-authored-by: Amp <amp@ampcode.com>