| Age | Commit message (Collapse) | Author |
|---|
|
cgit ran with cache-size=0 against scan-path over ~80 repos on NFS, so every
request reopened all of them to resolve the index age column. Measured 376ms
per index render, of which only ~50ms was CPU -- the rest was NFS wait. With
the cache enabled the same render takes ~19-29ms.
cache-root points at the existing node-local cgit-runtime emptyDir rather than
the NFS-backed /repos; caching onto NFS is what the old "permission issues"
comment was actually about. The startup script creates the directory since
cgit does not.
fcgiwrap ran as a single worker, so the 26 ArgoCD apps polling conf.git through
git-http-backend serialized against web UI renders; 4 concurrent requests
managed only ~1.5x the throughput of one. Preforking 4 workers separates them.
The cgit container also idled at 105m against a 250m limit and had accumulated
~2.9h of throttled time (nr_throttled 96979), so raise the ceiling and set the
request from measured idle usage.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
|
|
|
|
|
|
|
The quarterly backup script travels on the pool; its S3 sync needs
awscli + /root/.aws/credentials symlink to /opt/snonux/secrets/aws.credentials
on the hosting f-host. Snapshot export still works without it; only the
offsite S3 sync is skipped.
|
|
|
|
add startupProbe
- Run bridge via 'protonmail-bridge --cli' with stdin/tty enabled so the
CLI is attachable (kubectl attach) for first-time account login instead
of a separate --cli process against the gRPC daemon.
- Install libfido2-1 at container start (runtime dependency).
- Persist /root/.local (Bridge's self-updated binary) on the data PVC.
- Add a startupProbe (tcp 1143, 30x10s) so slow first starts aren't killed
by the livenessProbe; update the login/setup comments accordingly.
|
|
4-disk raidz2 USB-SATA pool used as offline backup storage, loaded ~once
per quarter. Migrated from t450 to the f3s USB-key scheme: zusb/data/enc
rekeyed from passphrase to a raw key on the F3S_KEYS sticks at
/keys/zusb.key. Scripts + key deployed to all f-hosts (f0/f1/f2/f3) so
the disk stack can be re-plugged to any host. Not auto-mounted.
|
|
Bump image.tag v2.7.5 -> v3.0.3 for both immich-server and immich-machine-learning.
Stage A already migrated Postgres to VectorChord (vchord 0.4.3), which 3.x requires
(pgvecto.rs dropped); the postgres bundle image preloads vchord.so + vectors.so per
the 3.0.1 requirement. Pre-upgrade ZFS snapshot: zdata/enc/nfsdata@immich-pre-3x-20260718 (f0).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
|
Swap the hand-rolled immich-postgres image from tensorchord/pgvecto-rs:pg16-v0.3.0
to the official Immich bundle ghcr.io/immich-app/postgres:16-vectorchord0.4.3-pgvector0.8.0-pgvectors0.3.0.
This makes vchord.so available (still preloading vectors.so) so Immich can migrate
smart-search/face vectors off the deprecated pgvecto.rs extension while still on the
2.x server, ahead of the 3.x bump (which drops pgvecto.rs entirely). Tag pins pgvecto.rs
0.3.0 to match the installed 'vectors' 0.3.0 catalog. Pre-upgrade ZFS snapshot taken:
zdata/enc/nfsdata@immich-pre-3x-20260718 on f0.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
|
|
|
The beets-art CronJob was hard-pinned to r1 via nodeSelector, and its
CPU limit was 2 (half the 4-core passively-cooled Beelink N100 on f1).
The daily noon fetchart/embedart sweep sustained ~35% CPU for ~28 min
and pushed f1 to ~95 C, tripping the FreebsdCpuTemperatureHigh alert
(>=80 C for 5m).
Two changes:
1. Cap the container CPU limit from 2 to 1. The run takes ~2x longer but
stays well under the 6h activeDeadlineSeconds and roughly halves the
sustained thermal load.
2. Unpin the CronJob from r1 so the scheduler can spread it. This is a
manifest/config change, not a storage migration: the music library is
already shared NFS (mounted at the same path on all three r-nodes).
The only thing forcing r1 was the music PV/PVC being declared RWO
(single-node mount) while shared with Navidrome, which is itself pinned
to r1 by its local-path SQLite data PVC. Flip navidrome-music PV/PVC
to ReadWriteMany so beets-art can mount it on any r-node alongside the
read-only Navidrome pod.
Also move beets-art-state from local-path (pinned to r1) to a static
NFS-backed RWX PV/PVC so the job's SQLite state is reachable from any
node. SQLite-over-NFS is acceptable here because concurrencyPolicy:
Forbid guarantees a single writer (no lock contention), and the state
is regenerable.
Application requires recreating the bound music PVC (access-mode changes
are immutable on a bound PVC) and the state PVC; see deploy notes.
|
|
- Makefile target dtail-netbsd: cross-compile netbsd/arm64 (CGO_ENABLED=0,
nozstd), package natively on pi0 with pkg_create, upload package plus
pkg_summary.gz to the PV at netbsd/10.1/packages/aarch64/ via f0
- scripts/pkg-dtail-netbsd.sh: NetBSD pkg_create packaging (prefix /,
root-owned files via @owner/@group plist directives)
- frontends: NetBSD dserver rc.d script, dtail.json (absolute cache
paths), and key-cache helper templates
- pkgrepo nginx: serve /netbsd/ with autoindex
- Fix stale ROCKY_ARM_BUILD_HOST: pi0 was re-imaged to NetBSD, aarch64
RPMs are built on pi2
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
|
|
|
|
|
|
|
|
|
Fixes the crash a vulnerability scanner triggered twice against the
live pod (std::out_of_range from a broken string::npos overflow check
in the GET/POST request-line parser).
|
|
|
|
|
|
Replaces the frameset-based chat layout with iframes + flexbox and adds
viewport meta tags across templates, so the chat is usable on phones.
|
|
Image 67babb2 used PRAGMA journal_mode=WAL, which silently lost writes
on the NFS-backed hostPath PV (WAL needs shared-memory mmap of a -shm
file that network filesystems don't support reliably) -- a registered
user could not log back in because the INSERT never actually landed.
b8faffb switches to the default rollback journal, which only needs
ordinary byte-range locks that NFS handles correctly. The stale,
partially-corrupted ychat.db/-wal/-shm files on the PV were wiped
before this rollout so the pod starts from a clean, empty database.
|
|
ychat now requires SQLite (no more no-DB build), so registered accounts
need to survive pod restarts. Adds a hostPath-backed PV/PVC for
/app/data, an NFS sentinel initContainer (same pattern as other f3s
charts), and points the image at the SQLite-mandatory build.
|
|
|
|
Adds ensure-wizard-completed initContainer after nfs-check-config. If
jellyfin.db already exists but IsStartupWizardCompleted flipped to false
(interrupted wizard run or partial config restore), restore it to true so a
working instance never drops back into the setup wizard. Fresh installs (no
jellyfin.db) keep their legitimate first-run wizard. Runs after the sentinel
check so config is only rewritten on the real NFS volume.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
fixes)
|
|
|
|
|
|
|
|
Add shellyfans rc.d service (boot-time auto-on for the rack fans on the
Shelly Plug M Gen 3 at 192.168.1.28) and a standalone shelly-plug.sh
helper to query/control the plug over its HTTP RPC API.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
|
relayd on the CARP VIP no longer terminates TLS — it forwards raw TLS to
Traefik, which terminates via the f3s-lan-tls ingress secret. The old
"export cert to relayd" steps were stale and misleading. Add the
restart-not-reload pitfall (a stale relayd process serving an expired
cached keypair caused the 2026-06-30 LAN-cert outage).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
|
|
|
|
|
|
|
In-memory guest chat, no DB. Deployment pulls
registry.lan.buetow.org:30001/ychat:f387bac, port 2000, non-root (UID 1000),
readOnlyRootFilesystem with emptyDir on /app/log. TCP-socket probes (ychat
speaks HTTP/0.9-style responses). Ingress: ychat.f3s.buetow.org +
ychat.f3s.lan.buetow.org.
|
|
After a CARP failover or f-host reboot the node host remounts NFS
cleanly but already-running pods keep their old stale bind-mount,
causing writes to return ESTALE (errno 116). Three changes to auto-heal:
1. immich-valkey: replace ping-only liveness with a write-probe
(valkey-cli ping + echo to /data/.nfs-write-probe), so a stale
/data mount fails the probe and kubelet restarts the container with
a fresh bind-mount.
2. immich-server: add label nfs.stale-restart/enabled=true so the
node-level nfs-mount-monitor can exec a write-probe into the pod
and force-delete it if writes fail. Cannot do this via Helm values —
the upstream chart hardcodes an httpGet liveness and its Sprig merge
refills httpGet even when set to null, producing an invalid
dual-handler probe rejected by the k8s API.
3. nfs-mount-monitor/check-nfs-mount.sh: add reap_stale_nfs_pods()
function (throttled to every 30 s) that handles two cases invisible
to the existing fix_mount probe — Case 1: pods stuck in
CreateContainerConfigError with a stale subPath message (git-server,
prometheus hit this); Case 2: opt-in Running+Ready pods whose
in-container write to the NFS volume fails twice in a row (the
immich-server pattern). Deployed to r0/r1/r2 via Rex.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|
|
|
|
