summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2026-04-08b0: document PI phase 0.2 hostname verificationPaul Buetow
2026-04-08a0: record PI Phase 0.1 baselinePaul Buetow
2026-04-08feat(f3s): deploy Trivy Operator for image CVE scanning (task h)Paul Buetow
- ArgoCD app: aquasecurity/trivy-operator in monitoring with ServiceMonitor - PrometheusRule for Critical/High trivy_image_vulnerabilities alerts - Alertmanager route/receiver for component=trivy (UI; webhook TBD) Made-with: Cursor
2026-04-08garage: bind S3 and admin endpoints on IPv4Paul Buetow
Ensure Garage listens on WireGuard IPv4 addresses so relay hosts can reach node S3/admin ports reliably. Made-with: Cursor
2026-04-08f3s/prometheus: add Garage admin scrape targets (task f)Paul Buetow
Add job_name garage for 192.168.2.130-132:3903 with os=freebsd label. Mirror config in additional-scrape-configs-secret for kube apply/ArgoCD. Made-with: Cursor
2026-04-08garage: Garage 2.2 TOML schema and deploy permissionsPaul Buetow
Align etc/garage.f*.toml with garage-2.2.0 (metadata_dir, data_dir, rpc_secret, rpc_bind_addr, rpc_public_addr per host, s3_api/admin, replication_factor). Bind RPC on 0.0.0.0:3901 so IPv4 LAN peers can reach nodes on FreeBSD. Install config as root:garage 640 so the rc.d garage user can read garage.toml. Made-with: Cursor
2026-04-08task b: add f3s/garage deploy layout (Rex, Just, host TOMLs)Paul Buetow
- Per-host garage.fN.toml with __RPC_SECRET__ placeholder and 23G data cap - Rexfile: paul@f0–f2, inject secret from secrets/rpc_secret, install to /usr/local/etc/garage.toml, restart garage - Justfile: deploy, init-secrets, status, layout, stats, bucket-create - Root Rexfile: require f3s/garage/Rexfile (not matched by */Rexfile glob) - Gitignore f3s/garage/secrets/ Made-with: Cursor
2026-04-08relayd: route garage.f3s.buetow.org to Garage S3 backends (task d)Paul Buetow
Add table <garage> (192.168.2.130-132), Host header match in f3s block, and forward to port 3900 with tcp health checks on https4/https6. Made-with: Cursor
2026-04-08frontends: add garage.f3s.buetow.org to @f3s_hosts (task 8)Paul Buetow
Include garage in f3s host list so DNS, TLS (acme), and httpd/relayd templates generate config for the new hostname. Made-with: Cursor
2026-04-07dserver: replace broken newsyslog rotation with daily.local find cleanupPaul Buetow
Amp-Thread-ID: https://ampcode.com/threads/T-019d6727-d603-72c5-97a0-c1e419211767 Co-authored-by: Amp <amp@ampcode.com>
2026-04-06immich: fix chart value structure - image tag under server/ml controllers, ↵Paul Buetow
remove duplicate controllers.server Amp-Thread-ID: https://ampcode.com/threads/T-019d6154-8fdf-74fe-b865-f796d8a4214a Co-authored-by: Amp <amp@ampcode.com>
2026-04-06immich: fix ML config to use chart's machine-learning.controllers.main structurePaul Buetow
Amp-Thread-ID: https://ampcode.com/threads/T-019d6154-8fdf-74fe-b865-f796d8a4214a Co-authored-by: Amp <amp@ampcode.com>
2026-04-06immich: tune ML throughput - add postgres anti-affinity, increase intra-op ↵Paul Buetow
threads, increase worker timeout Amp-Thread-ID: https://ampcode.com/threads/T-019d6154-8fdf-74fe-b865-f796d8a4214a Co-authored-by: Amp <amp@ampcode.com>
2026-04-05immich: relax postgres probes and add resource limitsPaul Buetow
- Increase liveness probe tolerance (60s delay, 30s period, 10s timeout, 6 failures) - Increase readiness probe tolerance (15s delay, 10s period, 5s timeout, 6 failures) - Add resource requests (100m CPU, 512Mi RAM) and limits (2Gi RAM) - Fixes crash loop caused by probe killing postgres during recovery Amp-Thread-ID: https://ampcode.com/threads/T-019d5f54-27f2-740c-ac41-0f980e7aecd3 Co-authored-by: Amp <amp@ampcode.com>
2026-04-04fix(immich): use dual-style values for resources and affinity to ensure they ↵Paul Buetow
apply
2026-04-04fix(immich): use correctly nested controllers structure for affinity and 4Gi ↵Paul Buetow
resources
2026-04-04fix(immich): increase memory limits to 4Gi to avoid OOMKilled for MLPaul Buetow
2026-04-04feat(immich): add preferred anti-affinity and resources to balance loadPaul Buetow
2026-04-01immich: separate PVs for videos RO/RW to avoid dual-PVC mount issuePaul Buetow
Amp-Thread-ID: https://ampcode.com/threads/T-019d47a3-2deb-75c3-8a75-b0f39006a35d Co-authored-by: Amp <amp@ampcode.com>
2026-04-01immich: per-user external library mounts with RO/RW separationPaul Buetow
Amp-Thread-ID: https://ampcode.com/threads/T-019d47a3-2deb-75c3-8a75-b0f39006a35d Co-authored-by: Amp <amp@ampcode.com>
2026-04-01immich: use bjw-s persistence for external library mountPaul Buetow
Amp-Thread-ID: https://ampcode.com/threads/T-019d47a3-2deb-75c3-8a75-b0f39006a35d Co-authored-by: Amp <amp@ampcode.com>
2026-04-01immich: replace yoga videos with general external library mountPaul Buetow
Amp-Thread-ID: https://ampcode.com/threads/T-019d47a3-2deb-75c3-8a75-b0f39006a35d Co-authored-by: Amp <amp@ampcode.com>
2026-03-29Add newsyslog rotation for dserver logsPaul Buetow
2026-03-28Add OpenBSD build VM and dtail package infrastructurePaul Buetow
Add a QEMU/KVM OpenBSD VM for native compilation of CGo packages (e.g. dtail with DataDog/zstd). The VM is fully automated via expect driving the serial console installer. - packages/buildvm/: setup, provision, start, stop scripts and expect installer - packages/scripts/pkg-dtail-openbsd.sh: multi-binary package with signify signing - packages/Makefile: build VM management and dtail-openbsd target using git archive - frontends/Rexfile: dtail_install task uses custom pkg repo, dtail task enabled Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28Move package build/upload scripts from gogios Magefile to conf/packagesPaul Buetow
Packaging logic is now OS-agnostic shell scripts + Makefile, reusable for any Go project. Cross-compiles locally, SCPs to target host for native packaging, and uploads to the PV. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28Sign OpenBSD packages with signify, drop -D unsignedPaul Buetow
Packages are now signed via pkg_sign with the custom-pkg signify key on the OpenBSD build host. The public key at /etc/signify/custom-pkg.pub on each client allows pkg_add to verify without -D unsigned. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28frontends: install gogios from pkg repo, add pkgrepo_setup taskPaul Buetow
Replace manual binary copy in gogios_install with pkg install (FreeBSD) and pkg_add (OpenBSD). Add pkgrepo_setup task that configures PKG_PATH in root's .profile on OpenBSD frontends. The gogios task now calls gogios_install automatically. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28pkgrepo: fix test package build scripts for FreeBSD and OpenBSDPaul Buetow
FreeBSD: use -p plist flag so files are actually included in the package. OpenBSD: use -D COMMENT flag and separate desc file as required by pkg_create, auto-detect OS version for repo path. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28pkgrepo: fix health probe path to /healthzPaul Buetow
The root path returns 404 by design, so probes need a dedicated /healthz endpoint that returns 200. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28pkgrepo: add FreeBSD/OpenBSD package repository servicePaul Buetow
Serve custom-built FreeBSD and OpenBSD packages via nginx in the k3s cluster. Includes helm chart, ArgoCD app, test artifact build script, and DNS entry via frontends Rexfile. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22fix: correct NFS sentinel filename in immich-postgres init containerPaul Buetow
The wait-for-nfs init container was checking for nfs.DO_NOT_REMOVE but the actual file on disk is k3svolumes.DO_NOT_REMOVE. This caused every new pod from the rolling update to be permanently stuck in Init:0/1, leaving two postgres pods running indefinitely (old + stuck new).
2026-03-22immich: add NFS mount check init container to postgresPaul Buetow
Amp-Thread-ID: https://ampcode.com/threads/T-019d14d5-4dbf-71a7-a619-d9c5afed3f7c Co-authored-by: Amp <amp@ampcode.com>
2026-03-21Remove obsolete documentation snippetsPaul Buetow
2026-03-21movedPaul Buetow
2026-03-20Parallelize delete-both VM teardownPaul Buetow
2026-03-20Add Pi VM launcher scriptsPaul Buetow
2026-03-20Add project Pi VM model switching configPaul Buetow
2026-03-20fix wireguard setup ssh host pinningPaul Buetow
2026-03-20task 301: extract provisioning collaboratorsPaul Buetow
2026-03-20task 300: persist effective service modePaul Buetow
2026-03-20task 299: clean up local state on deletePaul Buetow
2026-03-20task 298: pin SSH host keys per VM statePaul Buetow
2026-03-20task 297: lock down default ingress rulesPaul Buetow
2026-03-20Remove peers by allowed IPs from local WireGuard configPaul Buetow
2026-03-20Initial commit: add hyperstack-vm1.toml, hyperstack-vm2.toml, update ↵Paul Buetow
hyperstack.rb and wg1-setup.sh for multi-VM WireGuard support
2026-03-18vllm: skip docker pull on model switch, persist torch compile cachePaul Buetow
- model switch now passes pull_image: false to avoid surprise multi-GB image downloads when the upstream vLLM image was updated upstream; docker pull is still run on initial install (pull_image: true default) - mount /ephemeral/vllm_cache → /root/.cache/vllm so torch.compile artifacts survive container restarts; saves ~30-60 s on warm switches - add vllm_compile_cache_dir helper (sibling of hug_cache_dir) Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-03-18gpt-oss-120b: revert to 131072 — hard architecture limitPaul Buetow
max_position_embeddings=131072 in model config.json; exceeding it causes NaN/CUDA OOB. 163840 was rejected by vLLM at startup. The 135K error requires starting a fresh opencode conversation instead. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-03-18gpt-oss-120b: raise max_model_len to 163840 (160K)Paul Buetow
131K was still too small — observed 135K token conversations in practice. Physical KV capacity is 168K blocks so 160K is safe without OOM. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-03-18gpt-oss-120b: raise max_model_len to 131072Paul Buetow
MXFP4 KV cache is compact enough that vLLM allocated 168K token blocks (10560×16) at 0.92 utilization — the 40K limit was too conservative and caused negative max_tokens errors in long Claude Code sessions. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-03-18fix: handle bundler self_manager.rb error with Errno::ENOENTPaul Buetow
generated by cgit v1.2.3 (git 2.25.1) at 2026-08-10 18:45:35 +0000