| Age | Commit message (Collapse) | Author |
|
|
|
|
|
|
|
Grafana's SQLite-on-NFS persistence is unreliable across restarts (the
new pod can't reacquire a clean exclusive lock after any NFS bounce),
and with Loki + Tempo also gone there's nothing left for it to
visualize. Keeping Prometheus alone for metrics + alerting.
Changes:
- prometheus.yaml: add grafana.enabled=false in the kube-prometheus-stack
values so the subchart no longer renders the grafana deployment/pvc.
- loki.yaml, tempo.yaml, grafana-ingress.yaml: renamed to .disabled
(same pattern as commit 03a18c6) so 'kubectl apply -f argocd-apps/'
stops re-creating them; the cluster Applications were also deleted,
which cascade-removes the helm resources via the resources-finalizer.
- alloy.yaml: drop the loki.write and otelcol.* blocks (no destinations
to ship to). DaemonSet stays deployed with a minimal 'logging' block
so the chart can be re-enabled by restoring the blocks here.
Prometheus TSDB was also wiped (corrupted zero-byte WAL segments from
the same NFS blip that took grafana down) — done separately, not part
of this commit.
|
|
|
|
Both apps were causing high CPU pressure on r0 after a cold-start (Trivy
respawning vulnerability scans, multiple replicas competing for image
pulls). Disabled by renaming the ArgoCD Application manifests to
.disabled so 'kubectl apply -f argocd-apps/' no longer picks them up,
and the Applications themselves were deleted from the cluster (with
prune=true the helm-managed resources were removed).
Amp-Thread-ID: https://ampcode.com/threads/T-019e2be9-50a8-7089-b628-b6d844602c13
Co-authored-by: Amp <amp@ampcode.com>
|
|
Adds a beets-based CronJob that runs every night on r1 (where the
Navidrome music PVC lives), fetching external cover.jpg into each album
folder and embedding art into audio files. Idempotent on re-runs:
- import.incremental skips already-known album folders
- fetchart skips albums that already have cover art
- embedart with ifempty:no + compare_threshold:50 only fills missing
embeds and refuses risky overwrites
Navidrome picks new art up via its existing 1h scan; no Navidrome change
required. Reuses navidrome-music-pvc directly (RWO is fine because both
pods pin to r1 via nodeSelector). State (library.db, logs) lives on a
small local-path PVC, regenerable by deleting the PVC.
Files: f3s/beets-art/helm-chart/{Chart.yaml,README.md,templates/*.yaml}
f3s/beets-art/Justfile (status, logs, run-now, suspend, resume, shell)
f3s/argocd-apps/services/beets-art.yaml
Amp-Thread-ID: https://ampcode.com/threads/T-019e223a-d137-705e-879b-84130c0e78ea
Co-authored-by: Amp <amp@ampcode.com>
|
|
Use extraHostVolumeMounts (prometheus-node-exporter sub-chart key for
host path mounts) instead of extraVolumes/extraVolumeMounts, which are
for general volumes. This correctly wires /var/lib/node_exporter/
textfile_collector into the container so the textfile arg takes effect.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|
- check-nfs-mount.sh: write nfs_mount_monitor_consecutive_failures gauge
to /var/lib/node_exporter/textfile_collector/nfs_mount_monitor.prom on
every run (via write_textfile_metric helper, called from write_fail_count
and directly on healthy runs); atomic tmp+mv write prevents partial reads
- Rexfile: create /var/lib/node_exporter/textfile_collector dir on r-nodes
- prometheus.yaml (ArgoCD app): enable textfile_collector in node_exporter
DaemonSet via extraArgs/extraVolumes/extraVolumeMounts; mount host path
/var/lib/node_exporter/textfile_collector into container
- persistence-values.yaml: sync node_exporter textfile_collector config
- nfs-mount-monitor-alerts.yaml: PrometheusRule with two alerts:
NfsMountAutoRepairWarning (>= 3 consecutive failures, severity: warning)
NfsMountAutoRepairCritical (>= 5 consecutive failures, severity: critical)
wired into new 'nfs-alerts' Alertmanager receiver with 30m repeat_interval
Tested: rex deploy succeeded, .prom files present on r0/r1/r2, timer clean.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|
|
|
Introduce Docker build/push workflow, Helm manifests, and ArgoCD application wiring for goprecords so the cluster can deploy the new daemon API service from the private registry.
Made-with: Cursor
|
|
Add dnsmasq.d wildcard for *.f3s.lan.buetow.org → 192.168.1.138 and
example compose for Pis; refresh README (DNS on pi2/pi3, etc-dnsmasq.d).
Align dormant ArgoCD Helm customDnsEntries with the same wildcard.
Made-with: Cursor
|
|
|
|
Amp-Thread-ID: https://ampcode.com/threads/T-019d6da8-3a08-7079-bb2a-eb072c0bf17f
Co-authored-by: Amp <amp@ampcode.com>
|
|
- ArgoCD app: aquasecurity/trivy-operator in monitoring with ServiceMonitor
- PrometheusRule for Critical/High trivy_image_vulnerabilities alerts
- Alertmanager route/receiver for component=trivy (UI; webhook TBD)
Made-with: Cursor
|
|
remove duplicate controllers.server
Amp-Thread-ID: https://ampcode.com/threads/T-019d6154-8fdf-74fe-b865-f796d8a4214a
Co-authored-by: Amp <amp@ampcode.com>
|
|
Amp-Thread-ID: https://ampcode.com/threads/T-019d6154-8fdf-74fe-b865-f796d8a4214a
Co-authored-by: Amp <amp@ampcode.com>
|
|
threads, increase worker timeout
Amp-Thread-ID: https://ampcode.com/threads/T-019d6154-8fdf-74fe-b865-f796d8a4214a
Co-authored-by: Amp <amp@ampcode.com>
|
|
apply
|
|
resources
|
|
|
|
|
|
Amp-Thread-ID: https://ampcode.com/threads/T-019d47a3-2deb-75c3-8a75-b0f39006a35d
Co-authored-by: Amp <amp@ampcode.com>
|
|
Amp-Thread-ID: https://ampcode.com/threads/T-019d47a3-2deb-75c3-8a75-b0f39006a35d
Co-authored-by: Amp <amp@ampcode.com>
|
|
Amp-Thread-ID: https://ampcode.com/threads/T-019d47a3-2deb-75c3-8a75-b0f39006a35d
Co-authored-by: Amp <amp@ampcode.com>
|
|
Amp-Thread-ID: https://ampcode.com/threads/T-019d47a3-2deb-75c3-8a75-b0f39006a35d
Co-authored-by: Amp <amp@ampcode.com>
|
|
Serve custom-built FreeBSD and OpenBSD packages via nginx in the k3s
cluster. Includes helm chart, ArgoCD app, test artifact build script,
and DNS entry via frontends Rexfile.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
|
retries, apt lock waits, and model verification
|
|
|
|
Updated immich-server and immich-machine-learning images to v2.5.5.
This release includes major features:
- Free Up Space functionality
- Non-destructive photo editing
- Database backup and restore via web
- Upload improvements and visual refresh
- Progressive JPEGs support
- Additional fine-grained API key permissions
Release notes: https://github.com/immich-app/immich/releases/tag/v2.5.5
|
|
- Created custom ingress-lan.yaml for immich.f3s.lan.buetow.org with TLS
- Removed unsupported 'lan' ingress config from ArgoCD app values
- The Immich Helm chart doesn't support multiple named ingresses,
so we create the LAN ingress as a custom resource instead
This aligns immich with other services that have both regular and
LAN ingress endpoints.
|
|
Add *.f3s.lan.buetow.org ingress resources for all services to enable
LAN access with TLS termination. This allows direct access from the
192.168.1.0/24 network through the FreeBSD CARP/relayd setup.
Services updated:
- argocd: argocd.f3s.lan.buetow.org
- cgit: cgit.f3s.lan.buetow.org
- grafana: grafana.f3s.lan.buetow.org
- anki-sync-server: anki.f3s.lan.buetow.org
- apache: f3s.lan.buetow.org, www.f3s.lan.buetow.org, standby.f3s.lan.buetow.org
- audiobookshelf: audiobookshelf.f3s.lan.buetow.org
- filebrowser: filebrowser.f3s.lan.buetow.org
- immich: immich.f3s.lan.buetow.org
- ipv6test: ipv6test.f3s.lan.buetow.org (+ ipv4/ipv6 subdomains)
- keybr: keybr.f3s.lan.buetow.org
- koreader-sync-server: koreader.f3s.lan.buetow.org
- miniflux: flux.f3s.lan.buetow.org
- opodsync: gpodder.f3s.lan.buetow.org
- radicale: radicale.f3s.lan.buetow.org
- syncthing: syncthing.f3s.lan.buetow.org
- tracing-demo: tracing-demo.f3s.lan.buetow.org
- wallabag: bag.f3s.lan.buetow.org
- webdav: webdav.f3s.lan.buetow.org
All LAN ingresses use:
- TLS with f3s-lan-tls certificate (cert-manager)
- Traefik entrypoints: web,websecure
- Same backend services as external ingresses
Also fixed koreader-sync-server ingress to use modern annotations.
Co-authored-by: Cursor <cursoragent@cursor.com>
|
|
Configure Pi-hole DNS service to bind to 192.168.1.120 for LAN access.
This allows clients on the 192.168.1.0/24 network to use Pi-hole as
their DNS server without needing to be on the Wireguard mesh.
Co-authored-by: Cursor <cursoragent@cursor.com>
|
|
The pihole helm chart uses 'admin.existingSecret' not 'adminPasswordSecret'.
This ensures the deployment uses the pihole-admin-password secret instead
of creating a default 'pihole-password' secret with 'admin' password.
Co-authored-by: Cursor <cursoragent@cursor.com>
|
|
Pi-hole's web interface returns 403 Forbidden when accessed via the
root path. Add a Traefik middleware that redirects requests to the
root URL to /admin/ path where the web interface is accessible.
Also add the pihole ArgoCD application manifest.
Co-authored-by: Cursor <cursoragent@cursor.com>
|
|
Co-authored-by: Cursor <cursoragent@cursor.com>
|
|
- Add cert-manager for self-signed TLS certificates
- Create wildcard cert for *.f3s.lan.buetow.org
- Add LAN ingress to Navidrome (navidrome.f3s.lan.buetow.org)
- Document FreeBSD relayd configuration for LAN access
- Add comprehensive setup guide
LAN access uses existing CARP VIP (192.168.1.138) on f0/f1
with relayd forwarding HTTP/HTTPS to k3s Traefik NodePorts.
External access via OpenBSD relayd continues unchanged.
|
|
Adds Navidrome music streaming server with:
- Helm chart with deployment, service, ingress, and persistent volumes
- Two PVs: data (10Gi) and music library (200Gi)
- ArgoCD application for automated deployment
- Ingress at navidrome.f3s.buetow.org
- Justfile for operational commands
|
|
|
|
Amp-Thread-ID: https://ampcode.com/threads/T-019c1492-bec0-70f8-8d02-ef3596a7228b
Co-authored-by: Amp <amp@ampcode.com>
|
|
This reverts commit 6fd8d4e3d3f7abf54a9ae433f0c47ce18a84a3c1.
|
|
Amp-Thread-ID: https://ampcode.com/threads/T-019c10cd-6a62-75c9-a556-f2ed9611f278
Co-authored-by: Amp <amp@ampcode.com>
|
|
Amp-Thread-ID: https://ampcode.com/threads/T-019c10b2-ea57-752e-818e-33a56d69d9fa
Co-authored-by: Amp <amp@ampcode.com>
|
|
Amp-Thread-ID: https://ampcode.com/threads/T-019c1088-dc3e-701a-b064-5e50cf52b32a
Co-authored-by: Amp <amp@ampcode.com>
|
|
Amp-Thread-ID: https://ampcode.com/threads/T-019c1088-dc3e-701a-b064-5e50cf52b32a
Co-authored-by: Amp <amp@ampcode.com>
|
|
Amp-Thread-ID: https://ampcode.com/threads/T-019c1088-dc3e-701a-b064-5e50cf52b32a
Co-authored-by: Amp <amp@ampcode.com>
|
|
- Add Helm chart source to deploy immich-app/immich chart
- Include yoga-videos PVC volume mount in server config
- Persist configuration across redeployments
- Keep custom resources deployment for PVs and PostgreSQL
Amp-Thread-ID: https://ampcode.com/threads/T-019c1088-dc3e-701a-b064-5e50cf52b32a
Co-authored-by: Amp <amp@ampcode.com>
|
|
Amp-Thread-ID: https://ampcode.com/threads/T-019c086d-c760-779d-b740-0f748094b62a
Co-authored-by: Amp <amp@ampcode.com>
|
|
Amp-Thread-ID: https://ampcode.com/threads/T-019c086d-c760-779d-b740-0f748094b62a
Co-authored-by: Amp <amp@ampcode.com>
|
|
|