diff options
| author | Paul Buetow <paul@buetow.org> | 2026-07-10 15:38:46 +0300 |
|---|---|---|
| committer | Paul Buetow <paul@buetow.org> | 2026-07-10 15:38:46 +0300 |
| commit | 37d88cfedf8a387930332ee53091cff6bd8dc7a7 (patch) | |
| tree | ca86c8d727569cba700ee2d2bb880074e9855991 | |
| parent | 817c9e38c38152ddbd87cf381c1502ba5b504aa7 (diff) | |
beets-art: unpin from r1 + cap CPU to 1 (thermal fix)
The beets-art CronJob was hard-pinned to r1 via nodeSelector, and its
CPU limit was 2 (half the 4-core passively-cooled Beelink N100 on f1).
The daily noon fetchart/embedart sweep sustained ~35% CPU for ~28 min
and pushed f1 to ~95 C, tripping the FreebsdCpuTemperatureHigh alert
(>=80 C for 5m).
Two changes:
1. Cap the container CPU limit from 2 to 1. The run takes ~2x longer but
stays well under the 6h activeDeadlineSeconds and roughly halves the
sustained thermal load.
2. Unpin the CronJob from r1 so the scheduler can spread it. This is a
manifest/config change, not a storage migration: the music library is
already shared NFS (mounted at the same path on all three r-nodes).
The only thing forcing r1 was the music PV/PVC being declared RWO
(single-node mount) while shared with Navidrome, which is itself pinned
to r1 by its local-path SQLite data PVC. Flip navidrome-music PV/PVC
to ReadWriteMany so beets-art can mount it on any r-node alongside the
read-only Navidrome pod.
Also move beets-art-state from local-path (pinned to r1) to a static
NFS-backed RWX PV/PVC so the job's SQLite state is reachable from any
node. SQLite-over-NFS is acceptable here because concurrencyPolicy:
Forbid guarantees a single writer (no lock contention), and the state
is regenerable.
Application requires recreating the bound music PVC (access-mode changes
are immutable on a bound PVC) and the state PVC; see deploy notes.
| -rw-r--r-- | f3s/beets-art/helm-chart/templates/cronjob.yaml | 24 | ||||
| -rw-r--r-- | f3s/beets-art/helm-chart/templates/persistent-volume.yaml | 41 | ||||
| -rw-r--r-- | f3s/navidrome/helm-chart/templates/persistent-volume.yaml | 15 |
3 files changed, 60 insertions, 20 deletions
diff --git a/f3s/beets-art/helm-chart/templates/cronjob.yaml b/f3s/beets-art/helm-chart/templates/cronjob.yaml index 132160d..a5c0929 100644 --- a/f3s/beets-art/helm-chart/templates/cronjob.yaml +++ b/f3s/beets-art/helm-chart/templates/cronjob.yaml @@ -43,8 +43,11 @@ spec: template: spec: restartPolicy: Never - nodeSelector: - kubernetes.io/hostname: r1.lan.buetow.org + # Not pinned to any node: the music library is RWX NFS (shared + # across r0/r1/r2) and the state PVC is now NFS-backed too, so the + # job may land on any r-node. Letting the scheduler spread it avoids + # the thermal spike that occurred when it was hard-pinned to r1 + # (on the passively-cooled f1) with a CPU limit of 2. containers: - name: beets image: lscr.io/linuxserver/beets:latest @@ -102,20 +105,27 @@ spec: - name: tmp mountPath: /tmp resources: - # Generous because ImageMagick + ffprobe + SQLite scans can - # spike. Tighten after observing real usage. + # Capped at 1 CPU to limit thermal load on the passively-cooled + # Beelink N100 host (f1). A limit of 2 cores (half the 4-core + # N100) sustained during the ~28 min fetchart/embedart sweep + # pushed f1 to ~95 C and tripped the FreebsdCpuTemperatureHigh + # alert (>=80 C for 5m). Halving the cap roughly halves the + # sustained heat; the run takes ~2x longer but stays well under + # the 6h activeDeadlineSeconds. Request stays at 100m so the + # scheduler still treats it as a light batch job. requests: cpu: 100m memory: 256Mi limits: - cpu: "2" + cpu: "1" memory: 1Gi volumes: - name: music persistentVolumeClaim: # Reuse the existing Navidrome music PVC — single source of - # truth for the library tree. RWO is OK because both pods - # are pinned to r1. + # truth for the library tree. It is RWX (shared NFS), so this + # job can mount it on any r-node alongside Navidrome (which + # mounts it read-only). claimName: navidrome-music-pvc - name: state persistentVolumeClaim: diff --git a/f3s/beets-art/helm-chart/templates/persistent-volume.yaml b/f3s/beets-art/helm-chart/templates/persistent-volume.yaml index 0b6c422..fcb21d8 100644 --- a/f3s/beets-art/helm-chart/templates/persistent-volume.yaml +++ b/f3s/beets-art/helm-chart/templates/persistent-volume.yaml @@ -1,20 +1,41 @@ -# beets-art-state-pvc: small local-path PVC on r1 for the beets SQLite DB, -# logs, and any temp working files. Same reasoning as navidrome-data-pvc: -# - SQLite over NFS is fragile under concurrent locks. -# - The CronJob is pinned to r1 (see cronjob.yaml nodeSelector), and the -# local-path provisioner pins the PV to whichever node first consumes -# it (WaitForFirstConsumer), so r1 is always the home. -# Contents are essentially regenerable: deleting this PVC just forces the -# next run to re-import the library (slower one-off, then incremental again). +# beets-art-state: beets SQLite state DB, logs, and temp working files, kept +# on the shared NFS export (same mount as navidrome-music) so the CronJob can +# run on any r-node — it is no longer pinned to r1. +# +# Previously this was a local-path PVC pinned to r1 (SQLite-over-NFS +# avoidance). Moving to NFS is acceptable here because: +# - concurrencyPolicy: Forbid means exactly one writer ever runs at a time; +# there is no concurrent lock contention that makes SQLite-over-NFS +# fragile (unlike the Navidrome/Grafana rolling-restart case). +# - Contents are essentially regenerable: if the SQLite DB ever corrupts, +# deleting it just forces the next run to re-import the library (slower +# one-off, then incremental again). +# The directory is created on first use via DirectoryOrCreate (it lives on the +# shared NFS mount, so all r-nodes see the same state). +apiVersion: v1 +kind: PersistentVolume +metadata: + name: beets-art-state-pv +spec: + capacity: + storage: 2Gi + volumeMode: Filesystem + accessModes: + - ReadWriteMany + persistentVolumeReclaimPolicy: Retain + hostPath: + path: /data/nfs/k3svolumes/beets-art/state + type: DirectoryOrCreate +--- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: beets-art-state-pvc namespace: services spec: - storageClassName: local-path + storageClassName: "" accessModes: - - ReadWriteOnce + - ReadWriteMany resources: requests: storage: 2Gi diff --git a/f3s/navidrome/helm-chart/templates/persistent-volume.yaml b/f3s/navidrome/helm-chart/templates/persistent-volume.yaml index 3ca2867..3142c93 100644 --- a/f3s/navidrome/helm-chart/templates/persistent-volume.yaml +++ b/f3s/navidrome/helm-chart/templates/persistent-volume.yaml @@ -27,7 +27,16 @@ spec: storage: 10Gi --- # navidrome-music: keep on NFS — 200 GB library shared across nodes. -# This is a static PV backed by hostPath at the NFS mount point. +# This is a static PV backed by hostPath at the NFS mount point. The NFS +# export is mounted at the same path on all three r-nodes (r0/r1/r2), so the +# hostPath resolves identically on each node. +# +# ReadWriteMany (not RWO): Navidrome mounts this read-only as the library +# reader; the beets-art CronJob mounts it read-write to fetch and embed cover +# art. RWX lets beets-art run on any r-node (it no longer has to co-locate +# with Navidrome just to satisfy RWO's single-node-mount rule). At the FS +# level there is a single writer (beets-art; Forbid concurrency) and a single +# reader (Navidrome), so no write contention. apiVersion: v1 kind: PersistentVolume metadata: @@ -37,7 +46,7 @@ spec: storage: 200Gi volumeMode: Filesystem accessModes: - - ReadWriteOnce + - ReadWriteMany persistentVolumeReclaimPolicy: Retain hostPath: path: /data/nfs/k3svolumes/navidrome/music @@ -51,7 +60,7 @@ metadata: spec: storageClassName: "" accessModes: - - ReadWriteOnce + - ReadWriteMany resources: requests: storage: 200Gi |
