diff options
| author | Paul Buetow <paul@buetow.org> | 2026-08-01 20:58:40 +0300 |
|---|---|---|
| committer | Paul Buetow <paul@buetow.org> | 2026-08-01 20:58:40 +0300 |
| commit | 2b20e1232b6542e780a61e1832e72cc2babc8989 (patch) | |
| tree | f4dd64f94f90ae33eea0d4598b346aff60aba386 /f3s | |
| parent | 38b01c385a59e6f9bd71ebe54a24fcf74b8966ab (diff) | |
shuriken CronJob: bound memory + ImageMagick disk spill (OOM fix)
The first run OOM-killed (exit 137) on irregular.ninja: Alpine's
ImageMagick is Q16-HDRI (32 bytes/pixel), so a 37 MP AI-upscaled PNG
decodes to ~1.2 GiB per buffer and -auto-orient doubles it -- well
over the 1 Gi pod limit. The image now ships an ImageMagick policy
(memory/map 512 MiB, disk 8 GiB) that spills oversized decodes to a
disk cache instead of OOMing. Match it here: bump the pod memory
limit to 2 Gi for the working set, pin OMP_NUM_THREADS=1 (the build
is OpenMP-enabled; extra resample threads only multiply HDRI memory
pressure), and add a disk-backed /tmp emptyDir (sizeLimit 8 Gi) for
the pixel-cache spill.
Diffstat (limited to 'f3s')
| -rw-r--r-- | f3s/shuriken/helm-chart/templates/cronjob.yaml | 22 |
1 files changed, 20 insertions, 2 deletions
diff --git a/f3s/shuriken/helm-chart/templates/cronjob.yaml b/f3s/shuriken/helm-chart/templates/cronjob.yaml index ddb97b8..2641ce0 100644 --- a/f3s/shuriken/helm-chart/templates/cronjob.yaml +++ b/f3s/shuriken/helm-chart/templates/cronjob.yaml @@ -65,13 +65,24 @@ spec: # to shuriken as --image-jobs. - name: SHURIKEN_IMAGE_JOBS value: "1" + # ImageMagick 7 is Q16-HDRI and built with OpenMP. Pin OpenMP + # to a single thread: the pod is cpu-limited to 1 core anyway, + # and extra resample threads would only multiply the HDRI pixel + # buffer's memory pressure (the OOM trigger for large PNGs). + - name: OMP_NUM_THREADS + value: "1" resources: requests: cpu: 100m memory: 256Mi limits: cpu: "1" - memory: 1Gi + # 2 Gi: the ImageMagick policy (baked into the image) caps + # the in-RAM pixel cache at 512 MiB and spills oversized + # decodes to the /tmp disk cache, so this headroom covers the + # working set + bash + the HDRI buffers of one photo without + # the pod being OOM-killed on a multi-megapixel source image. + memory: 2Gi securityContext: allowPrivilegeEscalation: false runAsUser: 0 @@ -82,10 +93,17 @@ spec: - name: configs mountPath: /configs readOnly: true + - name: tmp + mountPath: /tmp volumes: - name: data persistentVolumeClaim: claimName: shuriken-data-pvc - name: configs configMap: - name: shuriken-config
\ No newline at end of file + name: shuriken-config + # Disk-backed scratch for ImageMagick's disk pixel cache (the + # policy spills oversized HDRI decodes here instead of OOMing). + - name: tmp + emptyDir: + sizeLimit: 8Gi
\ No newline at end of file |