diff options
| author | Paul Buetow <paul@buetow.org> | 2026-08-02 16:10:08 +0300 |
|---|---|---|
| committer | Paul Buetow <paul@buetow.org> | 2026-08-02 16:10:08 +0300 |
| commit | fa0703374e505f2f63010c5dd371a0b3863f4013 (patch) | |
| tree | 2913cbb4c1a43e0ea48ffb84c4e4eeddb1f9bc69 /f3s | |
| parent | c945fce11d4f2964257c59099ecb40ab1d6ef2bd (diff) | |
shuriken: flock mutex so generation and sync never overlap
The generation and sync CronJobs are separate, so each one's
concurrencyPolicy:Forbid only blocks itself, not the other. Add a
shared flock on /data/shuriken.sh/.lock (the NFS volume, auto-released
on pod death so a crash never leaves a stale lock):
- generation: blocking-acquire the lock around the whole multi-site
entrypoint run (waits for any in-progress sync, which is short).
- sync: non-blocking acquire; skip the tick if generation holds it.
flock is already in the image (util-linux), so no rebuild needed.
Diffstat (limited to 'f3s')
| -rw-r--r-- | f3s/shuriken/helm-chart/templates/cronjob.yaml | 10 | ||||
| -rw-r--r-- | f3s/shuriken/helm-chart/templates/sync-cronjob.yaml | 11 |
2 files changed, 21 insertions, 0 deletions
diff --git a/f3s/shuriken/helm-chart/templates/cronjob.yaml b/f3s/shuriken/helm-chart/templates/cronjob.yaml index 2641ce0..ead3efe 100644 --- a/f3s/shuriken/helm-chart/templates/cronjob.yaml +++ b/f3s/shuriken/helm-chart/templates/cronjob.yaml @@ -59,6 +59,16 @@ spec: - name: shuriken image: registry.lan.buetow.org:30001/shuriken:0.13.2 imagePullPolicy: Always + command: ["/bin/sh", "-c"] + args: + - | + set -e + # Hold the generation/sync mutex for the WHOLE multi-site run + # so the shuriken-sync CronJob can't publish mid-generation + # (and vice versa). flock auto-releases if the pod dies, so a + # crash never leaves a stale lock. Blocking acquire: if a sync + # is mid-publish, wait for it (it's short) then generate. + exec flock /data/shuriken.sh/.lock shuriken-entrypoint env: # Default 1 image job (single-threaded); override here if a # faster one-off run is acceptable. The entrypoint passes this diff --git a/f3s/shuriken/helm-chart/templates/sync-cronjob.yaml b/f3s/shuriken/helm-chart/templates/sync-cronjob.yaml index 5cb4a31..66109e7 100644 --- a/f3s/shuriken/helm-chart/templates/sync-cronjob.yaml +++ b/f3s/shuriken/helm-chart/templates/sync-cronjob.yaml @@ -76,6 +76,17 @@ spec: set -euo pipefail D=/data/shuriken.sh + # Mutually exclude with the generation CronJob: don't publish + # while a generation is running. Non-blocking -- if the lock is + # held (generation in progress), skip this tick and let the next + # one publish once generation finishes. flock auto-releases if + # the generation pod dies, so a crash never wedges the sync. + exec 9>"$D/.lock" + if ! flock -n 9; then + echo "shuriken-sync: generation in progress (lock held); skipping" + exit 0 + fi + # Only publish when a generation has completed since the last # sync. status.json is deleted at the start of a run and # written last on success, so its presence means "completed" |