| Age | Commit message (Collapse) | Author |
|---|
|
Colliding with the nightly generation CronJob's flock at the top of
some hours caused an intermittent EBADF failure. Move sync to two
fixed daytime ticks (10:00/18:00 Europe/Sofia) clear of the 04:00
generation run, and replace the mtime-based "did a generation
complete" check with a per-site content diff (image_count/
total_size_bytes vs a backed-up status.json from the last publish),
so a tick with no real content change is a no-op instead of an
unconditional publish.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
|
Track the shuriken.sh 0.14.0 release: Chart appVersion, generation +
sync CronJob image, docker-image/Justfile TAG, and README references.
|
|
The generation and sync CronJobs are separate, so each one's
concurrencyPolicy:Forbid only blocks itself, not the other. Add a
shared flock on /data/shuriken.sh/.lock (the NFS volume, auto-released
on pod death so a crash never leaves a stale lock):
- generation: blocking-acquire the lock around the whole multi-site
entrypoint run (waits for any in-progress sync, which is short).
- sync: non-blocking acquire; skip the tick if generation holds it.
flock is already in the image (util-linux), so no rebuild needed.
|
|
Per feedback: the shuriken-sync CronJob now uses the rsync daemon
protocol (rsync://) instead of SSH -- no key/Secret needed. The
frontends run rsyncd via inetd with hosts allow *.wg0; k3s pods run on
r-nodes with .wg0 (WireGuard) connectivity, so they're authorized to
push over the mesh. Add writable irregular-ninja and alt-irregular-ninja
modules to frontends/etc/rsyncd.conf.tpl (deploy with rex rsync).
It only publishes when a generation has COMPLETED since the last sync:
shuriken now deletes dist/status.json at the start of a run and writes
it last on success, so status.json presence+freshness vs a .last-sync
marker on NFS is the completed/not-yet-published signal. Most ticks
skip; a publish fires once after each daily generation. hostAliases pin
the .wg0 frontend names to mesh IPs (cluster DNS doesn't resolve *.wg0).
The nfs-check initContainer stays so rsync --delete can't wipe the live
site if NFS is down. shuriken --sync over SSH stays as a manual option.
|
|
Without an nfs-check initContainer, a run scheduled while NFS is
unmounted on the node would see an empty/stale /data source and
rsync --delete would wipe the live public site on fishfinger/blowfish.
Mirror the generation CronJob's sentinel guard (refuse to start if
/mnt/shuriken.sh/.nfs-sentinel is missing). Also run the publish
script under /bin/bash for reliable pipefail (ash's is version-gated).
|
|
Per request, decouple publishing from generation: add a second
shuriken-sync CronJob that rsyncs /data/shuriken.sh/<site>/dist to
admin@fishfinger.buetow.org and admin@blowfish.buetow.org (with
--delete) every 4h -- far more often than the daily generate. rsync
is incremental, so post-generation publishes are cheap and a failed
publish retries on the next tick without re-running generation.
Reuses the shuriken image (now with openssh-client). Requires a
shuriken-rsync-ssh-key Secret holding the admin key authorized on
fishfinger/blowfish; until it exists the publish pods fail to mount
the key and publish nothing (safe by design -- no live publish
without the key). README documents the one-time Secret creation and
the image rebuild/push for openssh.
|
|
The first run OOM-killed (exit 137) on irregular.ninja: Alpine's
ImageMagick is Q16-HDRI (32 bytes/pixel), so a 37 MP AI-upscaled PNG
decodes to ~1.2 GiB per buffer and -auto-orient doubles it -- well
over the 1 Gi pod limit. The image now ships an ImageMagick policy
(memory/map 512 MiB, disk 8 GiB) that spills oversized decodes to a
disk cache instead of OOMing. Match it here: bump the pod memory
limit to 2 Gi for the working set, pin OMP_NUM_THREADS=1 (the build
is OpenMP-enabled; extra resample threads only multiply HDRI memory
pressure), and add a disk-backed /tmp emptyDir (sizeLimit 8 Gi) for
the pixel-cache spill.
|
|
ArgoCD-managed CronJob that regenerates the irregular.ninja and
alt.irregular.ninja photo albums with the shuriken.sh Docker image,
writing them to /data/nfs/k3svolumes/shuriken.sh/<site>/dist on the
shared NFS export. Single image job by default (passively cooled N100
hosts). 04:00 Europe/Sofia daily, Forbid stacking, 6h deadline.
|
