From f6035ac0119fb65fc7ed955932729f3553a1ba6d Mon Sep 17 00:00:00 2001 From: Paul Buetow Date: Sun, 2 Aug 2026 09:04:43 +0300 Subject: shuriken: separate rsync sync CronJob (publish to web servers) Per request, decouple publishing from generation: add a second shuriken-sync CronJob that rsyncs /data/shuriken.sh//dist to admin@fishfinger.buetow.org and admin@blowfish.buetow.org (with --delete) every 4h -- far more often than the daily generate. rsync is incremental, so post-generation publishes are cheap and a failed publish retries on the next tick without re-running generation. Reuses the shuriken image (now with openssh-client). Requires a shuriken-rsync-ssh-key Secret holding the admin key authorized on fishfinger/blowfish; until it exists the publish pods fail to mount the key and publish nothing (safe by design -- no live publish without the key). README documents the one-time Secret creation and the image rebuild/push for openssh. --- f3s/shuriken/README.md | 30 ++++++++ .../helm-chart/templates/sync-cronjob.yaml | 88 ++++++++++++++++++++++ 2 files changed, 118 insertions(+) create mode 100644 f3s/shuriken/helm-chart/templates/sync-cronjob.yaml diff --git a/f3s/shuriken/README.md b/f3s/shuriken/README.md index 2025023..62e4e3f 100644 --- a/f3s/shuriken/README.md +++ b/f3s/shuriken/README.md @@ -71,6 +71,36 @@ just sync # refresh the ArgoCD app just argocd-status # argocd CLI view ``` +## Publishing (separate rsync CronJob) + +`shuriken-sync` is a second CronJob that rsyncs the generated +`/data/shuriken.sh//dist` trees to the public web servers +(`admin@fishfinger.buetow.org` and `admin@blowfish.buetow.org`, with +`--delete`) every 4h -- decoupled from the daily generation so a publish can +be retried far more often than a (re)generate. It reuses the shuriken image +(rsync + openssh-client) but overrides the command, so it never runs +generation. + +It needs the admin SSH key authorized on fishfinger/blowfish. Until that key +is provisioned the publish pods fail to mount the key and publish nothing (safe +by design -- no live publish without the key). Create the Secret once: + +```bash +kubectl create secret generic shuriken-rsync-ssh-key -n services \ + --from-file=id_ed25519=/path/to/admin_ed25519 +``` + +The generation CronJob deliberately has no `SYNC_*` settings -- it only writes +to NFS; all publishing goes through `shuriken-sync`. + +The image includes `openssh-client` (alongside rsync) so the same image serves +both jobs. If the registry still holds an older `shuriken:0.13.2` without it, +rebuild and push before activating sync (the generation job is unaffected): + +```bash +cd /home/paul/git/conf/f3s/shuriken && just build-push +``` + ## ArgoCD `argocd-apps/services/shuriken.yaml` points ArgoCD at diff --git a/f3s/shuriken/helm-chart/templates/sync-cronjob.yaml b/f3s/shuriken/helm-chart/templates/sync-cronjob.yaml new file mode 100644 index 0000000..f3155c6 --- /dev/null +++ b/f3s/shuriken/helm-chart/templates/sync-cronjob.yaml @@ -0,0 +1,88 @@ +# shuriken-sync: a SEPARATE CronJob that publishes the already-generated +# /data/shuriken.sh//dist trees to the public web servers (fishfinger + +# blowfish) over rsync/SSH. Decoupled from the generation CronJob so it can run +# more frequently than the daily generate -- rsync only transfers the diff, so +# an incremental publish after generation is cheap, and a failed publish is +# retried on the next tick without re-running the (expensive) generation. +# +# Requires a Secret `shuriken-rsync-ssh-key` holding the admin private key +# authorized on fishfinger/blowfish (the same key the local `shuriken --sync` +# uses). Until that Secret is provisioned this CronJob's pods fail to mount the +# key and publish nothing -- safe by design (no live publish without the key). +# Create it once: +# kubectl create secret generic shuriken-rsync-ssh-key -n services \ +# --from-file=id_ed25519=/path/to/admin_ed25519 +apiVersion: batch/v1 +kind: CronJob +metadata: + name: shuriken-sync + namespace: services +spec: + # Every 4h -- 6x/day, far more often than the daily 04:00 generation. rsync + # is incremental, so after the first publish each tick only ships the diff. + schedule: "0 */4 * * *" + timeZone: Europe/Sofia + concurrencyPolicy: Forbid + startingDeadlineSeconds: 300 + successfulJobsHistoryLimit: 3 + failedJobsHistoryLimit: 3 + jobTemplate: + spec: + # Transient SSH/rsync failures (gateway blip, NFS hiccup) are retried by + # the next tick; don't spam retries within one run. + backoffLimit: 0 + activeDeadlineSeconds: 3600 + template: + spec: + restartPolicy: Never + containers: + - name: shuriken-sync + image: registry.lan.buetow.org:30001/shuriken:0.13.2 + imagePullPolicy: Always + command: ["/bin/sh", "-c"] + args: + - | + set -euo pipefail + export HOME=/tmp + mkdir -p "$HOME/.ssh" + SSH="ssh -i /ssh-keys/id_ed25519 -o StrictHostKeyChecking=accept-new -o BatchMode=yes" + for site in irregular.ninja alt.irregular.ninja; do + for srv in fishfinger.buetow.org blowfish.buetow.org; do + echo "shuriken-sync: publishing $site -> $srv" + rsync -a --delete --info=stats1 -e "$SSH" \ + "/data/shuriken.sh/$site/dist/" \ + "admin@$srv:/var/www/htdocs/$site/" + done + done + echo "shuriken-sync: published both sites to both servers" + resources: + requests: + cpu: 50m + memory: 64Mi + limits: + cpu: "1" + memory: 512Mi + securityContext: + allowPrivilegeEscalation: false + runAsUser: 0 + runAsGroup: 0 + volumeMounts: + - name: data + mountPath: /data + readOnly: true + - name: ssh-keys + mountPath: /ssh-keys + readOnly: true + - name: tmp + mountPath: /tmp + volumes: + - name: data + persistentVolumeClaim: + claimName: shuriken-data-pvc + - name: ssh-keys + secret: + secretName: shuriken-rsync-ssh-key + defaultMode: 0400 + - name: tmp + emptyDir: + sizeLimit: 256Mi \ No newline at end of file -- cgit v1.2.3