diff options
| author | Paul Buetow <paul@buetow.org> | 2026-06-24 11:00:33 +0300 |
|---|---|---|
| committer | Paul Buetow <paul@buetow.org> | 2026-06-24 11:00:33 +0300 |
| commit | 928f28d0c51ca47f9958fda4c4bb7c8b0c571e34 (patch) | |
| tree | 5d9d7a80aeb845598d1bdac928ed5a1ed2e9540f /src/lib/image-pipeline.source.sh | |
| parent | 6697431d3855d3030b35c2ce232c9ea0e9282ba7 (diff) | |
Encapsulate job pool behind a single handle (job_pool_*)
Replace the four-parallel-nameref job-pool API (pids/statuses/labels/failed,
each needing its own `shellcheck disable=SC2034` at every call site) with a
single pool handle: a name prefix whose four backing variables
(${pool}_pids/_statuses/_labels/_failed) are derived on demand by the helpers.
Bash can't nest indexed arrays in an associative array, so a prefixed-handle
with declare -g backing vars is the simplest pure-nameref encoding (no eval).
New public API:
job_pool_init <pool>
job_pool_submit <pool> <label> <cmd...>
job_pool_wait <pool> # returns 1 if any job failed
Migrate all callers (scalephotos, create_all_photo_derivatives,
render_album_pages, render_view_redirects, render_filter_pages) to the handle.
queue_preview_page_render_job / queue_album_view_render_job /
_album_record_view_photo / _stats_enqueue_filter_album now take one pool arg
instead of four names.
Drop the dead wrappers and unused parameterization: wait_for_image_job_slot,
wait_for_template_render_job_slot and their _jobs variants only ever passed
IMAGE_JOBS, so the max_jobs parameter is gone and throttling is fixed at
IMAGE_JOBS inside the pool. Also removes wait_for_album_view_render_jobs (now
just job_pool_wait).
Throttling (max IMAGE_JOBS concurrent), failure detection and failed-job
propagation are unchanged; the parallel-throttling and failure-logging tests
pass unmodified. SC2034 disable-comments across the touched files drop 29->7.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Diffstat (limited to 'src/lib/image-pipeline.source.sh')
| -rw-r--r-- | src/lib/image-pipeline.source.sh | 35 |
1 files changed, 8 insertions, 27 deletions
diff --git a/src/lib/image-pipeline.source.sh b/src/lib/image-pipeline.source.sh index 876501b..934c3b5 100644 --- a/src/lib/image-pipeline.source.sh +++ b/src/lib/image-pipeline.source.sh @@ -38,41 +38,22 @@ create_all_photo_derivatives() { local -r photos_dir="$1"; shift local -r thumbs_dir="$1"; shift local -r blurs_dir="$1"; shift - local -i failed=0 - local -a image_job_pids=() - # Passed by name to wait_for_image_job_slot and wait_for_image_jobs. - # shellcheck disable=SC2034 - local -A image_job_labels=() - # Passed by name to wait_for_image_job_slot and wait_for_image_jobs. - # shellcheck disable=SC2034 - local -A image_job_statuses=() local photo + # Throttled background job pool (max IMAGE_JOBS concurrent), addressed by the + # single handle "image_jobs". job_pool_wait returns 1 if any job failed. + job_pool_init image_jobs + while IFS= read -r photo; do - wait_for_image_job_slot \ - image_job_pids \ - image_job_statuses \ - image_job_labels \ - failed - create_photo_derivatives "$photos_dir" "$thumbs_dir" "$blurs_dir" \ - "$photo" & - image_job_pids+=("$!") - # Read through a nameref in the job-pool helpers. - # shellcheck disable=SC2034 - image_job_labels["$!"]="image derivative job for photo $photo" + job_pool_submit image_jobs "image derivative job for photo $photo" \ + create_photo_derivatives "$photos_dir" "$thumbs_dir" "$blurs_dir" \ + "$photo" done < <( find "$DIST_DIR/$photos_dir" -maxdepth 1 -type f -printf '%f\n' \ | sort ) - wait_for_image_jobs \ - image_job_pids \ - image_job_statuses \ - image_job_labels \ - failed - if (( failed != 0 )); then - return 1 - fi + job_pool_wait image_jobs } prepare_generation_photo_assets() { |
