summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorPaul Buetow <paul@buetow.org>2026-05-24 12:33:00 +0300
committerPaul Buetow <paul@buetow.org>2026-05-24 12:33:00 +0300
commit008c30bca834447cad4f0aed427178cd155d4703 (patch)
treeb9a8e1a5d66b3a896ce3ddcfa1fa7daa42076020
parent76b1cc008afc878e7edbd46feb6680ccab6880f0 (diff)
chore: remove photo/ComfyUI top-level files
Delete hyperstack-vm-photo.toml, photo-enhance.rb, photo-enhance-review.md, smart_photo_node.py, workflows/photo-enhance.json (and empty workflows/ dir), and __pycache__/smart_photo_node.cpython-314.pyc (and empty __pycache__/ dir). No .hyperstack-vm-photo-state.json* state files were present. ComfyUI references in lib/hyperstack/*.rb intentionally left for task T2.
-rw-r--r--hyperstack-vm-photo.toml68
-rw-r--r--photo-enhance-review.md678
-rwxr-xr-xphoto-enhance.rb581
-rw-r--r--smart_photo_node.py796
-rw-r--r--workflows/photo-enhance.json106
5 files changed, 0 insertions, 2229 deletions
diff --git a/hyperstack-vm-photo.toml b/hyperstack-vm-photo.toml
deleted file mode 100644
index d6130b0..0000000
--- a/hyperstack-vm-photo.toml
+++ /dev/null
@@ -1,68 +0,0 @@
-[auth]
-api_key_file = "~/.hyperstack"
-
-[hyperstack]
-base_url = "https://infrahub-api.nexgencloud.com/v1"
-
-[state]
-file = ".hyperstack-vm-photo-state.json"
-
-[vm]
-name_prefix = "hyperstack-photo"
-hostname = "hyperstack-photo"
-environment_name = "snonux-ollama"
-
-# L40 (48GB GDDR6, ~$1.00/hr) is the recommended GPU for ComfyUI photo enhancement.
-# It provides ample VRAM for SUPIR (FP16 needs ~12-20GB) + Real-ESRGAN with large tile sizes.
-# Cheaper than A100 ($2/hr) while offering faster CUDA cores and FP8 support.
-flavor_name = "n3-L40x1"
-image_name = "Ubuntu Server 24.04 LTS R570 CUDA 12.8 with Docker"
-assign_floating_ip = true
-create_bootable_volume = false
-enable_port_randomization = false
-labels = ["comfyui", "photo-enhance", "wireguard"]
-
-[ssh]
-username = "ubuntu"
-private_key_path = "~/.ssh/id_rsa"
-hyperstack_key_name = "earth"
-port = 22
-connect_timeout_sec = 10
-
-[network]
-wireguard_udp_port = 56710
-wireguard_subnet = "192.168.3.0/24"
-# Photo VM uses .4; VM1=.1, VM2=.3 are the LLM VMs on the same wg1 tunnel.
-wireguard_server_ip = "192.168.3.4"
-# Reuse the same inference port constant; ComfyUI uses its own port 8188 below.
-ollama_port = 11434
-allowed_ssh_cidrs = ["auto"]
-allowed_wireguard_cidrs = ["auto"]
-
-[bootstrap]
-enable_guest_bootstrap = true
-install_wireguard = true
-configure_ufw = true
-configure_ollama_host = false
-
-[ollama]
-# Not needed on this VM; photo enhancement uses ComfyUI exclusively.
-install = false
-
-[vllm]
-# Not needed on this VM; ComfyUI handles all inference.
-install = false
-
-[comfyui]
-install = true
-# ComfyUI REST API port — opened on the WireGuard subnet only.
-port = 8188
-# Model weights on ephemeral NVMe for fast access; survives reboots on Hyperstack.
-models_dir = "/ephemeral/comfyui/models"
-output_dir = "/ephemeral/comfyui/output"
-container_name = "comfyui_photo"
-# Pre-downloaded model weights:
-# RealESRGAN_x4plus — fast 4x upscaling + sharpening (~65MB, upscale_models/)
-# SUPIR-v0Q — SDXL-based photo restoration, Photolemur-quality results (~8GB, checkpoints/)
-# SUPIR-v0F — SUPIR variant tuned for faithful fidelity over generative enhancement (~8GB)
-models = ["RealESRGAN_x4plus", "SUPIR-v0Q"]
diff --git a/photo-enhance-review.md b/photo-enhance-review.md
deleted file mode 100644
index 935c0b2..0000000
--- a/photo-enhance-review.md
+++ /dev/null
@@ -1,678 +0,0 @@
-Main technical risks
-1. Real-ESRGAN first, on every image, is your biggest quality risk
-
-Running every image through:
-
-4× ESRGAN
-then downscale back to original size
-
-can definitely improve some photos, but it can also introduce:
-
-hallucinated texture
-crispy foliage
-waxy skin after interaction with later steps
-fake edge detail
-zippering around fine geometry
-over-defined JPEG blocks on already compressed Fuji JPEGs
-
-This is the part I would treat as conditionally applied, not universal.
-
-My recommendation:
-
-gate ESRGAN based on image characteristics
-or at least use different strength paths for portrait vs landscape vs night
-
-Examples:
-
-portraits: maybe skip global ESRGAN or use a weaker path
-night/high-ISO: be careful, because ESRGAN can turn noise into invented detail
-landscapes/architecture: often benefit the most
-
-Right now the pipeline assumes “seen at 16K then downscaled” is always a win. It often is not.
-
-2. CodeFormer after global enhancement can amplify inconsistency
-
-CodeFormer is useful, but it can produce faces that look slightly detached from the rest of the frame if the global pipeline has already altered texture and local contrast.
-
-Potential issues:
-
-face crops look cleaner than surrounding skin/neck/hair
-restored face sharpness conflicts with depth blur/sharpen later
-multiple faces in one frame may get uneven treatment
-
-Things to consider:
-
-apply CodeFormer only when face size exceeds a threshold
-use a lower-strength/fidelity profile depending on scene
-skip CodeFormer for distant faces
-log face count and face bounding-box size into metadata
-
-That would make the workflow easier to debug when faces look “too AI.”
-
-3. Scene classification using 8 CLIP prompts is clever but brittle
-
-This is a nice lightweight idea, but it is likely the weakest decision point in the pipeline because eight prompts force coarse categorization.
-
-Possible failure cases:
-
-beach sunset might oscillate between beach, golden_hour, and landscape
-indoor portraits near a window may flip to portrait or indoor
-urban night scenes may misclassify between street and night
-cloudy mountain lake might be overcast vs landscape
-
-Because your grade profile changes exposure/contrast/saturation/detail/denoise, a wrong label can materially alter the image.
-
-Better approach:
-
-store the full prompt score distribution, not just argmax
-use top-2 or top-3 labels
-blend profiles based on confidence instead of hard-switching
-
-For example:
-
-60% landscape + 40% golden_hour
-instead of forcing one profile
-
-That would reduce sudden profile mistakes.
-
-4. CPU image ops at 4K are fine, but not yet optimized as a pipeline
-
-Your CPU-bound stages are sensible, but there are some efficiency concerns:
-
-guidedFilter and morphology/blur passes at full 4K are not trivial
-ImageScaleBy 16K → 4K on CPU may be heavier than it looks
-repeated color-space conversions and full-frame copies can become memory-bandwidth bound
-if you later parallelize multiple photos, CPU becomes the bottleneck before GPU memory does
-
-This matters because your throughput is already 40–50s/photo, and if you batch more aggressively you may saturate host CPU.
-
-I would especially watch:
-
-OpenCV allocations
-Python ↔ tensor conversion overhead inside custom nodes
-whether large intermediate tensors are duplicated unnecessarily
-5. Polling /history/<prompt_id> every 2s is workable but not ideal
-
-It is acceptable, but it is a weak point operationally.
-
-Risks:
-
-stale/incomplete history states
-long-run prompt ambiguity if ComfyUI restarts
-polling delay adds latency
-harder recovery when output partially exists but metadata doesn’t
-
-If ComfyUI or your wrapper supports websocket progress or event-driven status, that would be better. If not, I would at least strengthen state validation:
-
-ensure expected output files exist and are complete
-ensure metadata JSON corresponds to the same prefix
-distinguish timeout from partial success
-Biggest architectural improvement opportunities
-1. Add conditional routing, not one fixed pipeline for every photo
-
-Right now the graph is elegant, but it is still mostly single-path.
-
-A more robust system would route based on detected attributes:
-
-no faces → skip CodeFormer
-little/no sky → skip SkyEnhance
-low confidence scene label → use default conservative grade
-low-detail or noisy photo → reduce or skip ESRGAN
-already high-contrast/high-saturation image → apply weaker grade
-
-That would reduce over-processing and save time.
-
-2. Move from hardcoded profiles to measured image statistics
-
-Your scene profiles are sensible, but they are still hand-tuned guesses.
-
-A stronger next step would be to incorporate measured stats such as:
-
-luminance histogram
-highlight clipping ratio
-shadow floor occupancy
-saturation percentile
-edge density
-noise estimate
-face area percentage
-sky coverage
-
-Then use those stats to modulate:
-
-exposure
-saturation
-detail multiplier
-denoise
-background blur
-
-That would make the pipeline more adaptive and less prompt-dependent.
-
-3. Preserve and restore metadata more deliberately
-
-You correctly bake orientation before upload because ComfyUI strips EXIF. Good.
-
-But converting final PNG to JPEG without explicit metadata handling means you may be losing:
-
-original EXIF fields
-capture time
-lens/camera info
-ICC profile
-GPS if present
-copyright/author data
-
-That may be fine, but if the intent is “enhanced derivative of original photo,” I would consider:
-
-copying selected EXIF fields from source to final JPEG
-preserving or explicitly assigning ICC profile
-adding software tag / processing note
-optionally stripping privacy-sensitive fields by choice, not by accident
-
-Color profile handling is especially important. “No colour corrections” is not the same as “color managed.”
-
-4. Add resumability per stage, not just per photo
-
-Your manifest marks a photo done after full completion, which is good, but partial reruns still require redoing all remote processing for failed photos.
-
-You could get stronger resilience with stage-aware artifacts:
-
-oriented temp exists
-upload completed
-prompt submitted
-output downloaded
-JPEG written
-metadata written
-
-That might be too much overhead for a personal workflow, but even just logging prompt_id per source photo would help a lot with crash recovery.
-
-5. Treat JPEG as an output format decision, not a fixed end state
-
-JPEG quality 92 is reasonable, but for some images:
-
-foliage
-gradients in skies
-deep edits after enhancement
-
-JPEG may reintroduce artifacts after all that expensive work.
-
-Consider:
-
-archival output as PNG or TIFF
-delivery output as JPEG
-optional WebP/AVIF for web usage
-
-Even if you keep JPEG as primary, having a “master enhanced output” option would be useful.
-
-Specific comments on the custom stages
-AdaptivePhotoGrade
-
-This is the most promising custom logic in the workflow.
-
-Good:
-
-exposure in linear light
-contrast and saturation as explicit steps
-detail/base decomposition
-per-scene profiles
-
-Concerns:
-
-gamma 2.2 approximation is simple, but true sRGB transfer is not exactly 2.2
-clipping highlights at 1.0 can lose recoverable rolloff smoothness
-HSV saturation edits can behave poorly in skin tones and near highlights
-fixed midpoint contrast around 0.5 is simple but not content-aware
-
-If you keep evolving it, the next quality wins will likely come from:
-
-proper sRGB transfer functions
-luminance-aware saturation
-highlight/shadow selective controls
-local contrast constrained by noise estimate
-SkyEnhance
-
-Clever and cheap. Good for a CPU stage.
-
-Risks:
-
-blue clothing, windows, water, reflective buildings, and tinted glass can get caught
-sunset banding or haloing near trees/buildings
-vertical prior helps, but can still fail on mountains or upside-weighted compositions
-
-I would recommend logging:
-
-sky coverage %
-mean mask confidence
-whether sky enhancement was effectively skipped
-
-And maybe auto-disable when coverage is too low or too fragmented.
-
-DepthSelectiveSharpen
-
-This is an interesting stage, but also easy to overdo.
-
-Pros:
-
-more photographic than simple global sharpening
-can add subject separation
-
-Risks:
-
-relative depth is not segmentation
-hair, glasses, transparent objects, fences, and fine branches can create messy transitions
-background blur on an already naturally focused image may look synthetic
-blur-plus-sharpen in one stage can produce “smartphone portrait mode” artifacts
-
-I would strongly consider making this more conservative:
-
-lower default blur
-maybe sharpen foreground only, without explicit background blur
-or gate blur by scene type and depth confidence
-
-For many photos, foreground sharpening alone may be enough.
-
-Performance review
-
-Your breakdown is believable.
-
-The biggest performance cost drivers are probably:
-
-ESRGAN 4× inference
-memory movement around the 16K intermediate
-downscale from 16K to 4K
-Depth Anything inference
-
-This means the obvious speed/quality tradeoff lever is:
-
-reducing or conditionally skipping the 4× path
-
-That one decision could cut runtime materially.
-
-If you want better throughput later, likely gains are:
-
-batch submission queue with bounded concurrency
-reuse loaded models across jobs, which ComfyUI already helps with
-avoid oversized intermediates when not needed
-possibly move some CPU image ops to GPU if they become limiting
-
-But honestly, for 45 photos, the current runtime is already acceptable.
-
-Operational review
-
-This is better than average for reliability, but I would still tighten a few things:
-
-Add stronger failure modes
-
-Include distinct handling for:
-
-upload success but prompt submission failure
-prompt accepted but no output file
-PNG written but metadata JSON missing
-metadata JSON malformed
-partial download
-ComfyUI restart mid-run
-Record more run provenance
-
-In the metadata or manifest, store:
-
-prompt_id
-workflow hash
-custom node git commit or file hash
-model file hashes
-processing duration per stage if possible
-
-That will make future tuning much easier.
-
-Be careful with cache busting
-
-The random prefix solves ComfyUI cache problems, which is practical. But it also removes some chance of safe reuse. A better long-term version is:
-
-cache key derived from source hash + workflow hash + relevant params
-
-Then you can intentionally reuse identical results.
-
-Best improvements to do next
-
-If I were prioritizing, I would do these in order:
-
-make ESRGAN conditional
-store CLIP confidence distribution and blend scene profiles
-make depth stage more conservative
-preserve ICC/selected EXIF metadata
-improve failure recovery and provenance logging
-
-Those would give the best quality/reliability gains without rewriting the whole system.
-
-Bottom line
-
-This is a well-designed, thoughtful workflow with a good systems mindset behind it. The main weakness is not architecture — it is over-processing risk from always-on enhancement stages, especially:
-
-full-image ESRGAN
-hard scene-class selection
-depth-based blur/sharpen
-
-So my verdict is:
-
-architecture: strong
-operational design: strong
-quality strategy: promising, but should become more conditional and confidence-aware
-performance: acceptable for the workload
-next maturity step: adaptive routing and stronger provenance
-
-If you want, I can turn this into a more formal engineering review with sections for “keep / change / risks / recommended roadmap,” or I can propose a v2 pipeline that should improve both quality and speed.
-
-yes:
-
-# Photo Enhancement Workflow Review
-
-## Executive summary
-
-This workflow is well designed. It has a strong separation between orchestration, inference, deterministic image processing, and human review. The system is practical, understandable, and already close to a small production pipeline for batch photo enhancement.
-
-The main issue is not the infrastructure or code shape. The main issue is **quality control under an always-on enhancement strategy**. Several expensive stages are applied to every image, even though their benefit is scene-dependent and sometimes negative. The biggest gains now will come from making the workflow **conditional, confidence-aware, and slightly more conservative**.
-
----
-
-# What should stay
-
-## 1. Ruby as the control plane
-
-This is a good choice.
-
-It gives you:
-
-* clean batch orchestration
-* simple manifest handling
-* file lifecycle control
-* easy VM lifecycle integration
-* a place to keep business logic out of ComfyUI
-
-## 2. ComfyUI as the execution graph
-
-Also a good choice.
-
-It gives you:
-
-* model reuse
-* visual graph structure
-* easy injection of runtime parameters
-* modular custom node expansion
-
-## 3. Metadata sidecar generation
-
-This is one of the strongest parts of the system.
-
-The `_e.md` and JSON sidecars make the workflow:
-
-* debuggable
-* reviewable
-* reproducible
-* easier to tune later
-
-## 4. Human review tool
-
-The comparison tool is exactly the right final step. Enhancement pipelines often fail because they assume “processed” means “better.” Yours does not.
-
-## 5. EXIF orientation bake before upload
-
-Correct and necessary. Good defensive engineering.
-
----
-
-# What should change
-
-## 1. Stop treating enhancement as a single fixed path
-
-Right now the graph is elegant, but too uniform. The workflow should become a **decision tree**, not a single mandatory sequence.
-
-Some stages should be optional:
-
-* Real-ESRGAN
-* CodeFormer
-* SkyEnhance
-* DepthSelectiveSharpen
-* grading strength inside AdaptivePhotoGrade
-
-## 2. Make Real-ESRGAN conditional
-
-This is the highest-priority change.
-
-Current risks:
-
-* synthetic texture
-* over-crisp foliage
-* JPEG artifact amplification
-* invented microdetail
-* unnatural skin/hair
-
-### Recommendation
-
-Use ESRGAN only when:
-
-* high detail scenes (landscape, architecture)
-* strong edge density
-* visible softness or compression
-
-Avoid or weaken for:
-
-* portraits
-* night/high ISO
-* already sharp JPEGs
-
-## 3. Replace hard scene labels with blended grading
-
-Current approach uses argmax from CLIP.
-
-Problem: scenes are often mixed.
-
-### Recommendation
-
-* keep top 2–3 scene scores
-* normalize
-* blend profile parameters
-
-Example:
-
-* 0.55 landscape
-* 0.35 golden_hour
-* 0.10 overcast
-
-Blend exposure, contrast, saturation, detail, denoise.
-
-## 4. Make depth processing more conservative
-
-Default behavior should be:
-
-* foreground sharpening only
-* no background blur by default
-
-Enable blur only when:
-
-* strong subject separation
-* portrait-like composition
-
-## 5. Preserve metadata intentionally
-
-Current pipeline likely loses:
-
-* EXIF
-* ICC profile
-
-### Recommendation
-
-Preserve or explicitly manage:
-
-* capture timestamp
-* camera/lens info
-* ICC profile
-* add processing metadata
-
----
-
-# Main risks
-
-## Quality risks
-
-### Over-processing
-
-Stacked enhancements may lead to synthetic look.
-
-### Face inconsistency
-
-CodeFormer may produce mismatch with surrounding regions.
-
-### Masking errors
-
-Sky and depth masks may:
-
-* misclassify regions
-* create halos
-
-## Operational risks
-
-### Partial success ambiguity
-
-Need stronger validation for:
-
-* missing metadata
-* partial downloads
-
-### Weak provenance
-
-Should log:
-
-* prompt_id
-* workflow hash
-* model versions
-
-### CPU bottleneck
-
-Potential hotspots:
-
-* large rescaling
-* guided filtering
-* morphology operations
-
----
-
-# Performance review
-
-## Current state
-
-~40–50s/photo is acceptable.
-
-## Main optimization lever
-
-Make ESRGAN conditional.
-
-## Secondary lever
-
-Skip unnecessary stages when not needed.
-
----
-
-# Recommended v2 architecture
-
-## Goal
-
-Make workflow adaptive.
-
-## Pipeline
-
-### Stage 0 — Preflight analysis
-
-Compute:
-
-* brightness histogram
-* saturation
-* edge density
-* noise estimate
-* face stats
-* sky coverage
-* CLIP scores
-
-### Stage 1 — Policy selection
-
-Decide:
-
-* ESRGAN mode
-* CodeFormer usage
-* grading blend
-* sky enhance on/off
-* depth mode
-
-### Stage 2 — Enhancement
-
-Run only selected stages.
-
-### Stage 3 — Output + metadata
-
-Include:
-
-* policy decisions
-* confidence scores
-* timings
-
----
-
-# Example metadata (v2)
-
-```json
-{
- "workflow_version": "photo-enhance-v2",
- "analysis": {
- "scene_scores": {
- "landscape": 0.51,
- "golden_hour": 0.28
- },
- "face_count": 1,
- "sky_coverage_pct": 23.4
- },
- "policy": {
- "esrgan_mode": "weak",
- "depth_mode": "sharpen_only"
- }
-}
-```
-
----
-
-# Roadmap
-
-## Phase 1
-
-* conditional ESRGAN
-* blended scene grading
-* disable background blur default
-* preserve metadata
-
-## Phase 2
-
-* preflight analysis
-* gating logic for faces and sky
-* improved logging
-
-## Phase 3
-
-* better color handling (true sRGB)
-* noise-aware detail
-* improved saturation logic
-
----
-
-# Final verdict
-
-## Strengths
-
-* strong architecture
-* practical workflow
-* good separation of concerns
-
-## Weakness
-
-* over-processing risk from always-on stages
-
-## Key improvement
-
-Move from fixed pipeline → adaptive pipeline
-
-This will improve both quality and performance significantly.
-
diff --git a/photo-enhance.rb b/photo-enhance.rb
deleted file mode 100755
index 2d6b8de..0000000
--- a/photo-enhance.rb
+++ /dev/null
@@ -1,581 +0,0 @@
-#!/usr/bin/env ruby
-# frozen_string_literal: true
-
-# photo-enhance.rb — AI photo enhancer via ComfyUI on a Hyperstack GPU VM.
-#
-# Submits images from --indir to the ComfyUI REST API, downloads the AI-enhanced
-# results and saves alongside the originals with an _e suffix. Also downloads
-# a per-photo JSON metadata file written by the WritePhotoMetadata ComfyUI node
-# and converts it to a human-readable .md report alongside each enhanced photo.
-#
-# AI pipeline (ComfyUI, GPU):
-# 1. Real-ESRGAN realesr-general-x4v3 — 4× upscale at full 4K input, AI denoise
-# 2. CodeFormer fidelity=0.7 — neural face restoration
-# 3. CLIP ViT-B/32 — scene classification (portrait/landscape/…)
-# 4. AdaptivePhotoGrade — scene-tuned exposure/contrast/saturation/detail
-# 5. SkyEnhance — HSV sky mask + graduated sky correction
-# 6. Depth Anything V2 Small — depth map → foreground sharp, background soft
-#
-# Usage:
-# ruby photo-enhance.rb --config hyperstack-vm-photo.toml \
-# --indir ~/Pictures [--watch] [--workflow workflows/photo-enhance.json]
-#
-# Requirements:
-# - ComfyUI VM: ruby hyperstack.rb --config hyperstack-vm-photo.toml create
-# - WireGuard tunnel active (wg1)
-
-begin
- require 'bundler/setup'
-rescue LoadError, Gem::GemNotFoundException, Gem::LoadError, Errno::ENOENT
- nil
-end
-
-require 'json'
-require 'net/http'
-require 'optparse'
-require 'fileutils'
-require 'digest'
-require 'time'
-require 'set'
-
-begin
- require 'toml-rb'
-rescue LoadError
- warn "Missing dependency: toml-rb. Run `bundle install` in #{__dir__} first."
- exit 2
-end
-
-# ---------------------------------------------------------------------------
-# Config
-# ---------------------------------------------------------------------------
-
-class PhotoConfig
- attr_reader :host, :port, :workflow_path
-
- def initialize(config_path, workflow_path_override)
- raw = TomlRB.load_file(File.expand_path(config_path))
- hostname = raw.dig('vm', 'hostname') || 'hyperstack-photo'
- interface = raw.dig('local_client', 'interface_name') || 'wg1'
- @host = "#{hostname}.#{interface}"
- @port = Integer(raw.dig('comfyui', 'port') || 8188)
- @workflow_path = workflow_path_override ||
- File.join(File.dirname(File.expand_path(config_path)), 'workflows', 'photo-enhance.json')
- end
-end
-
-# ---------------------------------------------------------------------------
-# ComfyUI API client — upload, submit, poll, download.
-# ---------------------------------------------------------------------------
-
-class ComfyUIClient
- POLL_INTERVAL_SEC = 2
- POLL_TIMEOUT_SEC = 300 # 5 minutes; ESRGAN is fast on GPU
- # When ComfyUI crashes (OOM), systemd restarts it in ~30s.
- # We poll until reachable again, up to this many seconds total.
- RECOVERY_TIMEOUT_SEC = 300
- RECOVERY_POLL_SEC = 10
-
- def initialize(host:, port:, out: $stdout)
- @host = host
- @port = port
- @out = out
- end
-
- def upload_image(file_path)
- filename = File.basename(file_path)
- image_data = File.binread(file_path)
- boundary = "----RubyPhotoEnhance#{hex(8)}"
- body = [
- "--#{boundary}\r\n",
- "Content-Disposition: form-data; name=\"image\"; filename=\"#{filename}\"\r\n",
- "Content-Type: #{mime_type(file_path)}\r\n\r\n",
- image_data,
- "\r\n--#{boundary}\r\n",
- "Content-Disposition: form-data; name=\"overwrite\"\r\n\r\ntrue\r\n",
- "--#{boundary}--\r\n"
- ].join
- resp = post_raw('/upload/image', body, "multipart/form-data; boundary=#{boundary}")
- raise "Upload failed (#{resp.code}): #{resp.body}" unless resp.code == '200'
- JSON.parse(resp.body)['name'] || filename
- rescue Errno::ECONNREFUSED, Errno::EHOSTUNREACH, SocketError => e
- raise "Cannot reach ComfyUI at #{@host}:#{@port} — is WireGuard active? (#{e.message})"
- end
-
- def submit_prompt(workflow)
- resp = post_json('/prompt', { 'prompt' => workflow })
- raise "Prompt failed (#{resp.code}): #{resp.body}" unless resp.code == '200'
- JSON.parse(resp.body)['prompt_id'] or raise "No prompt_id in: #{resp.body}"
- end
-
- def wait_for_output(prompt_id)
- deadline = Time.now + POLL_TIMEOUT_SEC
- loop do
- raise "Timed out after #{POLL_TIMEOUT_SEC}s for #{prompt_id}" if Time.now > deadline
-
- resp = get("/history/#{prompt_id}")
- raise "History poll failed (#{resp.code})" unless resp.code == '200'
-
- result = JSON.parse(resp.body)[prompt_id]
- if result
- outputs = extract_filenames(result)
- return outputs unless outputs.empty?
-
- # ComfyUI cached the run (identical inputs) and wrote no new files — bail fast.
- status = result.dig('status', 'status_str')
- raise "ComfyUI cached execution returned no outputs for #{prompt_id}" \
- if result.dig('status', 'completed') && status == 'success'
- end
-
- sleep POLL_INTERVAL_SEC
- end
- end
-
- def download_output(filename, dest_path)
- resp = get("/view?filename=#{URI.encode_www_form_component(filename)}&type=output&subfolder=")
- raise "Download failed (#{resp.code}) for #{filename}" unless resp.code == '200'
- FileUtils.mkdir_p(File.dirname(dest_path))
- File.binwrite(dest_path, resp.body)
- end
-
- def check_connectivity!
- resp = get('/system_stats')
- raise "Health check failed (#{resp.code}): #{resp.body}" unless resp.code == '200'
- rescue Errno::ECONNREFUSED, Errno::EHOSTUNREACH, SocketError => e
- raise "Cannot reach ComfyUI at #{@host}:#{@port} — is WireGuard active? (#{e.message})"
- end
-
- # Polls ComfyUI until it responds again (or times out).
- # Called automatically when an upload or submit fails with a connection error.
- # ComfyUI crashes on OOM (large ESRGAN tensors) and systemd restarts it in ~30s.
- # Returns true if recovered, raises on timeout.
- def wait_for_recovery
- @out.puts " ComfyUI unreachable — waiting for restart (up to #{RECOVERY_TIMEOUT_SEC}s)..."
- deadline = Time.now + RECOVERY_TIMEOUT_SEC
- start = Time.now
- loop do
- raise "ComfyUI did not recover within #{RECOVERY_TIMEOUT_SEC}s — giving up" if Time.now > deadline
-
- sleep RECOVERY_POLL_SEC
- begin
- get('/system_stats')
- @out.puts ' ComfyUI recovered — resuming'
- return true
- rescue Errno::ECONNREFUSED, Errno::EHOSTUNREACH, SocketError, Net::OpenTimeout
- @out.puts " still waiting... (#{(Time.now - start).round}s elapsed)"
- end
- end
- raise "ComfyUI did not recover within #{RECOVERY_TIMEOUT_SEC}s — giving up"
- end
-
- private
-
- def extract_filenames(result)
- Array(result.dig('outputs'))
- .flat_map { |_id, node| Array(node['images']) }
- .map { |img| img['filename'] }
- .compact.reject(&:empty?)
- end
-
- def get(path)
- uri = URI("http://#{@host}:#{@port}#{path}")
- Net::HTTP.start(uri.host, uri.port, open_timeout: 10, read_timeout: 120) { |h| h.get(uri) }
- end
-
- def post_json(path, payload)
- uri = URI("http://#{@host}:#{@port}#{path}")
- req = Net::HTTP::Post.new(uri)
- req['Content-Type'] = 'application/json'
- req.body = JSON.generate(payload)
- Net::HTTP.start(uri.host, uri.port, open_timeout: 10, read_timeout: 120) { |h| h.request(req) }
- end
-
- def post_raw(path, body, content_type)
- uri = URI("http://#{@host}:#{@port}#{path}")
- req = Net::HTTP::Post.new(uri)
- req['Content-Type'] = content_type
- req.body = body
- Net::HTTP.start(uri.host, uri.port, open_timeout: 10, read_timeout: 120) { |h| h.request(req) }
- end
-
- def mime_type(path)
- case File.extname(path).downcase
- when '.jpg', '.jpeg' then 'image/jpeg'
- when '.png' then 'image/png'
- when '.webp' then 'image/webp'
- else 'application/octet-stream'
- end
- end
-
- def hex(n)
- Digest::SHA256.hexdigest(Time.now.to_f.to_s + rand.to_s)[0, n * 2]
- end
-end
-
-# ---------------------------------------------------------------------------
-# Manifest — avoids re-processing files across runs and in watch mode.
-# ---------------------------------------------------------------------------
-
-class ProcessedManifest
- FILE_NAME = '.photo-enhance-processed'
-
- def initialize(dir)
- @path = File.join(dir, FILE_NAME)
- @entries = load_entries
- end
-
- def processed?(file_path)
- @entries.include?(digest(file_path))
- end
-
- def mark_done(file_path)
- key = digest(file_path)
- @entries << key
- File.open(@path, 'a') { |f| f.puts(key) }
- end
-
- private
-
- def load_entries
- return Set.new unless File.exist?(@path)
- File.readlines(@path, chomp: true).map(&:strip).reject(&:empty?).to_set
- end
-
- # Covers basename + size + mtime so a re-shot of the same filename re-processes.
- def digest(file_path)
- stat = File.stat(file_path)
- Digest::SHA256.hexdigest("#{File.basename(file_path)}:#{stat.size}:#{stat.mtime.to_i}")
- rescue Errno::ENOENT
- Digest::SHA256.hexdigest(File.basename(file_path))
- end
-end
-
-# ---------------------------------------------------------------------------
-# Enhancer — orchestrates upload → AI → download → colour correct per image.
-# ---------------------------------------------------------------------------
-
-class PhotoEnhancer
- SUPPORTED_EXTENSIONS = %w[.jpg .jpeg .png .webp .raf .cr2 .cr3 .nef .arw .dng .rw2].freeze
- RAW_EXTENSIONS = %w[.raf .cr2 .cr3 .nef .arw .dng .rw2].freeze
-
- # No colour corrections — pure AI output from Real-ESRGAN is used as-is.
- # ImageMagick is only used to bake EXIF rotation and convert PNG→JPEG.
- COLOR_ARGS = [].freeze
-
- def initialize(config:, client:, workflow:, indir:, manifest:, out: $stdout)
- @config = config
- @client = client
- @workflow = workflow
- @indir = indir
- @manifest = manifest
- @out = out
- end
-
- def run(watch: false)
- @client.check_connectivity!
- @out.puts "ComfyUI ready at http://#{@config.host}:#{@config.port}"
- @out.puts "Enhancing photos in #{@indir}"
- @out.puts watch ? '(watch mode — Ctrl-C to stop)' : ''
-
- loop do
- find_pending.each { |path| enhance_one(path) }
- break unless watch
- sleep 5
- end
- end
-
- private
-
- def find_pending
- Dir.glob(File.join(@indir, '*'))
- .select { |f| File.file?(f) && SUPPORTED_EXTENSIONS.include?(File.extname(f).downcase) }
- .reject { |f| File.basename(f, '.*').end_with?('_e') }
- .reject { |f| File.basename(f).include?('.orient.') } # temp decode files (.orient.tiff etc.)
- .reject { |f| @manifest.processed?(f) }
- .sort
- end
-
- def enhance_one(src_path)
- ext = File.extname(src_path).downcase
- basename = File.basename(src_path, File.extname(src_path))
- # RAW files are always output as JPEG — there is no enhanced RAW format.
- out_ext = RAW_EXTENSIONS.include?(ext) ? '.jpg' : ext
- dest_path = File.join(File.dirname(src_path), "#{basename}_e#{out_ext}")
-
- @out.puts "[#{Time.now.strftime('%H:%M:%S')}] #{File.basename(src_path)}"
-
- # Bake in EXIF rotation before uploading — ComfyUI strips EXIF metadata.
- upload_path = auto_orient_tempfile(src_path)
-
- begin
- retried = false
- begin
- uploaded_name = @client.upload_image(upload_path)
- workflow = inject_input(@workflow, uploaded_name)
- prompt_id = @client.submit_prompt(workflow)
- @out.puts " prompt #{prompt_id}"
-
- filenames = @client.wait_for_output(prompt_id)
- raise "No outputs returned for #{src_path}" if filenames.empty?
- rescue RuntimeError => e
- # On connection refused (ComfyUI crashed / OOM), wait for systemd to restart
- # it and retry this photo once. Any other error propagates immediately.
- if !retried && e.message.include?('Cannot reach ComfyUI')
- retried = true
- @client.wait_for_recovery
- retry
- end
- raise
- end
-
- # ComfyUI outputs PNG; download then convert to output format.
- tmp_png = "#{dest_path}.tmp.png"
- @client.download_output(filenames.first, tmp_png)
- save_with_corrections(tmp_png, dest_path, out_ext)
-
- # Restore original EXIF metadata onto the enhanced JPEG.
- # ComfyUI strips all EXIF when it processes the image; this brings back
- # capture time, camera/lens info, ICC profile, and GPS coordinates.
- copy_exif(src_path, dest_path)
-
- # Download the JSON metadata written by WritePhotoMetadata and render it
- # as a human-readable .md report alongside the enhanced photo.
- # ComfyUI appends _NNNNN_ counter: "enhanced_abc123__00001_.png" → "enhanced_abc123_"
- prefix = filenames.first.sub(/_\d+_\.png$/, '')
- meta_file = "#{prefix}meta.json"
- md_path = File.join(File.dirname(dest_path),
- "#{File.basename(dest_path, File.extname(dest_path))}.md")
- download_and_write_