Merge pull request 'docs: Art Studio groups + focus dialog; photo stacking is desktop-only; stack→concept' (#1149) from docs/art-studio-groups-stacker-out into main

Reviewed-on: #1149
This commit was merged in pull request #1149.
This commit is contained in:
spikerj committed 2026-09-25 17:23:48 +00:00
commit 80526556d1
1 file changed
+13 -15
+13 -15
View File
@@ -308,7 +308,7 @@ The **Art Studio** is SpikerSoft's AI-assisted asset creation suite: students an
### What a user can do
The shell (`art-pipe-shell`) is a five-view app routed at `/tools/(tools:art-studio)` behind AuthGuard: **Pipeline · Library · Metrics · Builder · Photo Stack**.
The shell (`art-pipe-shell`) is a five-view app routed at `/tools/(tools:art-studio)` behind AuthGuard: **Concepts · Pipeline · Library · Metrics · Builder**. Art Studio is for fictional 2D/3D art; photo work (stacking, touch-up) lives in the [Desktop Stacker](#desktop-stacker-spikersoft-stacker) and the [Photography gallery](#photography--gallery).
- **Generate an asset** from a text prompt, an uploaded image, or both (`art-prompt-composer`). The composer exposes a per-stage "git rail" editor (concept → … → enrichment) with auto/manual/skip badges, per-stage model pickers, and workflow presets. Generation modes: **Generate**, **Batch**, **Preset Batch**, **Continuous**, and **Coverage Matrix** (the multi-asset modes are client-side fan-out over the single-asset submit endpoint). A distinct **Text → 3D** path skips the concept stage entirely (Shap-E / Hunyuan3D direct).
- **Pick a concept candidate** — the asset parks in `AwaitingSelection`, the concept stage's candidate images (typically 6) render as a grid, and `POST api/artstudio/{id}/select-concept` resumes the pipeline. Unpicked candidates stay reproducible (per-image seed recorded in provenance).
@@ -316,11 +316,11 @@ The shell (`art-pipe-shell`) is a five-view app routed at `/tools/(tools:art-stu
- **Create material variants** — `POST {id}/material-variant` runs a "Material Variation (Paint 2.1)" retexture that preserves the source topology/UVs and produces a **sibling asset** (blobs copied, `VariantOfAssetId` provenance) — e.g. "the same knight, but polished gold".
- **Watch live progress** — stage timeline with state chips, percent, queue position ("you are Nth in line", published by the GPU coordinator), and per-stage phase text. Delivered over SignalR (`hubs/notifications`, `art.asset.lifecycle` events) and degrades to 4-second polling.
- **View results in-browser** — a three.js GLB viewer with skeleton/animation-clip playback and PBR channel switching (`art-asset-viewer`).
- **Manage a personal library** — search/status/kind/tag filters (deep-linkable query params), cancel, delete, **restart from any stage**, download.
- **Manage a personal library** — search/status/kind/tag filters (deep-linkable query params), cancel, delete, **restart from any stage**, download. Tiles are uniform (a fixed five-slot action bar; an error line never shifts the row) and **Open** takes over the screen in a focus dialog (`?asset=` deep link, Escape/Back closes, focus and scroll return to the tile you opened).
- **Export** — per-artifact downloads, a streamed **ZIP bundle** (`GET api/artstudio/{id}/bundle`: mesh + textures + manifest), and a **games-handoff manifest** (`GET {id}/manifest`).
- **Share, remix, rate** — assets can be submitted to a class gallery (`art-gallery`), where staff moderate submissions and a flagged-content review queue (`/admin/art-review`, `/admin/art-gallery-review`). Approved gallery assets can be **remixed** by other users (`RemixOfAssetId` provenance) and rated.
- **Build chess piece sets** — one theme fans out into six per-piece prompts (`api/artstudio/chess-sets`); sets can be shared, moderated, and **equipped** into the chess game. See [Asset-to-Game Integration](#asset-to-game-integration).
- **Photo Stack (focus stacking)** — pick 2–60 frames from the photo gallery (grouped by upload session), choose a fusion engine, watch `develop → align_fuse` run, and get a fused JPEG preview plus 16-bit TIFF downloads. **Photo Touch-Up** then offers Zerene-style retouching: paint aligned source-frame pixels through a soft brush over the fused composite; the result saves back to the photo gallery as a new Touchup photograph (the original is never modified).
- **Group assets** — an `ArtAssetGroup` is a folder-like card inside My Assets (`api/artstudio/groups`; kinds `ChessSet` and `Custom`, e.g. a monster set for a dungeon), with a slot template and members you click into. A **chess-set** group fans one theme out into six per-piece prompts; its slot keys equal the game loadout slot ids, so a set can be shared, moderated, and **equipped** into the chess game. See [Asset-to-Game Integration](#asset-to-game-integration).
- **Use a published stack as a concept** — a stack published from the Desktop Stacker (a *Stacked* photograph in the gallery) can be imported as a Completed concept (`POST api/artstudio/concepts/from-photograph`; lightbox **Use as concept** or the concepts wall's **From my photos**) and then edited and **promoted** to a 3D model like any other concept. In-browser focus stacking and touch-up were retired on 2026-09-25 — the Desktop Stacker is the only stacker.
- **Staff/admin extras** — a d3 metrics dashboard (`/admin/art-metrics`), a live "Now Running" activity board, a **visual Pipeline Builder** node graph that saves as a workflow preset (`/admin/art-builder`), preset management (`/admin/art-presets`), and a usage meter.
Two adjacent GPU lanes are *not* Art Studio stages but ride the same worker fleet: **QR Art** (`api/qrart` — stylized QR code generation, `classic` SD1.5 QR Monster v2 and `sdxl` variants; the anonymous client-side QR tool stays anonymous, this authenticated lane adds the AI stylization) and **photo auto-tagging** (Florence-2 tags gallery photographs; see [Photography & Gallery](#photography--gallery)).
@@ -348,7 +348,7 @@ The Python side is three processes over HTTP (see `spikersoft-artpipe/CLAUDE.md`
- **Rigging fallback chain**: UniRig → RigNet → Mesh2Rig → smart_rig.
- **Enrichment** normalizes meshes, renders thumbnails / MP4/GIF animation previews / multi-view screenshots, and runs CV analysis (symmetry, silhouette, form, color auto-tags).
- **Blender daemon pool**: persistent headless Blender HTTP servers (ports 19200+) with session leasing, auto-recycle after 50 batches, and stale-lease reclamation.
- Each model backend lives in its own venv with an `artpipe.json` manifest as the contract. Current manifest inventory spans text-to-image (SDXL Lightning/Turbo, Flux Schnell), image editing (FLUX.2 Klein, Qwen-Image-Edit), image→3D (TripoSR, TripoSG, SF3D, InstantMesh, Pixal3D, Hunyuan3D, Hunyuan3D-Omni, Trellis-mac), text→3D (Shap-E, Hunyuan3D), texturing (Hunyuan3D-Paint 2.0/2.1, SD-Turbo-Tex), rigging (UniRig, RigNet), motion (MDM), pose (OpenPose), photo (photostack), QR stylization (QR Monster), tagging (Florence-2), and safety (safety_check).
- Each model backend lives in its own venv with an `artpipe.json` manifest as the contract. Current manifest inventory spans text-to-image (SDXL Lightning/Turbo, Flux Schnell), image editing (FLUX.2 Klein, Qwen-Image-Edit), image→3D (TripoSR, TripoSG, SF3D, InstantMesh, Pixal3D, Hunyuan3D, Hunyuan3D-Omni, Trellis-mac), text→3D (Shap-E, Hunyuan3D), texturing (Hunyuan3D-Paint 2.0/2.1, SD-Turbo-Tex), rigging (UniRig, RigNet), motion (MDM), pose (OpenPose), QR stylization (QR Monster), tagging (Florence-2), and safety (safety_check).
### Backend orchestration (SpikerSoft.EventHandlers.ArtPipeProcessor)
@@ -360,7 +360,7 @@ The .NET GPU worker that shells out to art_pipe. Three mutually exclusive execut
| **Resident** | `ArtPipe:ResidentModel` | Keeps one model in VRAM for the container lifetime; consumes `art.model.<modeldir>.tasks`; recycles after N jobs (e.g. SDXLLightning 500) |
| **Model-queue** | `ArtPipe:ModelQueue` | Same per-model queue, but a short-lived subprocess + per-job lease — an **idle container holds zero VRAM**. This is the production doctrine |
Routing is API-side via `ArtStudio:StageModelMap` (production defaults: `modeling → Pixal3D`, `texturing → Hunyuan3DPaint21`) with worker-side `ArtPipe:PublishToModelQueues`. Resilience pieces: a stage-run watchdog, capped failover down the per-stage method list (`ArtStudioStageFailoverOptions` — `PinnedOnly` methods like the edit models and material variation are excluded from failover chains and generation pickers), and display-derivative generation (400 px AVIF thumbnail + display AVIF via the shared `cavif` wrapper; TIFF path for PhotoStack). A separate single-replica `ArtStudioMetrics` handler folds lifecycle events into the metrics read model.
Routing is API-side via `ArtStudio:StageModelMap` (production defaults: `modeling → Pixal3D`, `texturing → Hunyuan3DPaint21`) with worker-side `ArtPipe:PublishToModelQueues`. Resilience pieces: a stage-run watchdog, capped failover down the per-stage method list (`ArtStudioStageFailoverOptions` — `PinnedOnly` methods like the edit models and material variation are excluded from failover chains and generation pickers), and display-derivative generation (400 px AVIF thumbnail + display AVIF via the shared `cavif` wrapper). A separate single-replica `ArtStudioMetrics` handler folds lifecycle events into the metrics read model.
### GPU coordinator & VRAM leases
@@ -370,7 +370,7 @@ Routing is API-side via `ArtStudio:StageModelMap` (production defaults: `modelin
- VRAM-aware concurrent grants with model-affinity scheduling; **queue-position publishing** feeds the UI's "Nth in line".
- A **durable Redis ledger** reloads booked VRAM + active leases on restart so a coordinator bounce can't double-grant into GPUs workers still hold.
- Deliberately **replicas: 1** (in-memory tracker is authoritative), pinned to node `SERVER`, HTTP `/health` + `/gpu/status` on port 8090 (not exposed through Traefik).
- **Two permanent GPU lanes**: the **4090** node (RTX 4090, 24,576 MB budget, permanent holder of the `artpipe-gpu` placement label — runs the artpipe model stacks and image description) and **SERVER** (RTX 3070 Ti, 8,192 MB — runs embeddings, quiz generation, and the CPU-only PhotoStack lane). Lease requests carry `requiredNodeId` so a grant is budgeted against the requester's own card.
- **Two permanent GPU lanes**: the **4090** node (RTX 4090, 24,576 MB budget, permanent holder of the `artpipe-gpu` placement label — runs the artpipe model stacks and image description) and **SERVER** (RTX 3070 Ti, 8,192 MB — runs embeddings and quiz generation). Lease requests carry `requiredNodeId` so a grant is budgeted against the requester's own card.
### Per-model container lanes (spikersoft-infrastructure)
@@ -396,14 +396,13 @@ Each model runs as its own Swarm stack with fully **baked weights** (`HF_HUB_OFF
| `-model-safety` | resident SafetyCheck gate | ~1,024 |
| `-model-qrmonster` | QR Art lane (on-demand, replicas 0 when idle) | 9,500 |
| `-model-florence2` | photo auto-tag lane (pinned to SERVER; replicas 0 when idle) | 2,048 |
| `photostack` | `develop` + `align_fuse`, **CPU-only** (`BypassGpuLease=true`, pinned to SERVER) | 0 |
VRAM figures are lease sizes (torch *reserved*, not allocated — measured on live runs, not taken from upstream quotes).
### Storage & data
- **MinIO buckets**: `art-asset-artifacts` (all pipeline artifacts — concept images, meshes, PBR texture maps, rigs, animations, exports, AVIF derivatives) and `art-asset-quarantine` (safety-flagged, staff-only). Artifacts are referenced by MinIO-native object keys — **not** GridFS.
- **Mongo collections**: `art-assets` (owner, type, prompt, stage plan, artifacts + selections, safety flags, gallery status, remix/variant/chess-set provenance, method selections + provenance, ratings), `art-asset-stage-runs` (per-run state/timings/retries), `chess-piece-sets`, `darkroom-develop-jobs`, `art-variant-upload-jobs` (ClamAV-scanned user uploads), `qr-art-jobs`, `art-studio-audit`, `art-workflow-presets`, `game-art-loadouts`, `art-studio-metrics`.
- **Mongo collections**: `art-assets` (owner, type, prompt, stage plan, artifacts + selections, safety flags, gallery status, remix/variant/group/photograph provenance, method selections + provenance, ratings), `art-asset-stage-runs` (per-run state/timings/retries), `art-asset-groups`, `art-variant-upload-jobs` (ClamAV-scanned user uploads), `qr-art-jobs`, `art-studio-audit`, `art-workflow-presets`, `game-art-loadouts`, `art-studio-metrics`.
- Artifact **kinds** are extension-aware (`ArtifactKinds.cs`): images `png|jpeg|webp|tiff`, meshes `glb|gltf|obj|fbx|ply|stl|usdz`, video `mp4|webm`; texturing images classify into a PBR map taxonomy. The only wired export action today is **`export_glb`** (Blender) — the wider mesh-format list is the classifier's vocabulary, not a user-facing export menu.
- Tracing propagates end-to-end: the API's stage-request publish carries trace context through the .NET worker into the Python subprocess (`JAEGER_ENDPOINT`), so one Jaeger trace covers submit → GPU job → artifact upload.
@@ -930,7 +929,7 @@ Note: this repo currently has **no CI** — in-engine test scripts only (`tests/
## Desktop Stacker (spikersoft-stacker)
A **local, offline-first** focus-stacking application (Zerene/Helicon class) for macro photography — one self-contained executable per OS (macOS / Windows / Linux). It ports the platform's PhotoStack pipeline (artpipe `align.py` + `fusion.py`) to a fast desktop workflow; algorithm parity with the Python implementation is verified by a test harness in the repo.
A **local, offline-first** focus-stacking application (Zerene/Helicon class) for macro photography — one self-contained executable per OS (macOS / Windows / Linux). It began as a port of the platform's former PhotoStack pipeline (artpipe `align.py` + `fusion.py`, retired 2026-09-25) to a fast desktop workflow; algorithm parity with the Python implementation is verified by a test harness in the repo.
### Workflow stages
@@ -948,7 +947,7 @@ A Rust workspace of four crates: `stacker-core` (the pipeline — no UI deps; `r
**Free app, optional account.** Stacker is free — nothing to buy, no registration key, and every stage above works offline without an account. **Account → Sign in with spikersoft.com** (the website's Keycloak SSO session, via a loopback PKCE flow) links the app to a spikersoft.com account, which unlocks **Publish to spikersoft.com** (source frames + finished stack with provenance and Darwin Core identification into the user's gallery, optional blog post, optional public share page at `/p/{token}`) and removes the small `spikersoft.com/stacker` credit that unlinked copies add to exported **share-set slides**. That credit points at the free app page, never at donations, and the finished TIFF / JPEG export never carries it. The public [`/stacker`](https://spikersoft.com/stacker) page lists per-platform installers proxied by `api.spikersoft.com` from the repo's rolling Gitea Release (`latest`).
Beyond publishing, the app shares *algorithms* with the platform's [Photo Stack](#art-studio) lane, not services: no S3 client, and every pipeline stage runs locally.
Beyond publishing, the app is self-contained: no S3 client, and every pipeline stage runs locally. It is the platform's only stacker since 2026-09-25 (the in-browser Photo Stack lane and its artpipe model were retired); a published stack re-enters the web side as an Art Studio *concept* (see [Art Studio](#art-studio)).
---
@@ -1632,7 +1631,7 @@ Completed online games (checkmate, resignation, draw, abandonment) are persisted
### Custom Piece Sets (Art Studio)
Chess pieces can be replaced with **Art-Studio-generated 3D sets**: one theme prompt fans out into six per-piece generations (`api/artstudio/chess-sets`), the finished set can be shared to the gallery and moderated, and an approved set is **equipped** into the game via the per-game art loadout (`api/game/chess/art-loadout`, 12 slots — 6 per side). The 3D board hot-swaps piece prototypes with byte-budget enforcement and falls back to the bundled classic mesh if a custom model is too large or fails to load. See [Asset-to-Game Integration](#asset-to-game-integration).
Chess pieces can be replaced with **Art-Studio-generated 3D sets**: a chess-set **asset group** fans one theme prompt out into six per-piece generations (`api/artstudio/groups`, kind `ChessSet`; slot keys `piece:{kind}` equal the loadout slot ids), the finished set can be shared to the gallery and moderated, and an approved set is **equipped** into the game via the per-game art loadout (`api/game/chess/art-loadout`, 12 slots — 6 per side). The 3D board hot-swaps piece prototypes with byte-budget enforcement and falls back to the bundled classic mesh if a custom model is too large or fails to load. See [Asset-to-Game Integration](#asset-to-game-integration).
### Key Files
@@ -2193,7 +2192,7 @@ A personal photo gallery built for **real camera workflows** — per-frame uploa
### Gallery
`libraries/features/photo-gallery/` renders three views: a **thumbnail wall**, **cards**, and a **geotagged map** (`photo-map`), with a lightbox. The gallery is **stack-aware**: frames consumed by a Stacked or Touchup photograph (from the Art Studio [Photo Stack](#art-studio) lane) collapse behind a representative **burst tile** (grouped by upload session), and a provenance endpoint returns the source frames in burst order (`GetPhotographProvenance`). A photo-info dialog exposes sanitized metadata.
`libraries/features/photo-gallery/` renders three views: a **thumbnail wall**, **cards**, and a **geotagged map** (`photo-map`), with a lightbox. The gallery is **stack-aware**: frames consumed by a Stacked or Touchup photograph (published from the [Desktop Stacker](#desktop-stacker-spikersoft-stacker)) collapse behind a representative **burst tile** (grouped by upload session), and a provenance endpoint returns the source frames in burst order (`GetPhotographProvenance`). A photo-info dialog exposes sanitized metadata.
### Tags
@@ -2205,10 +2204,9 @@ A personal photo gallery built for **real camera workflows** — per-frame uploa
The profile's **Photography** tab shows the user's cameras and lenses — derived **server-side from EXIF serial numbers** of their uploads (`GET api/photography/gear`); nothing is self-reported.
### Pipeline & Darkroom Companion
### Pipeline
- `SpikerSoft.EventHandlers.PhotographProcessor` runs the async photo pipeline: receive → ClamAV scan → metadata extract → move to final storage (MinIO).
- **Darkroom Companion** (`DarkroomController`, Mongo `darkroom-develop-jobs`): a staff-only desktop-agent lane. A machine with Adobe installed polls for parked PhotoStack develop jobs, runs the real **Camera Raw** develop recipe locally, and uploads 16-bit TIFFs back — the pipeline then resumes at `align_fuse`. This exists because faithful RAW development is the one step the server fleet can't do.
### Clipboard paste