[ArtStudio] ProArt capability parity — user/predefined workflows, multi-method stages, full PBR map sets, rig-method chain, curation (epic #346 phase 2) #378

Open
opened 2026-07-05 03:17:05 +00:00 by spikerj · 4 comments
Owner

Joey (2026-07-05): Eric's original was more capable than what the platform exposes — multiple model-generation methods, texturing with map production, multiple rigging methods via user-defined or predefined workflows, each operation an eventhandler (.py where best). Correct. The engine parity is nearly complete; the platform exposure was deliberately narrowed for the student MVP. This ticket restores the breadth deliberately.

Honest parity matrix (engine vs platform-exposed today)

Capability ProArt engine (in spikersoft-artpipe, works via its own server/batch UI today) Platform exposes
Modeling 7 methods (TRELLIS, Hunyuan3D-2, SF3D, TripoSR, TripoSG, InstantMesh, Shap-E), per-submission choice + failover 1 fixed per deployment (StageSettings)
Concept 4 backends + failover candidates 1 (sdxl_lightning)
Texturing/maps Hunyuan3DPaint full PBR (baseColor/metallic/roughness), SD-Turbo-tex, TRELLIS retexture; normal/ORM outputs prompt-only PNG (sd_turbo_tex); export accepts normal/orm params nothing produces them
Rigging method selection + fallback chain UniRig→RigNet→Mesh2Rig→smart_rig (rig_model config = the "user-defined workflow" seam) smart_rig only
Animation MDM/HY-Motion text-to-motion, OpenPose/MediaPipe mocap, retargeting procedural presets (licensing-gated; deliberate)
Workflows presets (saved pipeline configs), projects w/ engine targets, multi-take, continuous mode fixed Character/Prop plans
Curation library, ratings, CV gates, takes enrichment metrics stored, no curation UI
Operation shape per-model venv .py workers, NDJSON protocol same protocol preserved; #368 = one resident worker per model — Eric's shape, formalized

Phases

  • W1 — per-stage method choice: submission carries optional per-stage model within profile allowed_models (backend plumbing largely exists via StageSettings/model queues); composer "advanced" panel (staff first, students later behind a flag). Server-side failover chain per stage (ProArt's concept-failover pattern generalized).
  • W2 — workflow presets: named pipeline configs (stage→method map + params + prompt prefix/suffix) as platform docs (Mongo), predefined by staff, selectable at submit — the 1:1 of ProArt presets; project grouping later.
  • W3 — full PBR/texture map sets: Hunyuan3DPaint on CUDA deployments (+ #358 Metal port for Mac) producing baseColor/metallic/roughness (+normal/ORM where supported); export stage already accepts the params; artifact kinds + viewer channels (art_pipe's own viewer has PBR channel switching to mirror).
  • W4 — rigging method chain: expose rig_model (auto/unirig/rignet/mesh2rig/smart_rig) through StageSettings params; deploy UniRig/RigNet residents on CUDA (#368 R4 pattern); record which rigger succeeded (P4 ticket already asked for this).
  • W5 — Python-native eventhandlers where best: per Eric's design, allow a resident model service to BE a .py process directly (the original Trellis3D.Python eventhandler pattern) instead of .NET-wrapping, where that's simpler — evaluate against #368's supervisor (which already speaks the worker protocol; the .NET layer buys leases/queues/telemetry, so likely keep it, but decide explicitly).
  • W6 — curation/takes: multi-take variants + library ratings/CV gates surfaced to staff.

Interim note: Eric's full standalone experience (batch UI :5000, Blender addon, all methods) still runs from the spikersoft-artpipe checkout today — nothing was deleted; the platform simply hasn't exposed it all yet.

Joey (2026-07-05): Eric's original was more capable than what the platform exposes — multiple model-generation methods, texturing with map production, multiple rigging methods via user-defined or predefined workflows, each operation an eventhandler (.py where best). Correct. **The engine parity is nearly complete; the platform exposure was deliberately narrowed for the student MVP.** This ticket restores the breadth deliberately. ## Honest parity matrix (engine vs platform-exposed today) | Capability | ProArt engine (in `spikersoft-artpipe`, works via its own server/batch UI today) | Platform exposes | |---|---|---| | Modeling | **7 methods** (TRELLIS, Hunyuan3D-2, SF3D, TripoSR, TripoSG, InstantMesh, Shap-E), per-submission choice + failover | 1 fixed per deployment (StageSettings) | | Concept | 4 backends + failover candidates | 1 (sdxl_lightning) | | Texturing/maps | Hunyuan3DPaint **full PBR** (baseColor/metallic/roughness), SD-Turbo-tex, TRELLIS retexture; normal/ORM outputs | prompt-only PNG (sd_turbo_tex); export accepts normal/orm params **nothing produces them** | | Rigging | **method selection + fallback chain** UniRig→RigNet→Mesh2Rig→smart_rig (`rig_model` config = the "user-defined workflow" seam) | smart_rig only | | Animation | MDM/HY-Motion text-to-motion, OpenPose/MediaPipe mocap, retargeting | procedural presets (licensing-gated; deliberate) | | Workflows | **presets** (saved pipeline configs), **projects** w/ engine targets, multi-take, continuous mode | fixed Character/Prop plans | | Curation | library, ratings, CV gates, takes | enrichment metrics stored, no curation UI | | Operation shape | per-model venv **.py workers**, NDJSON protocol | same protocol preserved; #368 = one resident worker per model — Eric's shape, formalized | ## Phases - **W1 — per-stage method choice**: submission carries optional per-stage model within profile `allowed_models` (backend plumbing largely exists via StageSettings/model queues); composer "advanced" panel (staff first, students later behind a flag). Server-side failover chain per stage (ProArt's concept-failover pattern generalized). - **W2 — workflow presets**: named pipeline configs (stage→method map + params + prompt prefix/suffix) as platform docs (Mongo), predefined by staff, selectable at submit — the 1:1 of ProArt presets; project grouping later. - **W3 — full PBR/texture map sets**: Hunyuan3DPaint on CUDA deployments (+ #358 Metal port for Mac) producing baseColor/metallic/roughness (+normal/ORM where supported); export stage already accepts the params; artifact kinds + viewer channels (art_pipe's own viewer has PBR channel switching to mirror). - **W4 — rigging method chain**: expose `rig_model` (auto/unirig/rignet/mesh2rig/smart_rig) through StageSettings params; deploy UniRig/RigNet residents on CUDA (#368 R4 pattern); record which rigger succeeded (P4 ticket already asked for this). - **W5 — Python-native eventhandlers where best**: per Eric's design, allow a resident model service to BE a .py process directly (the original `Trellis3D.Python` eventhandler pattern) instead of .NET-wrapping, where that's simpler — evaluate against #368's supervisor (which already speaks the worker protocol; the .NET layer buys leases/queues/telemetry, so likely keep it, but decide explicitly). - **W6 — curation/takes**: multi-take variants + library ratings/CV gates surfaced to staff. Interim note: Eric's full standalone experience (batch UI :5000, Blender addon, all methods) still runs from the `spikersoft-artpipe` checkout today — nothing was deleted; the platform simply hasn't exposed it all yet.
spikerj added the enhancement label 2026-07-05 03:17:05 +00:00
Author
Owner

W6 (staff curation) backend slice merged in spikersoft-backend PR #88. Additive-only (embedded CurationRatings on ArtAsset mirroring QuarantinedArtifacts, #369 legacy-safe): staff quality ratings (POST gallery/{id}/rating, 1–5 + note, upsert-by-staff) and a curation query (GET gallery/curation — paged Approved assets with derived metrics/artifact sizes/chosen methods + ratings, sortable/filterable). 36 new tests, 437 ArtStudio suite green. Found no pre-existing student engagement to duplicate; enrichment metrics aren't stored as queryable fields (only a metadata-json GridFS blob) so the query surfaces queryable proxies — deep blob-parse deferred.

#378 status — where the phases stand:

  • W1 (per-stage method choice) merged · W1.5 (failover chains) merged · W2 (workflow presets) merged · W6 curation backend merged
  • W3 (full PBR map sets) / W4 (rig-method chain) — need the CUDA host + models (SERVER-gated)
  • W5 (.py-native eventhandler decision) — design/eval, pending
  • W6 remaining — multi-take variants (worker/pipeline change) + the Angular staff curation UI (endpoints ready; needs an angular PR — which I hand to you to merge given the branch-protection asymmetry)

Ticket stays open for W3/W4/W5 + the W6 remainders.

**W6 (staff curation) backend slice merged** in spikersoft-backend PR #88. Additive-only (embedded `CurationRatings` on ArtAsset mirroring `QuarantinedArtifacts`, #369 legacy-safe): staff quality ratings (`POST gallery/{id}/rating`, 1–5 + note, upsert-by-staff) and a curation query (`GET gallery/curation` — paged Approved assets with derived metrics/artifact sizes/chosen methods + ratings, sortable/filterable). 36 new tests, 437 ArtStudio suite green. Found no pre-existing student engagement to duplicate; enrichment metrics aren't stored as queryable fields (only a metadata-json GridFS blob) so the query surfaces queryable proxies — deep blob-parse deferred. **#378 status — where the phases stand:** - **W1** (per-stage method choice) ✅ merged · **W1.5** (failover chains) ✅ merged · **W2** (workflow presets) ✅ merged · **W6** curation ✅ backend merged - **W3** (full PBR map sets) / **W4** (rig-method chain) — need the CUDA host + models (SERVER-gated) - **W5** (.py-native eventhandler decision) — design/eval, pending - **W6 remaining** — multi-take variants (worker/pipeline change) + the Angular staff curation UI (endpoints ready; needs an angular PR — which I hand to you to merge given the branch-protection asymmetry) Ticket stays open for W3/W4/W5 + the W6 remainders.
Author
Owner

W4 (rig-method chain) domain slice merged in spikersoft-backend PR #97. Restores rig-method breadth + records which method actually produced each result — additive, builds on W1/W1.5.

  • Rig-method allow-list: rigging now exposes auto/unirig/rignet/mesh2rig alongside smart_rig (kept default → no-selection byte-identical); extras carry the new rig_model=<key> param seam (correction: rig_model didn't previously exist — the op is Blender/rig_mesh). Reaches the worker via the unchanged W1 StageMethods path; unknown rig key fails fast at startup.
  • Winning-method provenance (ArtAsset.StageMethodProvenance, #369-guarded like CurationRatings): records the actual method per stage, capturing the failover winner (the orchestrator rewrites the selection to the succeeding alternative). Optional method_used worker result field is additive/back-compat — worker NDJSON contract unchanged. Exposed on asset reads + staff curation projection (chosen-vs-actual).
  • 22 new tests, 621 ArtStudio+ArtPipe green. Rigging failover left OFF by default (flip ArtStudio:StageFailover:Stages:rigging when residents land).

#378 status: W1 · W1.5 · W2 · W4 · W6 (curation) backend. Remaining: W3 (full PBR/texture map sets — domain slice self-mergeable, actual maps CUDA-gated) · W5 (.py-native eventhandler decision) · CUDA execution for W3/W4 (UniRig/RigNet/Hunyuan3DPaint residents on SERVER, #368 R4) · Angular composer/curation UIs. Ticket stays open.

**W4 (rig-method chain) domain slice merged** in spikersoft-backend PR #97. Restores rig-method breadth + records which method actually produced each result — additive, builds on W1/W1.5. - **Rig-method allow-list**: rigging now exposes `auto`/`unirig`/`rignet`/`mesh2rig` alongside `smart_rig` (kept default → no-selection byte-identical); extras carry the new `rig_model=<key>` param seam (correction: `rig_model` didn't previously exist — the op is `Blender/rig_mesh`). Reaches the worker via the unchanged W1 `StageMethods` path; unknown rig key fails fast at startup. - **Winning-method provenance** (`ArtAsset.StageMethodProvenance`, #369-guarded like `CurationRatings`): records the actual method per stage, **capturing the failover winner** (the orchestrator rewrites the selection to the succeeding alternative). Optional `method_used` worker result field is additive/back-compat — worker NDJSON contract unchanged. Exposed on asset reads + staff curation projection (chosen-vs-actual). - 22 new tests, 621 ArtStudio+ArtPipe green. Rigging failover left OFF by default (flip `ArtStudio:StageFailover:Stages:rigging` when residents land). **#378 status:** W1 ✅ · W1.5 ✅ · W2 ✅ · **W4** ✅ · W6 (curation) ✅ backend. **Remaining:** W3 (full PBR/texture map sets — domain slice self-mergeable, actual maps CUDA-gated) · W5 (.py-native eventhandler decision) · CUDA execution for W3/W4 (UniRig/RigNet/Hunyuan3DPaint residents on SERVER, #368 R4) · Angular composer/curation UIs. Ticket stays open.
Author
Owner

W3 (full PBR map sets) domain slice merged in spikersoft-backend PR #98. The platform can now carry/store/expose/export full PBR map sets — additive, byte-identical for today's single-texture assets.

  • PBR taxonomy (PbrTextureMaps.cs): base-color/normal/orm/metallic/roughness/ao/emissive kinds → glTF-2.0 viewer channels + export params + include-flags + filename tokens. Additive to the existing texture-set fallback.
  • Storage: loose artifact refs, NO new field — a map set = texturing artifacts with a PBR kind (rejected the map-set subdoc as duplicating Artifacts; #369-trivial since zero new fields; legacy → empty set).
  • Export wiring: classifies each texture, sets *_path (as before) + turns on include_normal/orm/emissive — byte-identical when absent (unclassifiable single texture → no flags, proven).
  • Exposure: games manifest Textures ({kind, channel, url} — viewer channel-switcher contract) + curation projection (PbrMapCount/PbrChannels); safety-gate thresholds extended to new kinds. 40 new tests, 661 ArtStudio+ArtPipe green.
  • Contained caveat: art_pipe's exact export flag names are derived (not verified vs the artpipe checkout, which isn't in this repo). Inert today — flags only appear once CUDA map-generation lands; one-place fix if they differ.

#378 status: W1 · W1.5 · W2 · W3 · W4 · W6 curation backend. The ProArt parity DOMAIN layer is now substantially restored. Remaining: W5 (.py-native eventhandler decision) · CUDA/Metal model execution for W3/W4 (Hunyuan3DPaint full-PBR, UniRig/RigNet residents — SERVER #368 / Mac #358) · Angular composer/viewer/curation UIs · W6 multi-take. Ticket stays open.

**W3 (full PBR map sets) domain slice merged** in spikersoft-backend PR #98. The platform can now carry/store/expose/export full PBR map sets — additive, byte-identical for today's single-texture assets. - **PBR taxonomy** (`PbrTextureMaps.cs`): base-color/normal/orm/metallic/roughness/ao/emissive kinds → glTF-2.0 viewer channels + export params + include-flags + filename tokens. Additive to the existing `texture-set` fallback. - **Storage: loose artifact refs, NO new field** — a map set = texturing artifacts with a PBR kind (rejected the map-set subdoc as duplicating `Artifacts`; #369-trivial since zero new fields; legacy → empty set). - **Export wiring**: classifies each texture, sets `*_path` (as before) + turns on `include_normal/orm/emissive` — byte-identical when absent (unclassifiable single texture → no flags, proven). - **Exposure**: games manifest `Textures` (`{kind, channel, url}` — viewer channel-switcher contract) + curation projection (`PbrMapCount`/`PbrChannels`); safety-gate thresholds extended to new kinds. 40 new tests, 661 ArtStudio+ArtPipe green. - **Contained caveat**: art_pipe's exact export flag names are derived (not verified vs the artpipe checkout, which isn't in this repo). Inert today — flags only appear once CUDA map-generation lands; one-place fix if they differ. **#378 status:** W1 ✅ · W1.5 ✅ · W2 ✅ · **W3** ✅ · **W4** ✅ · W6 curation ✅ backend. **The ProArt parity DOMAIN layer is now substantially restored.** Remaining: W5 (.py-native eventhandler decision) · CUDA/Metal model execution for W3/W4 (Hunyuan3DPaint full-PBR, UniRig/RigNet residents — SERVER #368 / Mac #358) · Angular composer/viewer/curation UIs · W6 multi-take. Ticket stays open.
Author
Owner

W5 — Python-native eventhandlers: recommendation (for your sign-off)

Researched the .py-native-vs-.NET-wrapper question against the actual code. Recommendation: confirm your hypothesis — keep the .NET wrapper (status quo, formalized by #368). Do not build a pure-.py resident path. Evidence, specific to this stack:

The .py/.NET boundary is already drawn exactly at the model. The .py worker is pure inference over the stdin/stdout NDJSON protocol (ready/started/progress/result/error/pong); it knows nothing about RabbitMQ, GPU leases, GridFS, Jaeger, the safety gate, or SignalR. The .NET host (ArtPipeProcessor) is the entire distributed-systems plane around it. Eric's design intent is already fully expressed — byte-preserved worker protocol, per-model venv isolation, one always-loaded model per service (#368, Mac-validated). "Python-native where best" is satisfied today: the .py is the eventhandler body; .NET is only the transport shell.

What the .NET layer buys is all load-bearing here, and Option B (pure .py) doesn't remove a layer — it re-implements 6+ wire-compatible protocol clients in a second language: GPU leases (SharedGpuLeaseService RPC — there's a GpuLeaseWireCompatTests guard because this wire format is a forever-contract), per-model queues, OTLP+W3C trace-context (house rule: wrong key = silent no-exporter, #307 found 7 invisible services), DLQ/retry, the fail-closed child-safety gate (least-desirable code to fork into a 2nd language), progress events, and 700+ lines of orchestration. No process hop is actually eliminated — the model runs in Python either way; the .NET.py hop is a local pipe with the model already warm (#368: sdxl 4.2s warm).

Criterion if hybrid is ever revisited (door explicit, not just shut): a stage may be .py-native only if it simultaneously needs no GPU lease, produces no servable image (no safety gate), and needs no trace-continuity/DLQ/progress. For ProArt model stages that set is empty. So: ship Option A, don't hold W1–W4 breadth waiting on a .py path.

W5 disposition: decided → keep .NET wrapper; .py-native rejected. No implementation ticket needed (it's the status quo).

BUT the analysis surfaced a real orthogonal bug → filing separately (see next). The safety gate routes SafetyCheck through IArtPipeStageExecutor, which in resident mode is the ResidentArtPipeStageExecutor that throws on any non-resident model — so a resident image-gen deployment would fail-closed-quarantine every output. Must be fixed before the #368 SERVER cutover.

## W5 — Python-native eventhandlers: recommendation (for your sign-off) Researched the `.py`-native-vs-.NET-wrapper question against the actual code. **Recommendation: confirm your hypothesis — keep the .NET wrapper (status quo, formalized by #368). Do not build a pure-`.py` resident path.** Evidence, specific to this stack: **The `.py`/.NET boundary is already drawn exactly at the model.** The `.py` worker is pure inference over the stdin/stdout NDJSON protocol (`ready`/`started`/`progress`/`result`/`error`/`pong`); it knows nothing about RabbitMQ, GPU leases, GridFS, Jaeger, the safety gate, or SignalR. The .NET host (`ArtPipeProcessor`) is the entire distributed-systems plane around it. **Eric's design intent is already fully expressed** — byte-preserved worker protocol, per-model venv isolation, one always-loaded model per service (#368, Mac-validated). "Python-native where best" is satisfied today: the `.py` *is* the eventhandler body; .NET is only the transport shell. **What the .NET layer buys is all load-bearing here**, and Option B (pure `.py`) doesn't remove a layer — it re-implements 6+ wire-compatible protocol clients in a second language: GPU leases (`SharedGpuLeaseService` RPC — there's a `GpuLeaseWireCompatTests` guard *because* this wire format is a forever-contract), per-model queues, OTLP+W3C trace-context (house rule: wrong key = silent no-exporter, #307 found 7 invisible services), DLQ/retry, the **fail-closed child-safety gate** (least-desirable code to fork into a 2nd language), progress events, and 700+ lines of orchestration. No process hop is actually eliminated — the model runs in Python either way; the `.NET`↔`.py` hop is a local pipe with the model already warm (#368: sdxl 4.2s warm). **Criterion if hybrid is ever revisited** (door explicit, not just shut): a stage may be `.py`-native only if it simultaneously needs no GPU lease, produces no servable image (no safety gate), and needs no trace-continuity/DLQ/progress. For ProArt model stages that set is **empty**. So: ship Option A, don't hold W1–W4 breadth waiting on a `.py` path. **W5 disposition:** decided → keep .NET wrapper; `.py`-native rejected. No implementation ticket needed (it's the status quo). **BUT the analysis surfaced a real orthogonal bug** → filing separately (see next). The safety gate routes `SafetyCheck` through `IArtPipeStageExecutor`, which in **resident mode** is the `ResidentArtPipeStageExecutor` that throws on any non-resident model — so a resident image-gen deployment would fail-closed-quarantine every output. Must be fixed before the #368 SERVER cutover.
Sign in to join this conversation.