[Epic] ProArt 3D Art Studio — student asset pipeline (concept → model → texture → rig → animate → game-ready GLB) #346

New Issue

2026-07-04T05:09:57Z

spikerj commented

2026-07-04 05:09:57 +00:00

Goal

Turn Eric Spiker's donated ProArt (art_pipe) routed-art pipeline into a student-facing 3D Art Studio on learn.spikersoft.com: a student types a prompt or uploads a drawing and walks their asset through the same stages a real game studio uses — concept art → 3D model → PBR textures → rig/bones → animation → game-ready GLB — then uses that asset in our game playgrounds. This ties together the platform's three pillars: learning, coding, and video games, and gives students a genuine 3D-art-pipeline mental model.

Current state (verified in code)

A single-stage vertical already exists end-to-end — this epic generalizes it, it does not start from zero: SpikerSoft.Api/Domain/ThreeDModel/ThreeDModelController.cs → SpikerSoft.Business/Domain/ThreeDModel/{Commands,Queries} → SpikerSoft.Data/Mongos/ThreeDModel.cs (already has Status/CurrentStage/ProgressPercentage) → Python GPU worker SpikerSoft.EventHandlers.Trellis3D.Python → Angular libraries/features/trellis-3d-generator (three.js GLB viewer, SignalR Receive3DModelProgress).
ProArt decomposes cleanly. Stages are declared data, not code: art_pipe/src/artpipe/config.py::ASSET_TYPE_STAGES (concept / modeling / texturing / rigging / animation / export / enrichment). Every model backend runs as a subprocess worker (src/artpipe/worker.py: stdin JSON in → stdout NDJSON started|progress|result|error events) in its own per-model venv with a manifest (models/*/artpipe.json); results follow src/artpipe/result_contract.py. Model coverage: TRELLIS / Hunyuan3D-2 / SF3D / TripoSR (modeling), SDXL-Lightning / FLUX.schnell (concept), Hunyuan3DPaint / SD-Turbo (texturing), UniRig → RigNet → Mesh2Rig fallback chain (rigging, 56-bone canonical humanoid), MDM / HY-Motion (animation), headless Blender (export/enrichment).
What we replace: art_pipe's Flask + SQLite batch orchestrator (src/artpipe/batch/) — its job is done by our CQRS + RabbitMQ + Mongo orchestration. What we keep: the per-stage worker protocol, model manifests/venvs, and Blender headless tooling.
Platform primitives are ready: IRemoteCommand + RemoteDispatchBehavior (MediatR → RabbitMQ job dispatch), SpikerSoft.EventHandlers.GpuCoordinator VRAM lease broker (24 GB budget, single GPU host SERVER), GridFS blob services (IGridFsImageService pattern), SignalR hubs with Redis backplane, the lesson-video community-content moderation pipeline as template (#336–#339), infra GPU stack template (spikersoft-embeddings/docker-stack-gpu.yml), and the add-event-handler-worker scaffolding skill.

Constraints (drive the phase order)

One 24 GB CUDA card. TRELLIS and HY-Motion each want ~24 GB → stage jobs serialize behind GpuCoordinator VRAM leases; queue position must be visible to students, and model choice per stage is a cost decision (spike below).
art_pipe is CUDA-only today (its own TRELLIS backend is stubbed "until CUDA exts build"); Apple-Silicon dev path exists only via the separate trellis-mac port (spike below).
Students are minors. Text-to-image stages (concept, texture prompts) need prompt filtering + NSFW output checking, and anything shared beyond the owning student goes through staff moderation — safety is a phase, not a footnote.

Architecture (decomposition approach)

Command side: SubmitArtAssetCommand / RunStageCommand as IRemoteCommands → RabbitMQ (art.asset.lifecycle exchange, per-stage routing keys) → GPU worker leases VRAM → drives the art_pipe backend via its worker protocol → artifacts (GLB, textures, thumbnails, previews) into GridFS → publishes art.asset.stage.completed → next stage or SignalR notify. State: ArtAsset + per-stage workflow docs in Mongo (mirroring BlogMediaWorkflowState), read side via MediatR queries. Frontend: new libraries/features/art-studio lib copying the trellis-3d-generator structure.

Phases

Phase	Scope
P1	Backend foundation — `ArtAsset` domain + stage workflow docs, CQRS slice, submit/list/get/cancel/restart-from-stage endpoints, GridFS artifact storage
P2	ProArt GPU worker — RabbitMQ stage consumer wrapping art_pipe's worker protocol, GpuCoordinator leases, infra GPU stack; modeling stage first (parity with today's TRELLIS flow)
P3	Concept + texturing stages — SDXL concept image (with safety checker), PBR texturing, stage chaining
P4	Rigging + animation stages — bone fallback chain, canonical humanoid schema, animation presets + MP4/GIF previews
P5	Frontend Art Studio — pipeline wizard UI, three.js viewer (model/skeleton/animation), SignalR progress + queue position, my-assets library, downloads
P6	Safety, quotas & fairness — prompt moderation, NSFW output gating, per-student GPU-minute quotas, queue fairness
P7	Class gallery + staff moderation + use-your-asset-in-games handoff, curriculum tie-in
Spike	Model × VRAM fit matrix on the 24 GB host (which art_pipe backends we actually enable per stage; cold-start vs keep-warm)
Spike	Apple Silicon dev path — can `trellis-mac` (MPS port) back the modeling stage for local dev?

Credit: ProArt is Eric Spiker's open-source routed art project, donated as the foundation of this feature.

Sub-issues are linked in a comment below as they are filed.

## Goal Turn Eric Spiker's donated **ProArt** (`art_pipe`) routed-art pipeline into a student-facing **3D Art Studio** on learn.spikersoft.com: a student types a prompt or uploads a drawing and walks their asset through the same stages a real game studio uses — **concept art → 3D model → PBR textures → rig/bones → animation → game-ready GLB** — then uses that asset in our game playgrounds. This ties together the platform's three pillars: learning, coding, and video games, and gives students a genuine 3D-art-pipeline mental model. ## Current state (verified in code) - **A single-stage vertical already exists end-to-end** — this epic generalizes it, it does not start from zero: `SpikerSoft.Api/Domain/ThreeDModel/ThreeDModelController.cs` → `SpikerSoft.Business/Domain/ThreeDModel/{Commands,Queries}` → `SpikerSoft.Data/Mongos/ThreeDModel.cs` (already has `Status`/`CurrentStage`/`ProgressPercentage`) → Python GPU worker `SpikerSoft.EventHandlers.Trellis3D.Python` → Angular `libraries/features/trellis-3d-generator` (three.js GLB viewer, SignalR `Receive3DModelProgress`). - **ProArt decomposes cleanly.** Stages are declared data, not code: `art_pipe/src/artpipe/config.py::ASSET_TYPE_STAGES` (concept / modeling / texturing / rigging / animation / export / enrichment). Every model backend runs as a subprocess worker (`src/artpipe/worker.py`: stdin JSON in → stdout NDJSON `started|progress|result|error` events) in its **own per-model venv** with a manifest (`models/*/artpipe.json`); results follow `src/artpipe/result_contract.py`. Model coverage: TRELLIS / Hunyuan3D-2 / SF3D / TripoSR (modeling), SDXL-Lightning / FLUX.schnell (concept), Hunyuan3DPaint / SD-Turbo (texturing), UniRig → RigNet → Mesh2Rig fallback chain (rigging, 56-bone canonical humanoid), MDM / HY-Motion (animation), headless Blender (export/enrichment). - **What we replace:** art_pipe's Flask + SQLite batch orchestrator (`src/artpipe/batch/`) — its job is done by our CQRS + RabbitMQ + Mongo orchestration. What we keep: the per-stage worker protocol, model manifests/venvs, and Blender headless tooling. - **Platform primitives are ready:** `IRemoteCommand` + `RemoteDispatchBehavior` (MediatR → RabbitMQ job dispatch), `SpikerSoft.EventHandlers.GpuCoordinator` VRAM lease broker (24 GB budget, single GPU host `SERVER`), GridFS blob services (`IGridFsImageService` pattern), SignalR hubs with Redis backplane, the lesson-video community-content moderation pipeline as template (#336–#339), infra GPU stack template (`spikersoft-embeddings/docker-stack-gpu.yml`), and the `add-event-handler-worker` scaffolding skill. ## Constraints (drive the phase order) - **One 24 GB CUDA card.** TRELLIS and HY-Motion each want ~24 GB → stage jobs serialize behind GpuCoordinator VRAM leases; queue position must be visible to students, and model choice per stage is a cost decision (spike below). - **art_pipe is CUDA-only today** (its own TRELLIS backend is stubbed "until CUDA exts build"); Apple-Silicon dev path exists only via the separate `trellis-mac` port (spike below). - **Students are minors.** Text-to-image stages (concept, texture prompts) need prompt filtering + NSFW output checking, and anything shared beyond the owning student goes through staff moderation — safety is a phase, not a footnote. ## Architecture (decomposition approach) Command side: `SubmitArtAssetCommand` / `RunStageCommand` as `IRemoteCommand`s → RabbitMQ (`art.asset.lifecycle` exchange, per-stage routing keys) → GPU worker leases VRAM → drives the art_pipe backend via its worker protocol → artifacts (GLB, textures, thumbnails, previews) into GridFS → publishes `art.asset.stage.completed` → next stage or SignalR notify. State: `ArtAsset` + per-stage workflow docs in Mongo (mirroring `BlogMediaWorkflowState`), read side via MediatR queries. Frontend: new `libraries/features/art-studio` lib copying the `trellis-3d-generator` structure. ## Phases | Phase | Scope | |---|---| | P1 | Backend foundation — `ArtAsset` domain + stage workflow docs, CQRS slice, submit/list/get/cancel/restart-from-stage endpoints, GridFS artifact storage | | P2 | ProArt GPU worker — RabbitMQ stage consumer wrapping art_pipe's worker protocol, GpuCoordinator leases, infra GPU stack; modeling stage first (parity with today's TRELLIS flow) | | P3 | Concept + texturing stages — SDXL concept image (with safety checker), PBR texturing, stage chaining | | P4 | Rigging + animation stages — bone fallback chain, canonical humanoid schema, animation presets + MP4/GIF previews | | P5 | Frontend Art Studio — pipeline wizard UI, three.js viewer (model/skeleton/animation), SignalR progress + queue position, my-assets library, downloads | | P6 | Safety, quotas & fairness — prompt moderation, NSFW output gating, per-student GPU-minute quotas, queue fairness | | P7 | Class gallery + staff moderation + use-your-asset-in-games handoff, curriculum tie-in | | Spike | Model × VRAM fit matrix on the 24 GB host (which art_pipe backends we actually enable per stage; cold-start vs keep-warm) | | Spike | Apple Silicon dev path — can `trellis-mac` (MPS port) back the modeling stage for local dev? | Credit: ProArt is Eric Spiker's open-source routed art project, donated as the foundation of this feature. Sub-issues are linked in a comment below as they are filed.

spikerj added the enhancement label 2026-07-04 05:09:57 +00:00

Phase	Scope	Issue
M1	Device abstraction (cuda/mps/cpu) in art_pipe core — upstream PR to ProArt	#356
M2	Mac-native pipeline profile — full Tier-A chain on MPS, benchmarked on the M5	#357
M3	Metal extension reuse — TRELLIS via trellis-mac, then InstantMesh / Hunyuan paint / UniRig via mtldiffrast/mtlgemm	#358
M4	Entire stack on the MacBook — native ArtPipeProcessor (no GPU in Docker on macOS), one-command bring-up, dev-mode GpuCoordinator	#359