spikerj/spikersoft-issues
1
0
Fork 0
Code Issues 196 Pull Requests Actions Packages Projects Releases Wiki Activity

[ArtStudio Spike] Apple Silicon dev path — can trellis-mac (MPS port) back the modeling stage for local development? #355

New Issue
Closed
opened 2026-07-04 05:12:32 +00:00 by spikerj · 2 comments
spikerj commented 2026-07-04 05:12:32 +00:00
Owner
Copy Link
Copy Source

Spike for epic #346. Independent; improves developer loop for #348+.

Question: art_pipe is CUDA-only (main.py::_check_gpu_compat requires nvidia-smi; its TRELLIS backend is stubbed pending CUDA extension builds), which means every pipeline change needs the shared GPU host. trellis-mac (our fork of the PyTorch-MPS TRELLIS.2 port — generate.py, Metal backends in backends/, patches/mps_compat.py; ~5 min per GLB on an M4 Pro) already does image→3D locally. Can it serve as a local-dev modeling backend so the full submit→worker→GridFS→SignalR loop runs on a Mac?

Investigate (time-boxed):

  • Wrap trellis-mac behind art_pipe's worker protocol (stdin JSON → stdout NDJSON per src/artpipe/worker.py) as an alternate artpipe.json manifest — how much glue is it? (They differ: Python 3.10/cu128 vs 3.11/MPS — separate venv is fine, that's art_pipe's model anyway)
  • Or simpler: a dev-mode flag in the #348 worker that shells out to trellis-generate directly for the modeling stage and stubs the other stages
  • What can't run on Mac (rigging chain? Blender headless actually works on macOS — check art_pipe's Blender 5.1 assumptions) — define the honest local-dev boundary
  • Whether GpuCoordinator interaction should be bypassed or mocked in dev mode

Deliverable: a comment with the recommended local-dev story for pipeline work (wrap vs dev-flag vs "don't bother, use the GPU host"), and if viable, a follow-up ticket to implement it. Also note anything worth upstreaming to Eric's ProArt repo (e.g. an MPS backend seam).

Spike for epic #346. Independent; improves developer loop for #348+. **Question:** art_pipe is CUDA-only (`main.py::_check_gpu_compat` requires `nvidia-smi`; its TRELLIS backend is stubbed pending CUDA extension builds), which means every pipeline change needs the shared GPU host. `trellis-mac` (our fork of the PyTorch-MPS TRELLIS.2 port — `generate.py`, Metal backends in `backends/`, `patches/mps_compat.py`; ~5 min per GLB on an M4 Pro) already does image→3D locally. Can it serve as a local-dev modeling backend so the full submit→worker→GridFS→SignalR loop runs on a Mac? **Investigate (time-boxed):** - [ ] Wrap `trellis-mac` behind art_pipe's worker protocol (stdin JSON → stdout NDJSON per `src/artpipe/worker.py`) as an alternate `artpipe.json` manifest — how much glue is it? (They differ: Python 3.10/cu128 vs 3.11/MPS — separate venv is fine, that's art_pipe's model anyway) - [ ] Or simpler: a dev-mode flag in the #348 worker that shells out to `trellis-generate` directly for the modeling stage and stubs the other stages - [ ] What can't run on Mac (rigging chain? Blender headless actually works on macOS — check art_pipe's Blender 5.1 assumptions) — define the honest local-dev boundary - [ ] Whether GpuCoordinator interaction should be bypassed or mocked in dev mode **Deliverable:** a comment with the recommended local-dev story for pipeline work (wrap vs dev-flag vs "don't bother, use the GPU host"), and if viable, a follow-up ticket to implement it. Also note anything worth upstreaming to Eric's ProArt repo (e.g. an MPS backend seam).
spikerj added the questionenhancement labels 2026-07-04 05:12:32 +00:00
spikerj commented 2026-07-04 05:28:45 +00:00
Author
Owner
Copy Link
Copy Source

Static audit done (2026-07-04) — answers the feasibility half of this spike; scope has grown from "local dev path" to a first-class Mac workstream (M1–M4: #356–#359) targeting the M5 Pro Max / 128 GB machine.

Per-backend MPS verdicts:

Tier Backends Path
A — portable with device patch SDXL-Lightning, SDXL-Turbo, SD-Turbo-Tex, Shap-E, TripoSR, MDM, FLUX.schnell (fp16, 22 GB — fits unified memory) #356 + #357
B/C — blocked by CUDA exts with Metal drop-ins available TRELLIS (nvdiffrast/spconv/flash-attn), InstantMesh (nvdiffrast), Hunyuan3D paint (custom_rasterizer), UniRig (spconv) — trellis-mac's mtldiffrast/mtlgemm/mtlmesh/mtlbvh are API drop-ins #358
D — no Metal path TripoSG (fvdb) CUDA-only, documented
CPU-fine today OpenPose (cv2.dnn), MediaPipe, RigNet (pyg CPU wheels, slow), smart_rig/Blender —

Core findings: art_pipe has zero device abstraction — 13 backends hardcode "cuda", install always pulls cu128 wheels (torch_compat.py, venv_setup.py, bootstrap.py), main.py::_check_gpu_compat requires nvidia-smi, and blender/headless/_helpers.py never sets compute_device_type="METAL" so Cycles silently renders on CPU on macOS. Exact file:line inventory is in #356.

Recommendation (supersedes the wrap-vs-dev-flag question): do #356 (device seam, upstream to ProArt) → #357 (all-Tier-A profile = full pipeline on Mac without any Metal kernel work) → #358 (TRELLIS via trellis-mac, then the other Tier-B/C backends). Remaining hands-on validation work of this spike is absorbed into #357's per-stage benchmark checklist. GpuCoordinator handling on Mac is decided in #359 (bypass recommended).

Static audit done (2026-07-04) — answers the feasibility half of this spike; scope has grown from "local dev path" to a first-class Mac workstream (M1–M4: #356–#359) targeting the M5 Pro Max / 128 GB machine. **Per-backend MPS verdicts:** | Tier | Backends | Path | |---|---|---| | A — portable with device patch | SDXL-Lightning, SDXL-Turbo, SD-Turbo-Tex, Shap-E, TripoSR, MDM, FLUX.schnell (fp16, 22 GB — fits unified memory) | #356 + #357 | | B/C — blocked by CUDA exts with Metal drop-ins available | TRELLIS (nvdiffrast/spconv/flash-attn), InstantMesh (nvdiffrast), Hunyuan3D paint (custom_rasterizer), UniRig (spconv) — trellis-mac's `mtldiffrast`/`mtlgemm`/`mtlmesh`/`mtlbvh` are API drop-ins | #358 | | D — no Metal path | TripoSG (fvdb) | CUDA-only, documented | | CPU-fine today | OpenPose (cv2.dnn), MediaPipe, RigNet (pyg CPU wheels, slow), smart_rig/Blender | — | **Core findings:** art_pipe has zero device abstraction — 13 backends hardcode `"cuda"`, install always pulls cu128 wheels (`torch_compat.py`, `venv_setup.py`, `bootstrap.py`), `main.py::_check_gpu_compat` requires nvidia-smi, and `blender/headless/_helpers.py` never sets `compute_device_type="METAL"` so Cycles silently renders on CPU on macOS. Exact file:line inventory is in #356. **Recommendation (supersedes the wrap-vs-dev-flag question):** do #356 (device seam, upstream to ProArt) → #357 (all-Tier-A profile = full pipeline on Mac without any Metal kernel work) → #358 (TRELLIS via trellis-mac, then the other Tier-B/C backends). Remaining hands-on validation work of this spike is absorbed into #357's per-stage benchmark checklist. GpuCoordinator handling on Mac is decided in #359 (bypass recommended).
spikerj commented 2026-07-04 20:12:24 +00:00
Author
Owner
Copy Link
Copy Source

Spike answered and superseded: the static audit (comment above) became the M1–M4 workstream (#356–#359), and the implementation route chosen was the device seam + Tier-A profile rather than wrapping trellis-mac (which remains #358's flagship item for TRELLIS-class quality). Local dev on Apple Silicon is proven far beyond this spike's question — the full 7-stage Character plan runs end-to-end on the M5 through the platform stack. Closing; nothing left in this spike that isn't tracked in M1–M4.

Spike answered and superseded: the static audit (comment above) became the M1–M4 workstream (#356–#359), and the implementation route chosen was the device seam + Tier-A profile rather than wrapping trellis-mac (which remains #358's flagship item for TRELLIS-class quality). Local dev on Apple Silicon is proven far beyond this spike's question — the full 7-stage Character plan runs end-to-end on the M5 through the platform stack. Closing; nothing left in this spike that isn't tracked in M1–M4.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agentic
Actionable surfaced by the agentic architecture & security review
 bug
Something is not working
 duplicate
This issue or pull request already exists
 enhancement
New feature
 help wanted
Need some help
 invalid
Something is wrong
 question
More information is needed
 sonarqube
Code quality issue surfaced by SonarQube (bug / code smell)
 wontfix
This won't be fixed
No labels enhancement question
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
spikerj (Joseph Spiker)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: spikerj/spikersoft-issues#355