Art Studio: record what actually produced each stage run (#842) #843

New Issue

2026-07-25T20:21:54Z

spikerj commented

2026-07-25 20:21:54 +00:00

Backend capture half. Records the worker's optional generation_params block onto ArtAssetStageRun — model + revision, seed + where it came from, steps, guidance, scheduler, size, reference images, library versions.

Recorded on the RUN, not the asset: ArtAsset.StageMethodProvenance is upsert-by-stage and so destroys the previous entry on restart, which would make reproducing an earlier attempt impossible. Stage runs are append-only by their documented invariant and get a fresh row per failover hop. ArtAssetArtifactRef gains a StageRunId pointer (not a copy — one texturing run emits several refs and Artifacts is a value-converted JSON string whose comparer walks the whole list on every save).

Gotcha worth keeping: this is the first SINGULAR complex property in the art-studio model. Unlike the sibling embedded arrays, the EF-Mongo shaper binds its members directly and throws Document element is missing for required non-nullable property for any absent one — so EVERY scalar must be nullable, including Schema and CapturedAt, and Dictionary members too (EF maps those as properties, not navigations). A guard test pins it.

Backend capture half. Records the worker's optional `generation_params` block onto `ArtAssetStageRun` — model + revision, seed + where it came from, steps, guidance, scheduler, size, reference images, library versions. Recorded on the RUN, not the asset: `ArtAsset.StageMethodProvenance` is upsert-by-stage and so destroys the previous entry on restart, which would make reproducing an earlier attempt impossible. Stage runs are append-only by their documented invariant and get a fresh row per failover hop. `ArtAssetArtifactRef` gains a `StageRunId` pointer (not a copy — one texturing run emits several refs and `Artifacts` is a value-converted JSON string whose comparer walks the whole list on every save). Gotcha worth keeping: this is the first SINGULAR complex property in the art-studio model. Unlike the sibling embedded arrays, the EF-Mongo shaper binds its members directly and throws `Document element is missing for required non-nullable property` for any absent one — so EVERY scalar must be nullable, including Schema and CapturedAt, and Dictionary members too (EF maps those as properties, not navigations). A guard test pins it.

spikerj commented

2026-07-29 15:37:18 +00:00

Resolved in spikersoft-backend (91bd5a9c). Verified against origin/master:

ArtAssetGenerationParams.cs:47, attached to the run at ArtAssetStageRun.cs:119-120.
The #369/#606 schema-evolution rule is honoured throughout: every scalar is nullable including Schema (:55) and CapturedAt (:222); both dictionaries nullable with normalizing accessors (:189, :202); nested adapter/reference scalars nullable too. No required field can 500 an old document.
ArtAssetArtifactRef.StageRunId at ArtAsset.cs:469-471, populated at ArtPipeStageOrchestrator.cs:516.
Recorded on the run append-only (ArtPipeStageOrchestrator.cs:611-616), deliberately kept separate from the upsert-by-stage provenance write at :589-598 — so a retry adds a row rather than overwriting history.
Mapper ArtAssetGenerationParamsMapper.cs:22, total-by-contract, with unknown scalars flattened to Extra and the full block retained in RawJson.

Cross-repo contract verified, which is the part most likely to rot: the mapper's KnownKeys (:25-35) match spikersoft-artpipe/src/artpipe/generation_params.py:150-221 key-for-key, and artpipe pins its side with tests/test_generation_params_contract.py.

One weak spot worth knowing about rather than blocking on: the guard test at ArtAssetLegacyDocumentTests.cs:492-512 returns early and asserts nothing if the entity type isn't found (:497-502), so it can silently no-op. The behavioural pin for Schema/CapturedAt at :430-453 covers the actual gotcha regardless.

Closing.

Resolved in spikersoft-backend (`91bd5a9c`). Verified against `origin/master`: - `ArtAssetGenerationParams.cs:47`, attached to the run at `ArtAssetStageRun.cs:119-120`. - **The #369/#606 schema-evolution rule is honoured throughout**: every scalar is nullable including `Schema` (`:55`) and `CapturedAt` (`:222`); both dictionaries nullable with normalizing accessors (`:189`, `:202`); nested adapter/reference scalars nullable too. No required field can 500 an old document. - `ArtAssetArtifactRef.StageRunId` at `ArtAsset.cs:469-471`, populated at `ArtPipeStageOrchestrator.cs:516`. - Recorded on the run append-only (`ArtPipeStageOrchestrator.cs:611-616`), deliberately kept separate from the upsert-by-stage provenance write at `:589-598` — so a retry adds a row rather than overwriting history. - Mapper `ArtAssetGenerationParamsMapper.cs:22`, total-by-contract, with unknown scalars flattened to `Extra` and the full block retained in `RawJson`. **Cross-repo contract verified**, which is the part most likely to rot: the mapper's `KnownKeys` (`:25-35`) match `spikersoft-artpipe/src/artpipe/generation_params.py:150-221` key-for-key, and artpipe pins its side with `tests/test_generation_params_contract.py`. One weak spot worth knowing about rather than blocking on: the guard test at `ArtAssetLegacyDocumentTests.cs:492-512` returns early and asserts nothing if the entity type isn't found (`:497-502`), so it can silently no-op. The behavioural pin for `Schema`/`CapturedAt` at `:430-453` covers the actual gotcha regardless. Closing.

spikerj closed this issue

2026-07-29 15:37:22 +00:00

spikerj referenced this issue

2026-07-29 15:38:58 +00:00

EPIC: Art Studio — record and replay generation parameters (+ text+image concept) #842

spikerj referenced this issue

2026-07-29 16:55:44 +00:00

[Bug][Prod][Calendar] GET events threw for every pre-#781 event — required non-nullable AffectsAvailability missing from old documents (backfilled) #786

Sign in to join this conversation.