spikerj/spikersoft-issues
1
0
Fork 0
Code Issues 196 Pull Requests Actions Packages Projects Releases Wiki Activity

[Enhancement][ArtPipe][Angular] AI QR code generation — SDXL qrcode_monster ControlNet per-model lane + authenticated AI button in the QR tool #535

New Issue
Closed
opened 2026-07-13 01:50:09 +00:00 by spikerj · 2 comments
spikerj commented 2026-07-13 01:50:09 +00:00
Owner
Copy Link
Copy Source

Idea

The Angular client already has a client-side QR code generator tool (feature-dev-tools-qr-code, /development/tools/qr-code, ng-qrcode). Add an AI mode: a button that generates artistic, scannable QR codes from the tool's content (text/URL/WiFi payload) using monster-labs/control_v1p_sdxl_qrcode_monster (https://huggingface.co/monster-labs/control_v1p_sdxl_qrcode_monster) — a ControlNet conditioned on a plain QR image, producing stylized codes that still scan.

2026-07-28 rewrite. The original ticket predated much of the infra it proposed building; it chose control_v1p_sd15_qrcode_monster because SD 1.5 + ControlNet fit the SERVER 3070 Ti. That rationale is stale: the 4090 (24 GB) is a permanent lane, and artpipe ships no SD 1.5 base at all — its text-to-image family is SDXL (stabilityai/stable-diffusion-xl-base-1.0 + Lightning LoRA, SDXL-Turbo) and Flux. The SDXL Monster variant rides the already-baked SDXL base as ~2.5 GB of install.extra_weights, and diffusers builds a StableDiffusionXLControlNetPipeline from the loaded pipeline's components (same trick sdxl_lightning.py uses for img2img). The SD15 variant would mean adding and maintaining a whole second base-model family (new model dir, venv, baked image) — not worth it. Known tradeoff: the SDXL port typically needs controlnet_conditioning_scale tuning (~1.2–1.5) and more attempts per keeper than the battle-tested SD15 v2; that's one-time tuning, and an SD 1.5 lane could still be added later if scannability tuning truly disappoints. Also superseded below: GridFS → MinIO (#783 end state), and this is explicitly NOT an Art Studio / ArtAsset flow — it's a dedicated tools lane.

Architecture (dedicated tools lane — not ArtAsset)

Same delegated shape as the art lifecycle (CQRS command over RabbitMQ → per-model container → SignalR back), but its own command contract, queue, and storage prefix. No ArtAsset/ArtAssetStageRun rows, no art-studio UI involvement.

1. artpipe model (spikersoft-artpipe)

  • models/QRMonster/artpipe.json: SDXL base via the same install.weights/weight_patterns shape as SDXLLightning + install.extra_weights monster-labs/control_v1p_sdxl_qrcode_monster; pip adds qrcode[pil] + zxing-cpp; vram_mb ~9500 (SDXL base ~6 GB booking + ~2.5 GB ControlNet + headroom — do NOT reuse the 6000 MB concept booking, the coordinator would grant a concurrent lease that OOMs).
  • Backend src/artpipe/models/qr_monster.py (pattern: sdxl_lightning.py, incl. its _base_ref offline-snapshot quirk), capability qr_stylize:
    • params: content (payload to encode), error_correction_level, prompt (style), negative_prompt, controlnet_conditioning_scale, seed, size, steps, optional reference_image_path + strength (user-supplied style/blend image → img2img init alongside the ControlNet conditioning).
    • flow: render plain QR from content worker-side with the qrcode lib at error-correction level H (that redundancy is what the stylization spends) → ControlNet conditioning image → generate → self-verify scannability by decoding the output (zxing-cpp); retry with escalating conditioning scale up to N attempts before returning scannable: false in the result payload.
    • Start with plain SDXL base at ~25–30 steps for quality; Lightning-LoRA few-step generation is a later optimization if latency hurts (ControlNet + Lightning needs its own conditioning-scale retune).

2. Baked image (existing pipeline — no new pattern)

  • Add qrmonster → MODELS="QRMonster" to .gitea/workflows/model-env-images.yml. Prod model images run HF_HUB_OFFLINE=1 — the ControlNet weights must be baked before any stage mapping flips, or it's a hard runtime failure.

3. Backend lane (spikersoft-backend)

  • Endpoint: POST api/tools/qr-art (+ GET api/tools/qr-art/{jobId} for reconnect/polling fallback). [Authorize] — logged-in users only. This burns GPU minutes, so it is not a free anonymous operation like the client-side tool; add per-user rate limiting (e.g. small rolling quota) from day one. The button is the only gated part — the existing client-side QR tool stays anonymous.
  • New first-class command contract (e.g. RequestQrArtCommand in SpikerSoft.Contracts.Workers): JobId, UserId, Content, ErrorCorrectionLevel, StylePrompt, NegativePrompt?, Seed?, ConditioningScale?, Size?, ReferenceImageKey?, Strength?. All generation inputs are first-class fields on the command — deliberately not GenerationParamOverrides: the overridable allow-list (ArtAssetGenerationParamKeys.Overridable) reserves image_path/conditioning inputs as pipeline-only, and this lane shouldn't inherit art-studio's param plumbing at all. The client's generated QR is passed by value — Content + ErrorCorrectionLevel are exactly what the tool has encoded, and the worker re-renders that QR losslessly at ECC H print quality as the conditioning image (a screen-resolution client PNG at ECC M would waste the stylization headroom). The optional reference image IS a real upload: stored via the normal upload pipeline, referenced by ReferenceImageKey, fed as the img2img init image with Strength controlling how much of it survives.
  • Routing: publish to the per-model queue art.model.qrmonster.task (existing ResidentModelDir/ModelQueueStages convention) with W3C trace context.
  • Container: new swarm stack spikersoft-artpipe-model-qrmonster (copy an existing spikersoft-artpipe-model-* stack), artpipe-gpu placement label. Run in model-queue mode (per-job lease: acquire → load → run → unload; idle container holds 0 VRAM) — right fit for a sporadically-used tool; resident mode is the fallback if load latency proves annoying.
  • Safety gate mandatory, fail-closed: output is an arbitrary SDXL image on a minors platform — route through the existing SafetyCheck gate before the artifact becomes servable, exactly like art-studio stage outputs.
  • Storage: MinIO (S3 path, same as #783's art-asset end state) under a distinct prefix/bucket for tool artifacts; completion event carries the servable URL.
  • Progress/completion: worker NDJSON → qr.art.{started,progress,completed,failed} events → SignalR relay to the client (same delegated pattern as signalr-art-asset-progress-source.ts, own hub group keyed by JobId/UserId).

4. Angular client

  • "AI style" button/tab in the existing QR tool: visible always, enabled only when authenticated (anonymous users get a "log in to generate AI QR codes" affordance). Sends the tool's current payload + ECC level + style prompt + optional reference image upload; progress via SignalR; result rendered from the MinIO-backed URL with download.
  • Client-side decode badge ("scans OK / may not scan") using the worker's scannable flag, with one-click regenerate (new seed).

Sequencing / prerequisites

  • Weights baked into the env image before wiring the queue mapping (offline-mode hard failure otherwise).
  • Licensing: qrcode_monster is OpenRAIL-M — verify terms before shipping (commercial use allowed with use-based restrictions; outputs served on a minors platform → the safety gate is doing real work).
  • Optional later: LLM prompt-enrichment ("cyberpunk neon city" → full SD prompt) on the existing LLM worker infra; v1 ships with the raw prompt.
  • Related: #368 (per-model queues), #349 (safety gate), #783 (MinIO artifacts), #367 (GPU capacity).
## Idea The Angular client already has a client-side QR code generator tool (`feature-dev-tools-qr-code`, `/development/tools/qr-code`, ng-qrcode). Add an **AI mode**: a button that generates artistic, scannable QR codes from the tool's content (text/URL/WiFi payload) using **monster-labs/control_v1p_sdxl_qrcode_monster** (https://huggingface.co/monster-labs/control_v1p_sdxl_qrcode_monster) — a ControlNet conditioned on a plain QR image, producing stylized codes that still scan. > **2026-07-28 rewrite.** The original ticket predated much of the infra it proposed building; it chose `control_v1p_sd15_qrcode_monster` because SD 1.5 + ControlNet fit the SERVER 3070 Ti. That rationale is stale: the 4090 (24 GB) is a permanent lane, and artpipe ships **no SD 1.5 base at all** — its text-to-image family is SDXL (`stabilityai/stable-diffusion-xl-base-1.0` + Lightning LoRA, SDXL-Turbo) and Flux. The **SDXL Monster variant** rides the already-baked SDXL base as ~2.5 GB of `install.extra_weights`, and diffusers builds a `StableDiffusionXLControlNetPipeline` from the loaded pipeline's components (same trick `sdxl_lightning.py` uses for img2img). The SD15 variant would mean adding and maintaining a whole second base-model family (new model dir, venv, baked image) — not worth it. Known tradeoff: the SDXL port typically needs `controlnet_conditioning_scale` tuning (~1.2–1.5) and more attempts per keeper than the battle-tested SD15 v2; that's one-time tuning, and an SD 1.5 lane could still be added later if scannability tuning truly disappoints. Also superseded below: GridFS → **MinIO** (#783 end state), and this is explicitly **NOT an Art Studio / ArtAsset flow** — it's a dedicated tools lane. ## Architecture (dedicated tools lane — not ArtAsset) Same delegated shape as the art lifecycle (CQRS command over RabbitMQ → per-model container → SignalR back), but its own command contract, queue, and storage prefix. No `ArtAsset`/`ArtAssetStageRun` rows, no art-studio UI involvement. **1. artpipe model (spikersoft-artpipe)** - `models/QRMonster/artpipe.json`: SDXL base via the same `install.weights`/`weight_patterns` shape as `SDXLLightning` + `install.extra_weights` `monster-labs/control_v1p_sdxl_qrcode_monster`; pip adds `qrcode[pil]` + `zxing-cpp`; `vram_mb` ~9500 (SDXL base ~6 GB booking + ~2.5 GB ControlNet + headroom — do NOT reuse the 6000 MB concept booking, the coordinator would grant a concurrent lease that OOMs). - Backend `src/artpipe/models/qr_monster.py` (pattern: `sdxl_lightning.py`, incl. its `_base_ref` offline-snapshot quirk), capability `qr_stylize`: - params: `content` (payload to encode), `error_correction_level`, `prompt` (style), `negative_prompt`, `controlnet_conditioning_scale`, `seed`, `size`, `steps`, optional `reference_image_path` + `strength` (user-supplied style/blend image → img2img init alongside the ControlNet conditioning). - flow: render plain QR from `content` worker-side with the `qrcode` lib at **error-correction level H** (that redundancy is what the stylization spends) → ControlNet conditioning image → generate → **self-verify scannability** by decoding the output (`zxing-cpp`); retry with escalating conditioning scale up to N attempts before returning `scannable: false` in the result payload. - Start with plain SDXL base at ~25–30 steps for quality; Lightning-LoRA few-step generation is a later optimization if latency hurts (ControlNet + Lightning needs its own conditioning-scale retune). **2. Baked image (existing pipeline — no new pattern)** - Add `qrmonster` → `MODELS="QRMonster"` to `.gitea/workflows/model-env-images.yml`. Prod model images run `HF_HUB_OFFLINE=1` — the ControlNet weights must be baked before any stage mapping flips, or it's a hard runtime failure. **3. Backend lane (spikersoft-backend)** - **Endpoint**: `POST api/tools/qr-art` (+ `GET api/tools/qr-art/{jobId}` for reconnect/polling fallback). **`[Authorize]` — logged-in users only.** This burns GPU minutes, so it is not a free anonymous operation like the client-side tool; add per-user rate limiting (e.g. small rolling quota) from day one. The button is the only gated part — the existing client-side QR tool stays anonymous. - **New first-class command contract** (e.g. `RequestQrArtCommand` in `SpikerSoft.Contracts.Workers`): `JobId`, `UserId`, `Content`, `ErrorCorrectionLevel`, `StylePrompt`, `NegativePrompt?`, `Seed?`, `ConditioningScale?`, `Size?`, `ReferenceImageKey?`, `Strength?`. All generation inputs are **first-class fields on the command** — deliberately not `GenerationParamOverrides`: the overridable allow-list (`ArtAssetGenerationParamKeys.Overridable`) reserves `image_path`/conditioning inputs as pipeline-only, and this lane shouldn't inherit art-studio's param plumbing at all. The client's generated QR is passed **by value** — `Content` + `ErrorCorrectionLevel` are exactly what the tool has encoded, and the worker re-renders that QR losslessly at ECC H print quality as the conditioning image (a screen-resolution client PNG at ECC M would waste the stylization headroom). The optional **reference image** IS a real upload: stored via the normal upload pipeline, referenced by `ReferenceImageKey`, fed as the img2img init image with `Strength` controlling how much of it survives. - **Routing**: publish to the per-model queue `art.model.qrmonster.task` (existing `ResidentModelDir`/`ModelQueueStages` convention) with W3C trace context. - **Container**: new swarm stack `spikersoft-artpipe-model-qrmonster` (copy an existing `spikersoft-artpipe-model-*` stack), `artpipe-gpu` placement label. Run in **model-queue mode** (per-job lease: acquire → load → run → unload; idle container holds 0 VRAM) — right fit for a sporadically-used tool; resident mode is the fallback if load latency proves annoying. - **Safety gate mandatory, fail-closed**: output is an arbitrary SDXL image on a minors platform — route through the existing SafetyCheck gate before the artifact becomes servable, exactly like art-studio stage outputs. - **Storage**: MinIO (S3 path, same as #783's art-asset end state) under a distinct prefix/bucket for tool artifacts; completion event carries the servable URL. - **Progress/completion**: worker NDJSON → `qr.art.{started,progress,completed,failed}` events → SignalR relay to the client (same delegated pattern as `signalr-art-asset-progress-source.ts`, own hub group keyed by JobId/UserId). **4. Angular client** - "AI style" button/tab in the existing QR tool: visible always, **enabled only when authenticated** (anonymous users get a "log in to generate AI QR codes" affordance). Sends the tool's current payload + ECC level + style prompt + optional reference image upload; progress via SignalR; result rendered from the MinIO-backed URL with download. - Client-side decode badge ("scans OK / may not scan") using the worker's `scannable` flag, with one-click regenerate (new seed). ## Sequencing / prerequisites - Weights baked into the env image **before** wiring the queue mapping (offline-mode hard failure otherwise). - Licensing: qrcode_monster is OpenRAIL-M — verify terms before shipping (commercial use allowed with use-based restrictions; outputs served on a minors platform → the safety gate is doing real work). - Optional later: LLM prompt-enrichment ("cyberpunk neon city" → full SD prompt) on the existing LLM worker infra; v1 ships with the raw prompt. - Related: #368 (per-model queues), #349 (safety gate), #783 (MinIO artifacts), #367 (GPU capacity).
spikerj added the enhancement label 2026-07-13 01:50:09 +00:00
spikerj commented 2026-07-13 01:58:14 +00:00
Author
Owner
Copy Link
Copy Source

Design correction (owner feedback 2026-07-13): no GridFS for QR artifacts — store in MinIO.

The integration doc above says 'artifacts up to GridFS (same as art_pipe stages)' — superseded: art_pipe stages happen to use GridFS today, but the program end-state (#413) is MinIO object storage, and this feature is NEW surface — it should be born on MinIO, not add another GridFS consumer to migrate later.

Revised artifact flow:

  • New qr-codes bucket + scoped write key for the worker (per-service key pattern from docs/minio-storage-migration.md).
  • Worker (or the .NET EventHandler after the safety gate) PUTs the generated PNG to qr-codes/<userId>/<jobId>.png via the existing AWSSDK.S3 wiring (Storage:UseS3 services already show the pattern; #533's shared IObjectStore is the ideal seam if it lands first).
  • Safety gate still fail-closed BEFORE the object becomes reachable: generate → gate → PUT on pass; quarantine path on flag stays in whatever the gate uses today.
  • Serving: presigned URL from the API (or the S3-backed media endpoint from #528) — the Angular tool gets a URL, never a host path.
  • Bonus: with MinIO in the loop, the QR worker container needs no volume of any kind — image is baked (#515), inputs via queue, outputs via S3. Fully floating.
**Design correction (owner feedback 2026-07-13): no GridFS for QR artifacts — store in MinIO.** The integration doc above says 'artifacts up to GridFS (same as art_pipe stages)' — superseded: art_pipe stages happen to use GridFS today, but the program end-state (#413) is MinIO object storage, and this feature is NEW surface — it should be born on MinIO, not add another GridFS consumer to migrate later. Revised artifact flow: - New `qr-codes` bucket + scoped write key for the worker (per-service key pattern from docs/minio-storage-migration.md). - Worker (or the .NET EventHandler after the safety gate) PUTs the generated PNG to `qr-codes/<userId>/<jobId>.png` via the existing AWSSDK.S3 wiring (`Storage:UseS3` services already show the pattern; #533's shared IObjectStore is the ideal seam if it lands first). - Safety gate still fail-closed BEFORE the object becomes reachable: generate → gate → PUT on pass; quarantine path on flag stays in whatever the gate uses today. - Serving: presigned URL from the API (or the S3-backed media endpoint from #528) — the Angular tool gets a URL, never a host path. - Bonus: with MinIO in the loop, the QR worker container needs no volume of any kind — image is baked (#515), inputs via queue, outputs via S3. Fully floating.
spikerj changed title from [Enhancement][ArtPipe][Angular] AI QR code generation — qrcode_monster ControlNet resident behind a per-model queue + AI mode in the QR tool to [Enhancement][ArtPipe][Angular] AI QR code generation — SDXL qrcode_monster ControlNet per-model lane + authenticated AI button in the QR tool 2026-07-28 02:13:52 +00:00
spikerj commented 2026-07-28 03:20:11 +00:00
Author
Owner
Copy Link
Copy Source

Implementation is up across all four repos (2026-07-28):

  • spikersoft-artpipe PR #44 — QRMonster model: manifest + qr_monster.py backend (qr_stylize: worker-rendered ECC-H conditioning QR with H→Q→M→L auto-downgrade, SDXL + control_v1p_sdxl_qrcode_monster ControlNet, optional img2img reference blend, zxing-cpp scannability self-verify with escalating conditioning scale), 13 tests, qrmonster CI image mapping (not in DEFAULT_IMAGES yet).
  • spikersoft-backend PR #501 — the qr-art lane: QrArtMessaging/GenerateQrArtCommand (first-class fields, QR by value, content never persisted — sha256 only), POST api/qrart/generate [Authorize] + rate limit + reference upload, QrArtConsumer co-hosted in ArtPipeProcessor (per-job 9500 MB lease, fail-closed SafetyCheck, MinIO), ReceiveQrArtStatus SignalR relay, 28 tests. Also adds the qrmonster stack to the deploy loop (merge infra first).
  • spikersoft-infrastructure PR #154 — spikersoft-artpipe-model-qrmonster stack (QrArt__Enabled=true, replicas: 0 until images exist).
  • spikersoft-angular PR #596 — the AI Artistic Style panel in the QR tool: auth-gated (login CTA for anonymous), style prompt + optional reference image, live SignalR progress with polling fallback, scannable badge + download/regenerate, en+es i18n.

Rollout order: artpipe #44 → build env image (dispatch images=qrmonster) → infra #154 → backend #501 → build artpipe-model-qrmonster (dispatch) → raise stack replicas to 1 → angular #596 → flip qrmonster into both DEFAULT_IMAGES lists.

Implementation is up across all four repos (2026-07-28): - **spikersoft-artpipe PR #44** — QRMonster model: manifest + `qr_monster.py` backend (`qr_stylize`: worker-rendered ECC-H conditioning QR with H→Q→M→L auto-downgrade, SDXL + `control_v1p_sdxl_qrcode_monster` ControlNet, optional img2img reference blend, zxing-cpp scannability self-verify with escalating conditioning scale), 13 tests, `qrmonster` CI image mapping (not in DEFAULT_IMAGES yet). - **spikersoft-backend PR #501** — the qr-art lane: `QrArtMessaging`/`GenerateQrArtCommand` (first-class fields, QR by value, content never persisted — sha256 only), `POST api/qrart/generate` `[Authorize]` + rate limit + reference upload, `QrArtConsumer` co-hosted in ArtPipeProcessor (per-job 9500 MB lease, fail-closed SafetyCheck, MinIO), `ReceiveQrArtStatus` SignalR relay, 28 tests. Also adds the qrmonster stack to the deploy loop (merge infra first). - **spikersoft-infrastructure PR #154** — `spikersoft-artpipe-model-qrmonster` stack (`QrArt__Enabled=true`, replicas: 0 until images exist). - **spikersoft-angular PR #596** — the AI Artistic Style panel in the QR tool: auth-gated (login CTA for anonymous), style prompt + optional reference image, live SignalR progress with polling fallback, scannable badge + download/regenerate, en+es i18n. **Rollout order**: artpipe #44 → build env image (dispatch `images=qrmonster`) → infra #154 → backend #501 → build `artpipe-model-qrmonster` (dispatch) → raise stack replicas to 1 → angular #596 → flip `qrmonster` into both DEFAULT_IMAGES lists.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agentic
Actionable surfaced by the agentic architecture & security review
 bug
Something is not working
 duplicate
This issue or pull request already exists
 enhancement
New feature
 help wanted
Need some help
 invalid
Something is wrong
 question
More information is needed
 sonarqube
Code quality issue surfaced by SonarQube (bug / code smell)
 wontfix
This won't be fixed
No labels enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
spikerj (Joseph Spiker)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: spikerj/spikersoft-issues#535