spikerj/spikersoft-issues
1
0
Fork 0
Code Issues 196 Pull Requests Actions Packages Projects Releases Wiki Activity

[Feature][Art Studio] Serve display images as AVIF derivatives (small card + full), keep PNG for download #805

New Issue
Closed
opened 2026-07-22 23:02:51 +00:00 by spikerj · 1 comment
spikerj commented 2026-07-22 23:02:51 +00:00
Owner
Copy Link
Copy Source

Summary

Art Studio display images are served as full-resolution PNGs (concept / enrichment Blender renders). Convert display-only images to AVIF to cut bytes dramatically, and produce a genuinely small card-width variant — today the card thumbnail is the same full-res PNG as the detail view. Keep the original PNG for download / export fidelity.

Pairs with the caching ticket #804 (which handles direct-MinIO delivery + cache headers); this ticket is the format/size layer.

Capability landscape (verified)

  • The only server-side AVIF encoder in the stack is the cavif Rust binary via AvifConverter.EncodeImageAsync (SpikerSoft.EventHandlers.MetadataExtractor/Services/AvifConverter.cs:21-67, quality 85 / speed 6) — the same engine that produces book covers and their responsive -{w}.avif ladder (CoverVariantGenerator.cs). cavif is currently baked only into the metadata-extractor Docker image.
  • The Python artpipe cannot emit AVIF (no image library; enrichment thumbs are raw Blender PNGs — pyproject.toml deps are stdlib-only). So AVIF must be a backend ingest-side transcode, not a pipeline change.
  • Blog/avatar AVIF is client-side (@jsquash/avif WASM, pre-upload) — not applicable here, since art images are backend-generated and never uploaded by the browser.
  • Bonus option: SpikerSoft.Business already references NetVips (libvips) (SpikerSoft.Business.csproj:73-74), which can write AVIF in-process (heifsave) if the pinned NetVips.Native.linux-x64 was built with libheif+AV1. A quick spike decides cavif-vs-NetVips.

Proposed work

  1. Spike (small): verify whether the pinned NetVips native can encode AVIF (load a PNG, attempt .avif write). If yes → in-process encode, no Docker/binary plumbing. If no → use cavif.
  2. If cavif: add the existing cavif-build Rust stage + COPY --from=cavif-build /usr/local/bin/cavif into the Dockerfile of the service that ingests art artifacts (ArtPipeProcessor / whichever writes to bucket art-asset-artifacts). Lift AvifConverter into a shared project (it's a self-contained static subprocess wrapper).
  3. At art-artifact ingest, for display-image kinds (image-png / concept-image / enrichment thumb), generate:
    • a small card-width AVIF (~320–400px) for grid tiles, and
    • a full-size AVIF for the detail / viewer,
      using ImageSharp for resize/letterbox → PNG → AVIF (mirroring CoverVariantGenerator), stored as sibling objects.
  4. Record the derivative keys/widths on the artifact ref (like Book.CoverVariantWidths). Serve AVIF URLs for display (via #804's presigned/cached delivery); keep the original PNG artifact for download/export.
  5. Frontend: pickCardThumbnailArtifact / display paths prefer the small AVIF for cards and full AVIF for detail; graceful fallback to PNG when no AVIF variant is recorded (older assets).

Out of scope / notes

  • Animated turntable previews (GIF / MP4) are not still images — leave as-is (animated AVIF is a separate effort).
  • Backfill of existing assets' AVIF derivatives is an optional follow-up: new assets get AVIF at ingest; old assets fall back to PNG until backfilled.

Acceptance criteria

  • New art assets expose small + full AVIF display variants; the card grid loads the small AVIF, not the full-res PNG.
  • Original PNG remains downloadable / exportable unchanged.
  • Assets without AVIF variants (pre-feature) still render via PNG fallback.
  • Delivered on top of #804 (presigned + cache headers).

References

  • Engine: AvifConverter + cavif; book covers #618 / #719, CoverVariantGenerator
  • Depends on / pairs with: #804 (caching + presigned delivery)
## Summary Art Studio display images are served as full-resolution PNGs (concept / enrichment Blender renders). Convert display-only images to **AVIF** to cut bytes dramatically, and produce a genuinely **small card-width variant** — today the card thumbnail is the same full-res PNG as the detail view. Keep the original PNG for download / export fidelity. Pairs with the caching ticket #804 (which handles direct-MinIO delivery + cache headers); this ticket is the format/size layer. ## Capability landscape (verified) - The **only** server-side AVIF encoder in the stack is the `cavif` Rust binary via `AvifConverter.EncodeImageAsync` (`SpikerSoft.EventHandlers.MetadataExtractor/Services/AvifConverter.cs:21-67`, quality 85 / speed 6) — the same engine that produces book covers and their responsive `-{w}.avif` ladder (`CoverVariantGenerator.cs`). `cavif` is currently baked **only** into the metadata-extractor Docker image. - The Python **artpipe cannot emit AVIF** (no image library; enrichment thumbs are raw Blender PNGs — `pyproject.toml` deps are stdlib-only). So AVIF must be a **backend ingest-side transcode**, not a pipeline change. - Blog/avatar AVIF is **client-side** (`@jsquash/avif` WASM, pre-upload) — not applicable here, since art images are backend-generated and never uploaded by the browser. - Bonus option: `SpikerSoft.Business` already references **NetVips (libvips)** (`SpikerSoft.Business.csproj:73-74`), which can write AVIF in-process (`heifsave`) **if** the pinned `NetVips.Native.linux-x64` was built with libheif+AV1. A quick spike decides cavif-vs-NetVips. ## Proposed work 1. **Spike (small):** verify whether the pinned NetVips native can encode AVIF (load a PNG, attempt `.avif` write). If yes → in-process encode, no Docker/binary plumbing. If no → use `cavif`. 2. **If cavif:** add the existing `cavif-build` Rust stage + `COPY --from=cavif-build /usr/local/bin/cavif` into the Dockerfile of the service that ingests art artifacts (`ArtPipeProcessor` / whichever writes to bucket `art-asset-artifacts`). Lift `AvifConverter` into a shared project (it's a self-contained static subprocess wrapper). 3. **At art-artifact ingest**, for display-image kinds (`image-png` / `concept-image` / enrichment `thumb`), generate: - a **small card-width AVIF** (~320–400px) for grid tiles, and - a **full-size AVIF** for the detail / viewer, using ImageSharp for resize/letterbox → PNG → AVIF (mirroring `CoverVariantGenerator`), stored as sibling objects. 4. Record the derivative keys/widths on the artifact ref (like `Book.CoverVariantWidths`). Serve AVIF URLs for display (via #804's presigned/cached delivery); **keep the original PNG artifact** for download/export. 5. **Frontend:** `pickCardThumbnailArtifact` / display paths prefer the small AVIF for cards and full AVIF for detail; graceful fallback to PNG when no AVIF variant is recorded (older assets). ## Out of scope / notes - Animated turntable previews (GIF / MP4) are not still images — leave as-is (animated AVIF is a separate effort). - Backfill of existing assets' AVIF derivatives is an optional follow-up: new assets get AVIF at ingest; old assets fall back to PNG until backfilled. ## Acceptance criteria - New art assets expose small + full AVIF display variants; the card grid loads the **small** AVIF, not the full-res PNG. - Original PNG remains downloadable / exportable unchanged. - Assets without AVIF variants (pre-feature) still render via PNG fallback. - Delivered on top of #804 (presigned + cache headers). ## References - Engine: `AvifConverter` + `cavif`; book covers `#618` / `#719`, `CoverVariantGenerator` - Depends on / pairs with: #804 (caching + presigned delivery)
spikerj commented 2026-07-29 15:48:07 +00:00
Author
Owner
Copy Link
Copy Source

Resolved in spikersoft-backend PR #459 (f7b58846) + spikersoft-angular PR #548 (31d753f9). Verified against origin/master:

Backend

  • ArtPipeProcessor/Services/ArtImageDerivativeGenerator.cs — ~400px card + full-size AVIF generated at ingest.
  • cavif build stage grafted into the ArtPipeProcessor Dockerfile; AvifConverter lifted to SpikerSoft.Common/Imaging/AvifConverter.cs so it's shared rather than duplicated.
  • ThumbnailAvifId / DisplayAvifId added to ArtAssetArtifactRef (ArtAsset.cs), minted at ArtStudioController.cs:148,152.
  • The original PNG is untouched, which was the ticket's constraint — downloads still get the lossless file.

Angular — and the consumers are actually wired, which is the failure mode I specifically checked for after finding several tickets this week where a service method shipped with no caller:

  • art-studio.service.ts:746-748 — fetchArtifactThumbnailUrl resolving thumbnailUrl ?? displayUrl ?? url ?? blob; :736-738 the display path; :890-897 the gallery counterparts.
  • art-studio-shell.component.ts:753 calls fetchArtifactThumbnailUrl from the card-grid loader, and art-gallery.component.ts switched over too. So the AVIF path is genuinely on the hot rendering route, not dormant.

One informational note, deliberately not counted against this ticket: the AVIF backfill of pre-feature assets did not ship and no backfill tool exists for it. The ticket body scopes that as "an optional follow-up" and it isn't in the acceptance criteria. (tools/BackfillPhotoStackDerivatives belongs to #806 and is a different pipeline — don't mistake it for this one.) If you want old assets to benefit, that needs its own ticket.

Closing.

Resolved in spikersoft-backend PR #459 (`f7b58846`) + spikersoft-angular PR #548 (`31d753f9`). Verified against `origin/master`: **Backend** - `ArtPipeProcessor/Services/ArtImageDerivativeGenerator.cs` — ~400px card + full-size AVIF generated at ingest. - cavif build stage grafted into the ArtPipeProcessor `Dockerfile`; `AvifConverter` lifted to `SpikerSoft.Common/Imaging/AvifConverter.cs` so it's shared rather than duplicated. - `ThumbnailAvifId` / `DisplayAvifId` added to `ArtAssetArtifactRef` (`ArtAsset.cs`), minted at `ArtStudioController.cs:148,152`. - **The original PNG is untouched**, which was the ticket's constraint — downloads still get the lossless file. **Angular — and the consumers are actually wired**, which is the failure mode I specifically checked for after finding several tickets this week where a service method shipped with no caller: - `art-studio.service.ts:746-748` — `fetchArtifactThumbnailUrl` resolving `thumbnailUrl ?? displayUrl ?? url ?? blob`; `:736-738` the display path; `:890-897` the gallery counterparts. - `art-studio-shell.component.ts:753` calls `fetchArtifactThumbnailUrl` from the card-grid loader, and `art-gallery.component.ts` switched over too. So the AVIF path is genuinely on the hot rendering route, not dormant. One informational note, deliberately not counted against this ticket: the **AVIF backfill of pre-feature assets did not ship** and no backfill tool exists for it. The ticket body scopes that as "an optional follow-up" and it isn't in the acceptance criteria. (`tools/BackfillPhotoStackDerivatives` belongs to #806 and is a different pipeline — don't mistake it for this one.) If you want old assets to benefit, that needs its own ticket. Closing.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agentic
Actionable surfaced by the agentic architecture & security review
 bug
Something is not working
 duplicate
This issue or pull request already exists
 enhancement
New feature
 help wanted
Need some help
 invalid
Something is wrong
 question
More information is needed
 sonarqube
Code quality issue surfaced by SonarQube (bug / code smell)
 wontfix
This won't be fixed
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
spikerj (Joseph Spiker)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: spikerj/spikersoft-issues#805