[ArtStudio] Honest GPU queue position — GpuCoordinator lease-queue emit → worker → SignalR → timeline #366

Closed
opened 2026-07-04 22:37:57 +00:00 by spikerj · 1 comment
Owner

Split from #364 (closed); epic #346. The stage timeline's queuePosition input has been plumbed since P5 but never fed, because IGpuLeaseService.AcquireAsync blocks inside the RPC with no queue visibility.

Scope (end-to-end):

  • GpuCoordinator: while a lease request waits, publish periodic queue-position updates (position + waiting count) keyed by the requesting service/job — either events on the existing gpu.lease.* direct-exchange family or a new lightweight topic; in-memory ledger stays authoritative (single replica unchanged)
  • SharedGpuLeaseService (or a callback seam on AcquireAsync): surface position updates to the caller while waiting
  • ArtPipeProcessor: forward position changes as art.asset.stage.progress.<stage> events (state Queued, new queuePosition field on ArtAssetStageProgressEvent) — throttled like progress
  • Relay/SignalR: field passes through ReceiveArtAssetStageProgress untouched
  • Angular: SignalRArtAssetProgressSource maps queuePosition into the existing timeline input; polling fallback may show position from the stage-run doc if the worker also persists it (optional)

Acceptance: with two submissions racing for the GPU, the second student's timeline shows "waiting for GPU — #2 in line" (or equivalent i18n), updating as the queue drains; staff-priority behavior unchanged; no events emitted when a lease is acquired immediately.

Split from #364 (closed); epic #346. The stage timeline's `queuePosition` input has been plumbed since P5 but never fed, because `IGpuLeaseService.AcquireAsync` blocks inside the RPC with no queue visibility. **Scope (end-to-end):** - [ ] GpuCoordinator: while a lease request waits, publish periodic queue-position updates (position + waiting count) keyed by the requesting service/job — either events on the existing `gpu.lease.*` direct-exchange family or a new lightweight topic; in-memory ledger stays authoritative (single replica unchanged) - [ ] `SharedGpuLeaseService` (or a callback seam on AcquireAsync): surface position updates to the caller while waiting - [ ] ArtPipeProcessor: forward position changes as `art.asset.stage.progress.<stage>` events (state Queued, new `queuePosition` field on `ArtAssetStageProgressEvent`) — throttled like progress - [ ] Relay/SignalR: field passes through `ReceiveArtAssetStageProgress` untouched - [ ] Angular: `SignalRArtAssetProgressSource` maps `queuePosition` into the existing timeline input; polling fallback may show position from the stage-run doc if the worker also persists it (optional) **Acceptance:** with two submissions racing for the GPU, the second student's timeline shows "waiting for GPU — #2 in line" (or equivalent i18n), updating as the queue drains; staff-priority behavior unchanged; no events emitted when a lease is acquired immediately.
spikerj added the enhancement label 2026-07-04 22:37:57 +00:00
Author
Owner

Resolved: backend PR #73 + angular PR #122, both merged to master. GpuCoordinator emits change-only queue-status (plus 5 s heartbeat) on a new additive gpu.lease.queue-status exchange; IGpuLeaseService gained an optional callback via a default interface method (all other GPU tenants zero-diff); ArtPipeProcessor forwards position changes as Queued-state progress events and persists queuePosition on the stage run (SignalR and polling covered); the timeline now shows "Waiting for GPU — #N in line" live. 418/418 backend + 128/128 lib tests. Live two-submission race noted for the next full-stack session (unit tests pin ordering semantics). Closing.

Resolved: backend PR #73 + angular PR #122, both merged to `master`. GpuCoordinator emits change-only queue-status (plus 5 s heartbeat) on a new additive `gpu.lease.queue-status` exchange; `IGpuLeaseService` gained an optional callback via a default interface method (all other GPU tenants zero-diff); ArtPipeProcessor forwards position changes as Queued-state progress events and persists `queuePosition` on the stage run (SignalR *and* polling covered); the timeline now shows "Waiting for GPU — #N in line" live. 418/418 backend + 128/128 lib tests. Live two-submission race noted for the next full-stack session (unit tests pin ordering semantics). Closing.
Sign in to join this conversation.