Root-caused WITH production data and fixed — PR #234. Forensics (read-only via mongo-router): the bad book holds pages [1,1,2,2,3,3] under TWO ISBNs (both uploads of the same PDF); the…
Second code layer landed: PR #233 — the true mechanism of the zero-question quiz was the fixed pagesToSkip=3 making a 3-page book start at page 4 (its whole content skipped). New `ComputePag…
Reliability half fixed in spikersoft-backend PR #232 — both triage suspects confirmed and addressed: (a) SafetyCheckRpcConsumer now retries transport wiring with backoff instead of rethrowing…
✅ Fixed & verified — closing. Commit a4e9cd7b (fix(image-description): COPY *.py into the image + import smoke test in CI (#508)) landed the durable fix on master:
- Dockerfile: individual…
Fix up in spikersoft-infrastructure PR #42: drive-aware healthcheck. One correction to this ticket's option 1: curl was REMOVED from MinIO images in 2023, so the probe is mc-based — `mc ready…
Fix up in spikersoft-backend PR #231 (worker Failed-state + StartQuizAttempt guard + 400 mapping + 3 xUnit tests — details on the PR). The page-count-mismatch investigation (pages 4–6 'not…
Immediate outage is RESOLVED — image-description is back to 1/1, imports cleanly, consuming (message_failure.py committed to master + image rebuilt). BUT keep this open: the **durable root cause…
✅ Fixed & verified — closing. Landed on origin/master:
- New torch-free
message_failure.py:GpuClientNotReadyError(transient/retryable), `PERMANENT_FAILURES=(ValueError,KeyError,FileNotFound…
Fix up in spikersoft-backend PR #230, per this ticket's fix list:
COPY *.py ./in the Dockerfile (individual COPYs were the trap — a new sibling module could be imported by code yet…
⚠️ The fix for this is currently deployed but BROKEN — filed #508. The pushed image-description :latest (UpdatedAt 17:40 UTC) imports `from message_failure import GpuClientNotReadyError,…
Fix up in spikersoft-backend PR #229, implementing exactly the recommended combination — (1)+(3) durable, (2) defense-in-depth:
AttributeErroris no longer a permanent failure.…
Follow-up #506 traced to this incident's fallout path: the image-description worker's GPU-lease client can be None when the GpuCoordinator (SERVER-pinned) is transiently unreachable during a flap…
Related: filed #506 — a separate image-description defect surfaced while investigating this service. When the GPU-lease client is momentarily None (startup race before _gpu_client is set, or…