QA watch 2026-07-13 ~20:29Z — status changed since filing: /healthz now returns HTTP 200 with overall status 'Degraded' (was 503). The TileServerGL check is now classified Degraded rather than…
Cutover prep landed as two PRs: backend #264 (deploy-secret guards for blog-media/lesson-video — the only batch members missing them) + infra #60 GATED (Storage__ env for those two stacks +…
Backend PR #263: S3MediaFallbackMiddleware behind the four disk mounts — streaming proxy chosen over presigned (URLs and semantics unchanged → no Angular work; presign stays a later optimization)…
Backend PR #262: LessonVideoMoveConsumer dual-writes moved video+thumbnail to the lesson-videos bucket (same relative keys served today → #528/#531 cutover = prefix swap). Transcoder deliberately…
Backend PR #261: ObjectFileCache in SpikerSoft.Storage (local-wins → atomic bucket download → null), embeddings resolves its gguf directly, quiz-generation via a ModelPathResolver delegate so…
Backend PR #260: BlogMediaMoveConsumer dual-writes moved media to the blogs bucket (same relative keys nginx serves — #531 cutover becomes a prefix swap); file-movement routes /app/blogs.…
Correction + resolution of the CI thread (2026-07-13 ~19:57Z): the API's 'silent no-op deploy' I described was me misreading interleaved task lists — spikersoft-api.yml has NO deploy job at all…
Backend PR #259 (stacked on #258→#257): file-movement routes /app/assets moves to the assets bucket; ThreeDModelController dual-writes source images and falls back to the bucket for GLB downloads…
QA watch 2026-07-13 ~19:52Z — concrete CI impact of the dead arm runner: backend's new code is NOT live despite green-looking runs. Sequence: urgent-fix build pushed spikersoft-backend:amd64 +…
Backend PR #258 (stacked on #257): S3MovedFileMirror now routes moved files by destination tree — /app/dlls → dlls bucket via a keyed store; ebooks route unchanged. Finding: UploadCoordinator…
QA watch 2026-07-13 ~19:47Z — arm runner crash-loop root-caused: CONTAINER DNS, not gitea. Every restart logs: 'fail to invoke Declare … lookup git.spikersoft.com on 127.0.0.11:53 … i/o…
Backend PR #257: both quarantine writers (FileMovementService root tree + BlogMediaOrchestrator blogs/ subdir) dual-write to the quarantine bucket behind Storage:UseS3 — disk authoritative,…
QA watch 2026-07-13 ~19:41Z — dreamstream6 REJOINED at ~19:38Z (outage ~16:30→19:38Z, ≈3h). Host pings again, node State=ready.
Workload recovery: redis-cluster_redis-node-6 Running (one…
Additional finding while explaining the image size (2026-07-13 ~19:35Z) — the bloat has an exact inverse bug: the weights the model DOES need at runtime are NOT all baked.
Bloat mechanics,…
Pre-flight hardening PR: spikersoft-infrastructure#59 — --preserve-digests on every copy + optional authenticated Hub pulls (32/37 refs are Hub; an anonymous multi-arch sweep is ~100+ manifest…
Status split (verified live 2026-07-13 ~15:35Z):
FIXED — the 503 half: 41a25ab (failureStatus: Degraded + check honors Registration.FailureStatus) is deployed; /healthz now returns HTTP 200 with…
Postal half appears RESOLVED in prod: /healthz (checked 2026-07-13 ~15:35Z) shows 'Postal Code Data' Healthy — only TileServerGL (#484) is non-Healthy. GeoIP half: fix chain is in review (backend…
Hardening PR: spikersoft-infrastructure#58 — builder prune size cap (--keep-storage 60GB, LRU; the until=48h filter was actually exempting the hours-old outage cache), hourly timer, emergency…
Status: (1) exit-0 root cause fixed on master in 28c62fb (missing IDockerClient DI registration; ValidateOnBuild regression tests added). (3) CI stays-up smoke test: backend PR #255 — 15s no-deps…
QA watch 2026-07-13 ~19:18Z — the mirror is EMPTY and all three runs so far failed as outage collateral. MirrorUpstreamImages runs 10776/10779/10780 (02:10, 03:07, 04:00Z — manual dispatches;…