file-movement CI: secret/ci/backend/minio/movement never provisioned — pipeline can't deploy #918

New Issue

2026-08-05T15:55:17Z

spikerj commented

2026-08-05 15:55:17 +00:00

Surfaced 2026-08-05 when the #917 workflow merge fired every service pipeline: spikersoft-file-movement.yml failed in create_manifest at the bao-secrets fetch —

::error::no value at 'secret/ci/backend/minio/movement' field 'secret_key' ... : []

Reproduced identically on a quiet rerun, so not the burst-rate-limit class. This was the FIRST run of the file-movement pipeline in recent history — the gap is pre-existing, not caused by today's changes. The workflow itself carries the prerequisite hint: "run openbao/provision-minio-svc-users.sh first: secret/ci/backend/minio/movement (#613)" — that provisioning apparently never happened for movement (or the flaky mc admin user add skipped the kv write; movement IS in the script's ALL_SVCS and line ~167 writes exactly this path+field, and the ci-backend policy covers secret/ci/backend/*).

Fix (needs a human with BAO_TOKEN — CI AppRole creds deliberately never live on dev machines):

export BAO_TOKEN=<admin token>
ONLY_SVCS=movement openbao/provision-minio-svc-users.sh

then rerun backend run 19459 (POST /repos/spikerj/spikersoft-backend/actions/runs/19459/rerun) and it should go green end-to-end. Until then the file-movement service keeps running its previous image (deploy blocked, service itself unaffected).

Surfaced 2026-08-05 when the #917 workflow merge fired every service pipeline: `spikersoft-file-movement.yml` failed in `create_manifest` at the bao-secrets fetch — ``` ::error::no value at 'secret/ci/backend/minio/movement' field 'secret_key' ... : [] ``` Reproduced identically on a quiet rerun, so not the burst-rate-limit class. This was the FIRST run of the file-movement pipeline in recent history — the gap is pre-existing, not caused by today's changes. The workflow itself carries the prerequisite hint: *"run openbao/provision-minio-svc-users.sh first: secret/ci/backend/minio/movement (#613)"* — that provisioning apparently never happened for `movement` (or the flaky `mc admin user add` skipped the kv write; `movement` IS in the script's ALL_SVCS and line ~167 writes exactly this path+field, and the `ci-backend` policy covers `secret/ci/backend/*`). Fix (needs a human with BAO_TOKEN — CI AppRole creds deliberately never live on dev machines): ``` export BAO_TOKEN=<admin token> ONLY_SVCS=movement openbao/provision-minio-svc-users.sh ``` then rerun backend run 19459 (`POST /repos/spikerj/spikersoft-backend/actions/runs/19459/rerun`) and it should go green end-to-end. Until then the file-movement service keeps running its previous image (deploy blocked, service itself unaffected).

spikerj referenced this issue

2026-08-07 13:46:37 +00:00

[Bug][Infra][MinIO] file-movement's metadata-svc key can't write the quarantine bucket (Access Denied) — blocks #526/#529 cutover (epic #413) #613

spikerj commented

2026-08-07 13:46:38 +00:00

Duplicate of umbrella #613 — same root cause, same one-command fix. Both are closed into
spikerj/spikersoft-infrastructure#168, which carries the richer diagnosis.

Verified 2026-08-07 — docker service inspect spikersoft-file-movement_spikersoft-file-movement
still reports Storage__AccessKey=metadata-svc on a task last updated 12:25Z today, i.e. the
merged movement-svc stack env has never rolled because the deploy still fails at the Bao fetch.
openbao/provision-minio-svc-users.sh:20 and rotate-minio-ci-keys.sh:55 are both on infra
master and both honour ONLY_SVCS, so the fix really is
ONLY_SVCS=movement sudo -E bash rotate-minio-ci-keys.sh with an admin token.

Status: not started (the rotation has never been run).

Closing here. Work now lives in the repo that holds the fix, so fixes #168 in a PR will
auto-close it on merge. The umbrella tracker keeps cross-repo epics only.

— Opus 5 Agent

Duplicate of umbrella #613 — same root cause, same one-command fix. Both are closed into **spikerj/spikersoft-infrastructure#168**, which carries the richer diagnosis. Verified 2026-08-07 — `docker service inspect spikersoft-file-movement_spikersoft-file-movement` still reports `Storage__AccessKey=metadata-svc` on a task last updated 12:25Z today, i.e. the merged `movement-svc` stack env has never rolled because the deploy still fails at the Bao fetch. `openbao/provision-minio-svc-users.sh:20` and `rotate-minio-ci-keys.sh:55` are both on infra master and both honour `ONLY_SVCS`, so the fix really is `ONLY_SVCS=movement sudo -E bash rotate-minio-ci-keys.sh` with an admin token. Status: not started (the rotation has never been run). Closing here. Work now lives in the repo that holds the fix, so `fixes #168` in a PR will auto-close it on merge. The umbrella tracker keeps cross-repo epics only. — Opus 5 Agent

spikerj closed this issue

2026-08-07 13:46:38 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: spikerj/spikersoft-issues#918