spikerj/spikersoft-issues
1
0
Fork 0
Code Issues 196 Pull Requests Actions Packages Projects Releases Wiki Activity

[Bug][CI][E2E] e2e.staff credential drift — OpenBao staff_password no longer authenticates against Keycloak (student from same secret works) #628

New Issue
Closed
opened 2026-07-17 03:10:21 +00:00 by spikerj · 2 comments
spikerj commented 2026-07-17 03:10:21 +00:00
Owner
Copy Link
Copy Source

Symptom

A password grant for e2e.staff against Keycloak (spikersoft realm, spikersoft-web client, direct-access) returns:

{"error":"invalid_grant","error_description":"Invalid user credentials"}

even after freshly hydrating with pnpm e2e:secrets -- --staff (so the local .env.e2e staff field matches OpenBao's current staff_password). The e2e.student account from the same OpenBao secret (secret/ci/angular/e2e) authenticates fine via the same client — so this is specific to the staff account, not the client/flow.

Meaning

The staff_password stored in secret/ci/angular/e2e does not match the actual password on the Keycloak e2e.staff user — they've drifted. "Invalid user credentials" (rather than "account not fully set up") points to a plain password mismatch, not a pending required-action.

Impact

  • e2e-nightly uses the staff account (per the E2E accounts notes) → the staff/admin authenticated walk (P6 #319) would fail auth. Because authenticated specs are nightly-only and self-skip without creds, this can sit dark. See the CI test-coverage-gaps context.
  • Blocks staff-token verification generally (surfaced while trying to run a staff-token check for the reader PDF work — worked around via the app owner's own session instead).

Fix

Reconcile the two: either reset the Keycloak e2e.staff password to OpenBao's staff_password, or rotate secret/ci/angular/e2e staff_password to a new value and set it on the KC user in the same pass (write the whole secret at once — bao kv put replaces all fields). Then confirm: password grant for e2e.staff returns a token whose realm_access.roles includes staff.

Discovered 2026-07-17.

## Symptom A password grant for **`e2e.staff`** against Keycloak (`spikersoft` realm, `spikersoft-web` client, direct-access) returns: ``` {"error":"invalid_grant","error_description":"Invalid user credentials"} ``` even after freshly hydrating with `pnpm e2e:secrets -- --staff` (so the local `.env.e2e` staff field matches OpenBao's current `staff_password`). The **`e2e.student`** account from the **same** OpenBao secret (`secret/ci/angular/e2e`) authenticates fine via the same client — so this is specific to the staff account, not the client/flow. ## Meaning The `staff_password` stored in `secret/ci/angular/e2e` does **not** match the actual password on the Keycloak `e2e.staff` user — they've drifted. `"Invalid user credentials"` (rather than "account not fully set up") points to a plain password mismatch, not a pending required-action. ## Impact - **e2e-nightly uses the staff account** (per the E2E accounts notes) → the staff/admin authenticated walk (P6 #319) would fail auth. Because authenticated specs are nightly-only and self-skip without creds, this can sit dark. See the CI test-coverage-gaps context. - Blocks staff-token verification generally (surfaced while trying to run a staff-token check for the reader PDF work — worked around via the app owner's own session instead). ## Fix Reconcile the two: either reset the Keycloak `e2e.staff` password to OpenBao's `staff_password`, or rotate `secret/ci/angular/e2e` `staff_password` to a new value and set it on the KC user in the same pass (write the whole secret at once — `bao kv put` replaces all fields). Then confirm: password grant for `e2e.staff` returns a token whose `realm_access.roles` includes `staff`. Discovered 2026-07-17.
spikerj commented 2026-07-29 16:00:21 +00:00
Author
Owner
Copy Link
Copy Source

Audited — UNVERIFIABLE FROM GIT, with no positive evidence of a fix. Treat as still open.

Passwords live in OpenBao and Keycloak, neither of which is in git — correctly so, per the central secrets rule. So no commit could record the rotation, and git silence proves nothing either way. Saying that plainly rather than inferring.

What I could establish:

  • No indirect record of a reconciliation. openbao/provision-angular-secrets.sh exists among the provisioning scripts, but no commit or note records a staff_password fix.
  • The ticket has zero comments since filing on 2026-07-17 — twelve days.
  • Corroborating that it's still live: #605's 2026-07-22 comment states "#628 — e2e.staff credential drift — currently blocks the staff half," five days after this was filed.

Why this is worse than a stalled ticket. .gitea/workflows/e2e-nightly.yml:52-53 feeds staff_password into E2E_USER_PASSWORD, and authenticated specs self-skip when credentials fail (e2e/support/auth.ts:45, NO_CREDENTIALS_REASON) rather than erroring. So the nightly has most likely been passing green while executing no authenticated tests at all for twelve days. There's no alarm to notice, because a skip isn't a failure.

That also means this blocks more than #605: any regression in an authenticated flow during that window would have gone uncaught.

What would settle it: a password grant for e2e.staff against the spikersoft realm returning a token whose realm_access.roles includes staff, run after pnpm e2e:secrets -- --staff. Or simply an e2e-nightly run where the authenticated specs actually execute instead of skipping.

Worth fixing independently of the credential: make the suite fail rather than skip when credentials are expected but don't work. A silent skip in CI is indistinguishable from a pass, which is the property that let this sit for twelve days.

Audited — **UNVERIFIABLE FROM GIT, with no positive evidence of a fix. Treat as still open.** Passwords live in OpenBao and Keycloak, neither of which is in git — correctly so, per the central secrets rule. So no commit *could* record the rotation, and git silence proves nothing either way. Saying that plainly rather than inferring. What I could establish: - No indirect record of a reconciliation. `openbao/provision-angular-secrets.sh` exists among the provisioning scripts, but no commit or note records a `staff_password` fix. - The ticket has **zero comments** since filing on 2026-07-17 — twelve days. - Corroborating that it's still live: **#605**'s 2026-07-22 comment states "#628 — e2e.staff credential drift — currently blocks the staff half," five days after this was filed. **Why this is worse than a stalled ticket.** `.gitea/workflows/e2e-nightly.yml:52-53` feeds `staff_password` into `E2E_USER_PASSWORD`, and authenticated specs **self-skip** when credentials fail (`e2e/support/auth.ts:45`, `NO_CREDENTIALS_REASON`) rather than erroring. So the nightly has most likely been **passing green while executing no authenticated tests at all** for twelve days. There's no alarm to notice, because a skip isn't a failure. That also means this blocks more than #605: any regression in an authenticated flow during that window would have gone uncaught. **What would settle it:** a password grant for `e2e.staff` against the `spikersoft` realm returning a token whose `realm_access.roles` includes `staff`, run after `pnpm e2e:secrets -- --staff`. Or simply an e2e-nightly run where the authenticated specs actually execute instead of skipping. **Worth fixing independently of the credential:** make the suite **fail** rather than skip when credentials are expected but don't work. A silent skip in CI is indistinguishable from a pass, which is the property that let this sit for twelve days.
spikerj commented 2026-08-03 15:00:27 +00:00
Author
Owner
Copy Link
Copy Source

Verified fixed 2026-08-01. A password grant for e2e.staff against the spikersoft realm (spikersoft-web client, direct-access), using the current staff_password from secret/ci/angular/e2e, now succeeds:

  • grant returns HTTP 200
  • token's realm_access.roles = ["offline_access", "staff", "uma_authorization", "default-roles-spikersoft"] — includes staff

That is exactly the confirmation this ticket's Fix section asks for, so the OpenBao value and the Keycloak user are back in sync. The drift was reconciled at some point without the ticket being closed.

Note for anyone who lands here from the nightly: #628 is not the cause of the e2e-nightly failures. The authenticated specs self-skip wholesale when E2E_USER_* is unset, and they have been running; every failing test also passes the not.toContain("ids.spikersoft.com") assertion, so the Keycloak login works. Details in #726.

Closing.

Verified fixed 2026-08-01. A password grant for `e2e.staff` against the `spikersoft` realm (`spikersoft-web` client, direct-access), using the current `staff_password` from `secret/ci/angular/e2e`, now succeeds: - grant returns HTTP 200 - token's `realm_access.roles` = `["offline_access", "staff", "uma_authorization", "default-roles-spikersoft"]` — includes `staff` That is exactly the confirmation this ticket's **Fix** section asks for, so the OpenBao value and the Keycloak user are back in sync. The drift was reconciled at some point without the ticket being closed. Note for anyone who lands here from the nightly: **#628 is not the cause of the e2e-nightly failures.** The authenticated specs self-skip wholesale when `E2E_USER_*` is unset, and they have been *running*; every failing test also passes the `not.toContain("ids.spikersoft.com")` assertion, so the Keycloak login works. Details in #726. Closing.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agentic
Actionable surfaced by the agentic architecture & security review
 bug
Something is not working
 duplicate
This issue or pull request already exists
 enhancement
New feature
 help wanted
Need some help
 invalid
Something is wrong
 question
More information is needed
 sonarqube
Code quality issue surfaced by SonarQube (bug / code smell)
 wontfix
This won't be fixed
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
spikerj (Joseph Spiker)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: spikerj/spikersoft-issues#628