[CI][Backend] sonar-scan has uploaded nothing since 2026-07-14 — scanner gets a wrong SonarQube host and continue-on-error hides the failure #632
Assignees
spikerj (Joseph Spiker)
Clear assignees
spikerj (Joseph Spiker)
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: spikerj/spikersoft-issues#632
Reference in New Issue
Block a user
Found while triaging SonarQube findings (2026-07-17).
api.spikersoft.com's last analysis is 2026-07-14T03:58Z despite ~35 master pushes since. The workflow check reports green on every one of them.Diagnosis (from the run 8437 job log, backend repo):
beginstep fails every run with:Downloading from https://www.sonarsource.com/products/sonarqube/api/server/version/ failed ... Pre-processing failed. Exit code: 1.sonar.host.url— i.e. the scanner is being handed the sonarscan-dotnet action's default host, not https://sonarqube.spikersoft.com. The workflow feedssonarHostname: ${{ env.SONARQUBE_HOST }}fromsecret/ci/backend/sonar#host(Bao, migrated in #545 batch 4), so thehostfield in Bao holds the wrong value (it is masked in logs, but the derived URL gives it away).continue-on-error: true(added for the #344 CE crash) swallows the exit code, so the check stays green — the same silent-inert failure mode as #583, on the quality pipeline itself.Consequences: every SonarQube finding for the backend reflects 07-14 code — 12+ of the 40 open vulnerabilities point at lines that no longer exist (e.g. SpikerSoft.Api/appsettings.json secrets already moved to Bao by #546 Stage 0). Triage on stale data wastes effort in both directions.
Fix (PR incoming):
sonarHostname: https://sonarqube.spikersoft.com— the host is not a secret; only the token needs Bao. Also side-steps the badhostfield until it is corrected (bao kv put secret/ci/backend/sonar host=... token=...— remember KV v2 put replaces the whole secret, write both fields).continue-on-error: trueand setsonar.qualitygate.wait=false: scanner/infra failures fail the check loudly again, while quality-gate outcomes stay non-blocking so the #344 auto-merge-deadlock concern cannot return.Unmasking worked as designed: with the host fixed, the PR run surfaced failure layer two — 13 QuizGenerationServiceTests fail inside the sonarscan container (LLama.Native type initializer; the container lacks the native lib prerequisites), which continue-on-error had been hiding along with everything else. Pushed a second commit to PR #341 excluding that one class from the scan container's test run only — tests.yml still runs them on every PR. Watching CI again.
Third commit on PR #341: dropped the pull_request trigger entirely (master push + workflow_dispatch only). Two independent reasons, both verified: (1) Community Edition submits PR-event scans as the MAIN analysis — the baseline silently stops describing master (this was quietly happening before 07-14); (2) each PR push queued a ~15-min full-solution scan, which is most of the current runner backlog, for feedback the tests.yml gate already provides. Branch protection requires no status contexts, so nothing can deadlock. Validation now happens on the post-merge master run — failures are loud since commit 1.
PR #341 got merged after commit 1 only (the host pin + unmasking); the LLama-test exclusion and master-only-trigger commits missed that merge. Re-landed via PR #345 (merged). The post-merge master scan on this state is the full validation — still watching for the CE task.
Milestone + a new (final) blocker, fully diagnosed:
The client-side chain is FIXED and proven: master run 8611 went begin → build → tests (LLama class excluded) → upload, end to end, and its report reached the server at 06:05. (Run 8582 failed on the LLama tests because it scanned the pre-#345 merge state — historical, self-resolved by #345.)
New blocker is server-side: the CE task processing that report FAILED in 257ms with
java.lang.ArrayIndexOutOfBoundsException: Index 58 out of bounds for length 55atorg.postgresql.util.PGbytea.getHexinsideExtractReportStep.appendPart— the stored report blob reads back as malformed bytea from the ce_task_input table. This is the concrete root cause of the original #344 'CE Task finished abnormally' crash: it only bites the backend's LARGE multi-part reports (learn.spikersoft.com's smaller reports process fine, most recent 06:0x successes). Suspect list: postgresbytea_output/driver mismatch or blob corruption on the sonarqube DB — needs a look at the SonarQube stack's postgres on the swarm (SERVER-local; will use a one-shot swarm job per the usual pattern if it reproduces).Reproducibility test in flight: dispatched a fresh master scan (workflow_dispatch). If its CE task fails identically → deterministic DB-layer bug → separate infra ticket; if it processes → the 06:05 blob was one-off corruption and the baseline lands.
RESOLVED — the fresh baseline is live. Proof chain: (1) CI run 8611 completed the fixed client pipeline end-to-end and uploaded; (2) a local-scanner submission processed server-side in ~45s; (3) analysisDate now 2026-07-17T06:29, open findings re-baselined 8,825 → 7,282, stale Api vulnerability lines auto-closed (37→25, the rest map to #633), BUG count 0 after re-applying one re-keyed FP. The one remaining defect from the diagnosis — the CE PGbytea failure on LARGE (coverage-bearing) container reports — is server-side and now tracked separately in #642 with full evidence. The scan itself is fixed: correct host, loud failures, master-only triggers, LLama-native tests excluded in the scan container. Closing.