[CI][Backend] Scheduled benchmark run for regression detection + fresh baseline — SpikerSoft.Benchmarks is compile-only in CI, baseline is from 2025-12-08 with pre-move namespaces #567

New Issue

2026-07-14T14:23:08Z

spikerj commented

2026-07-14 14:23:08 +00:00

QA Team — filed 2026-07-14 from the README audit, at spikerj's direction (decision: "want a scheduled CI job").

Current state:

CI (tests.yml) only compiles SpikerSoft.Benchmarks as part of the solution build — no benchmark ever runs in CI, so performance regressions are invisible until someone notices in-game.
The checked-in baseline in BenchmarkResults/ is from 2025-12-08 and its CSVs still use the pre-move SpikerSoft.Tests.Unit.* namespaces — it can't be diffed cleanly against a current run.
A full run spans ~7+ hours (17 suites), so it can't hang off a PR.

Asks:

Fresh baseline on current hardware — run the full suite once and commit the results as the new reference (the 4090 amd64 Gitea runner is the natural host; it's the only consistently-available x86 machine with headroom).
Scheduled job (e.g. weekly, schedule: cron in a dedicated workflow, workflow_dispatch too) that runs the suites and fails/annotates when a metric regresses beyond a threshold vs. the committed baseline. Consider splitting into per-suite jobs so a 7-hour serial run becomes several parallel shorter ones.
While in there: run-single-benchmark.ps1's ValidateSet lists only 8 names (one, SpatialIndex, isn't a real suite) vs run-benchmarks.ps1's 17 — sync or delete the script.

Context: the canonical tick rate is 30 ticks/sec (33.3 ms) — as of 2026-07-14 the benchmark code passes deltaTime = 0.0333f to match the budget tables (it previously passed 0.016f/~60 Hz, which skewed every per-tick figure — so the old baseline is doubly stale).

**QA Team** — filed 2026-07-14 from the README audit, at spikerj's direction (decision: "want a scheduled CI job"). **Current state:** - CI (`tests.yml`) only **compiles** SpikerSoft.Benchmarks as part of the solution build — no benchmark ever runs in CI, so performance regressions are invisible until someone notices in-game. - The checked-in baseline in `BenchmarkResults/` is from **2025-12-08** and its CSVs still use the pre-move `SpikerSoft.Tests.Unit.*` namespaces — it can't be diffed cleanly against a current run. - A **full run spans ~7+ hours** (17 suites), so it can't hang off a PR. **Asks:** 1. **Fresh baseline** on current hardware — run the full suite once and commit the results as the new reference (the 4090 amd64 Gitea runner is the natural host; it's the only consistently-available x86 machine with headroom). 2. **Scheduled job** (e.g. weekly, `schedule:` cron in a dedicated workflow, `workflow_dispatch` too) that runs the suites and fails/annotates when a metric regresses beyond a threshold vs. the committed baseline. Consider splitting into per-suite jobs so a 7-hour serial run becomes several parallel shorter ones. 3. While in there: `run-single-benchmark.ps1`'s ValidateSet lists only 8 names (one, `SpatialIndex`, isn't a real suite) vs `run-benchmarks.ps1`'s 17 — sync or delete the script. **Context:** the canonical tick rate is **30 ticks/sec (33.3 ms)** — as of 2026-07-14 the benchmark code passes `deltaTime = 0.0333f` to match the budget tables (it previously passed `0.016f`/~60 Hz, which skewed every per-tick figure — so the old baseline is doubly stale).

spikerj referenced this issue

2026-07-24 18:58:41 +00:00

Epic: Sponsor page — unified card grid, declutter donate buttons, and family shared-trip funds #827

spikerj referenced this issue

2026-07-24 19:24:14 +00:00

Epic: Sponsor page — unified card grid, declutter donate buttons, and family shared-trip funds #827

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: spikerj/spikersoft-issues#567