[Bug][Tests][QuizGeneration] ValidRequest_IsConsumedAcked_QuizCompletesWithFakedQuestions_AndNotifies fails on master — quiz contains questions from pages other than the expected page 4 #586

New Issue

2026-07-14T20:17:13Z

spikerj commented

2026-07-14 20:17:13 +00:00

Found while working #564 (PR #283) — not caused by it: I stashed the branch and re-ran the test on a clean master tree to confirm it fails there too, so filing it rather than smuggling a fix into an unrelated PR.

The failure — SpikerSoft.Tests.Integration/Quiz/QuizGenerationMessagePathIntegrationTests.cs:

Expected quiz.Questions to contain only items matching (q.SourcePageNumber == 4)
because pages 1-3 are front-matter the worker intentionally skips, but {...}

The test seeds a 4-page book and publishes a request with MaxPages=1, expecting the worker's fixed 3-page front-matter skip to leave exactly page 4 processed. The persisted quiz comes back holding questions whose SourcePageNumber is not 4.

Two candidate causes, and they matter very differently:

Test isolation — the quiz/book documents (or quiz.Questions) leak between tests in the TestContainers collection, so the assertion sees questions a previous test generated. Annoying, contained.
A real ComputePageWindow / resume bug — the worker is processing pages it should have skipped, or a resume path (quiz.CurrentPage > 0 && quiz.Questions.Count > 0) is re-appending. That would mean real quizzes are built from front-matter (copyright pages, ToC), which is a content-quality bug worth its own fix.

Worth resolving which before assuming it's "just a flaky test" — the assertion is checking a real product rule (don't quiz students on the copyright page), and it is currently failing.

Repro:

dotnet test SpikerSoft.Tests.Integration/SpikerSoft.Tests.Integration.csproj \
  --filter "FullyQualifiedName~ValidRequest_IsConsumedAcked"

The other two tests in the class pass. Possibly one of the two known non-cluster integration failures still awaiting triage.

Found while working #564 (PR #283) — **not caused by it**: I stashed the branch and re-ran the test on a clean `master` tree to confirm it fails there too, so filing it rather than smuggling a fix into an unrelated PR. **The failure** — `SpikerSoft.Tests.Integration/Quiz/QuizGenerationMessagePathIntegrationTests.cs`: ``` Expected quiz.Questions to contain only items matching (q.SourcePageNumber == 4) because pages 1-3 are front-matter the worker intentionally skips, but {...} ``` The test seeds a 4-page book and publishes a request with `MaxPages=1`, expecting the worker's fixed 3-page front-matter skip to leave exactly page 4 processed. The persisted quiz comes back holding questions whose `SourcePageNumber` is not 4. **Two candidate causes, and they matter very differently:** 1. **Test isolation** — the quiz/book documents (or `quiz.Questions`) leak between tests in the `TestContainers` collection, so the assertion sees questions a *previous* test generated. Annoying, contained. 2. **A real `ComputePageWindow` / resume bug** — the worker is processing pages it should have skipped, or a resume path (`quiz.CurrentPage > 0 && quiz.Questions.Count > 0`) is re-appending. That would mean real quizzes are built from front-matter (copyright pages, ToC), which is a content-quality bug worth its own fix. Worth resolving which before assuming it's "just a flaky test" — the assertion is checking a real product rule (don't quiz students on the copyright page), and it is currently failing. **Repro:** ```bash dotnet test SpikerSoft.Tests.Integration/SpikerSoft.Tests.Integration.csproj \ --filter "FullyQualifiedName~ValidRequest_IsConsumedAcked" ``` The other two tests in the class pass. Possibly one of the two known non-cluster integration failures still awaiting triage.

spikerj commented

2026-07-14 20:48:11 +00:00

Resolved in spikersoft-backend PR #284 (merged to master as 5085e57).

Answering the two hypotheses in the report: neither. Not test isolation, and not a front-matter bug — the worker was right and the test was stale. #507 (0ff694d) deliberately clamped the skip to min(3, (totalPages-1)/2) so short books aren't skipped past their own content, so a 4-page book correctly starts at page 2; PageWindowTests already pinned that with [InlineData(4, 6, 2, 4)]. The integration test kept asserting the pre-#507 fixed skip. Changing the worker to satisfy it would have re-introduced the #507 bug.

The test now seeds 7 pages (the smallest book that can afford the full 3-page skip), and a new ShortBook_IsQuizzedFromItsOwnContent_NotSkippedPastIt pins the clamp on the real message path — that guard's absence is why this sat failing instead of being caught. Closing.

Resolved in spikersoft-backend PR #284 (merged to `master` as `5085e57`). Answering the two hypotheses in the report: **neither**. Not test isolation, and not a front-matter bug — the worker was right and the test was stale. #507 (`0ff694d`) deliberately clamped the skip to `min(3, (totalPages-1)/2)` so short books aren't skipped past their own content, so a 4-page book correctly starts at page 2; `PageWindowTests` already pinned that with `[InlineData(4, 6, 2, 4)]`. The integration test kept asserting the pre-#507 fixed skip. Changing the worker to satisfy it would have re-introduced the #507 bug. The test now seeds 7 pages (the smallest book that can afford the full 3-page skip), and a new `ShortBook_IsQuizzedFromItsOwnContent_NotSkippedPastIt` pins the clamp on the real message path — that guard's absence is why this sat failing instead of being caught. Closing.

spikerj closed this issue

2026-07-14 20:48:16 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: spikerj/spikersoft-issues#586