Integration tests: Phase 5 multi-language grading (Python/Node/C/C++) delivered #452

New Issue

2026-07-07T16:15:28Z

spikerj commented

2026-07-07 16:15:28 +00:00

Summary

Phase 5 of the Lessons + code-execution/grading integration coverage expansion is complete and merged: spikersoft-backend#177.

Previous phases: #448 (0-2), #449 (3), #451 (4).

What landed

SpikerSoft.Tests.Integration.Lessons.MultiLanguageGradingIntegrationTests:
- Python (lesson 20105) / JavaScript (lesson 30104): real attempt->submit->pass/fail loop through the actual CPython/Node executors, now registered unconditionally in the in-process worker fixture (mirroring production DI). Capability-gated via a startup probe (TestEnvironmentFixture.PythonToolchainAvailable / NodeToolchainAvailable) - runs for real wherever the interpreter is on PATH, early-returns with a logged reason otherwise (xUnit 2.9.3 has no dynamic runtime Skip).
- C (lesson 80100) / C++ (lesson 70100): controller-level contract only - real attempt token, 202 + ExecutionId, and confirmation the worker consumed the published RabbitMQ message. The Clang/wasi-sdk/wasmtime toolchain is intentionally never registered in-process (it only exists baked into the worker's Docker image), so there's no "sometimes available" case for C/C++ the way there is for Python/Node.
Fixed a probe/config mismatch discovered while writing these tests: the fixture's Python capability probe accepted either python3 or python, but the worker's CodeExecution:PythonExecutable always defaulted to python3 - on hosts where only python exists (e.g. Windows dev boxes), this reported the toolchain "available" and then failed every real grading attempt. Fixed by feeding whichever binary the probe resolved into the worker's configuration.

Verification

dotnet test --filter FullyQualifiedName~MultiLanguageGradingIntegrationTests -> 6/6 passed locally (Python/JS exercised for real on this host; C/C++ via the publish-contract path).
Full SpikerSoft.Tests.Integration.Lessons namespace -> 42/42 passed locally, no regressions across all 5 phases.

Known follow-ups (not fixed here, out of scope for test-writing)

Free-play (/api/CCodeRunner/run, /api/CppCodeRunner/run) appears to route through the Roslyn C# path (ExecuteGeneralCodeAsync) rather than the registered CFreePlayCodeExecutor/CppFreePlayCodeExecutor in CodeExecutionWorkerService.ExecuteCodeAsync - the usePython || useJavaScript dispatch gate has no C/C++ equivalent, so free-play C/C++ submissions never reach the Clang executors even in production. Lesson submissions are unaffected (they dispatch via the strategy-registry's GradingRuntime correctly). Worth its own investigation/ticket; this Phase 5 work only covers the lesson-submission path for C/C++, which is unaffected by this.
Pre-existing test-host teardown flake (#450) did not reproduce during this phase's runs but remains open/unrelated.
The open follow-up already noted in the plan: containerize the code-execution worker image with wasi-sdk/wasmtime/python/node so this suite (and CI) can exercise the full multi-language grading loop unconditionally.

All 6 to-dos from the Lessons integration test plan (infra, catalog, anon, grading, tutorial, multilang) are now complete; only the cross-cutting delivery item (branch/PR/ticket per phase) remains, which this issue closes out for Phase 5.

## Summary Phase 5 of the Lessons + code-execution/grading integration coverage expansion is complete and merged: [spikersoft-backend#177](https://git.spikersoft.com/spikerj/spikersoft-backend/pulls/177). Previous phases: #448 (0-2), #449 (3), #451 (4). ## What landed - `SpikerSoft.Tests.Integration.Lessons.MultiLanguageGradingIntegrationTests`: - **Python** (lesson 20105) / **JavaScript** (lesson 30104): real attempt->submit->pass/fail loop through the actual CPython/Node executors, now registered unconditionally in the in-process worker fixture (mirroring production DI). Capability-gated via a startup probe (`TestEnvironmentFixture.PythonToolchainAvailable` / `NodeToolchainAvailable`) - runs for real wherever the interpreter is on PATH, early-returns with a logged reason otherwise (xUnit 2.9.3 has no dynamic runtime `Skip`). - **C** (lesson 80100) / **C++** (lesson 70100): controller-level contract only - real attempt token, `202` + `ExecutionId`, and confirmation the worker consumed the published RabbitMQ message. The Clang/wasi-sdk/wasmtime toolchain is intentionally never registered in-process (it only exists baked into the worker's Docker image), so there's no "sometimes available" case for C/C++ the way there is for Python/Node. - Fixed a probe/config mismatch discovered while writing these tests: the fixture's Python capability probe accepted either `python3` or `python`, but the worker's `CodeExecution:PythonExecutable` always defaulted to `python3` - on hosts where only `python` exists (e.g. Windows dev boxes), this reported the toolchain "available" and then failed every real grading attempt. Fixed by feeding whichever binary the probe resolved into the worker's configuration. ## Verification - `dotnet test --filter FullyQualifiedName~MultiLanguageGradingIntegrationTests` -> 6/6 passed locally (Python/JS exercised for real on this host; C/C++ via the publish-contract path). - Full `SpikerSoft.Tests.Integration.Lessons` namespace -> 42/42 passed locally, no regressions across all 5 phases. ## Known follow-ups (not fixed here, out of scope for test-writing) - Free-play (`/api/CCodeRunner/run`, `/api/CppCodeRunner/run`) appears to route through the Roslyn C# path (`ExecuteGeneralCodeAsync`) rather than the registered `CFreePlayCodeExecutor`/`CppFreePlayCodeExecutor` in `CodeExecutionWorkerService.ExecuteCodeAsync` - the `usePython || useJavaScript` dispatch gate has no C/C++ equivalent, so free-play C/C++ submissions never reach the Clang executors even in production. Lesson submissions are unaffected (they dispatch via the strategy-registry's `GradingRuntime` correctly). Worth its own investigation/ticket; this Phase 5 work only covers the lesson-submission path for C/C++, which is unaffected by this. - Pre-existing test-host teardown flake (#450) did not reproduce during this phase's runs but remains open/unrelated. - The open follow-up already noted in the plan: containerize the code-execution worker image with wasi-sdk/wasmtime/python/node so this suite (and CI) can exercise the full multi-language grading loop unconditionally. All 6 to-dos from the Lessons integration test plan (infra, catalog, anon, grading, tutorial, multilang) are now complete; only the cross-cutting `delivery` item (branch/PR/ticket per phase) remains, which this issue closes out for Phase 5.

spikerj closed this issue

2026-07-07 16:21:59 +00:00

spikerj referenced this issue

2026-07-22 02:46:00 +00:00

[Angular][Tech-debt] Migrate off deprecated @angular/animations DSL + animations modules (Angular 22 — SonarQube S1874 ×85) #686

spikerj referenced this issue

2026-07-22 03:51:12 +00:00

[Infra][MinIO] art-asset-artifacts: the ACTUAL last GridFS bucket → MinIO (ArtStudio stage outputs, ~4.5GB) (epic #413) #783

spikerj referenced this issue

2026-07-22 04:10:44 +00:00

[Perf][ArtStudio] Serve artifact images via short-TTL presigned MinIO URLs + HTTP caching — stop streaming every image through the API (follow-up to #783, epic #413) #784

Sign in to join this conversation.