[Bug][Tests][Curriculum] CurriculumSmokeTests picks whatever python3 is first on PATH — macOS system Python 3.9 fails the match-statement lessons (21300/21301) with a raw SyntaxError #591

New Issue

2026-07-14T20:43:33Z

spikerj commented

2026-07-14 20:43:33 +00:00

Found while running the full SpikerSoft.Business.Tests suite during #574. Fails on a clean master (verified by stashing the branch and re-running), so filing separately rather than folding it into an unrelated PR.

The failure — 2 of 93 CurriculumSmokeTests:

Python reference solution failed for lesson 21301:
  File "/var/folders/.../spikersoft_py_....py", line 3
    match p:
          ^
SyntaxError: invalid syntax

Same for lesson 21300 (match event:).

Root cause. match is structural pattern matching — Python 3.10+. The harness resolves its interpreter as:

var pythonExe = Environment.GetEnvironmentVariable("SPIKERSOFT_PYTHON")
                ?? (OperatingSystem.IsWindows() ? "python" : "python3");

On macOS, python3 on PATH is /usr/bin/python3 = Python 3.9.6 (the system interpreter). Lessons 21300/21301 exist precisely to teach match, so they cannot parse. Python 3.11 and 3.13 are both installed on this machine — they're just not what bare python3 resolves to.

This is not a curriculum bug and not a product bug. The codebase already knows the floor: CodeExecutionConfiguration.MinimumPythonVersion = "3.10", and PythonInterpreterProbeHostedService warns at startup with a message that literally cites "match statements, Python 3.10+". Production's coderunner Dockerfile installs a current python3. The test harness is the only place that doesn't apply the floor — it runs whatever the OS hands it.

Two things are wrong, and the second is the one that matters:

The harness doesn't select an interpreter capable of running the curriculum it is asserting.
When the interpreter is too old, the failure surfaces as a raw SyntaxError on line 3 of a temp file — which reads exactly like "the lesson's reference solution is broken". It cost real time to work out that the curriculum was fine and the interpreter was stale. A version mismatch must say so.

Fix shape: resolve a Python that meets MinimumPythonVersion (probe python3, then python3.13/3.12/3.11/3.10, take the first that satisfies the floor — PythonVersionGate.ParseVersion/MeetsMinimum already exist and are already unit-tested). Honour SPIKERSOFT_PYTHON when set, but validate it and fail loudly if it's below the floor. If no adequate interpreter exists, fail with an actionable message naming the found version and the required one — never a bare SyntaxError, and never a silent skip (a skip would hide genuine curriculum breakage, which is the whole point of these tests).

Found while running the full `SpikerSoft.Business.Tests` suite during #574. **Fails on a clean `master`** (verified by stashing the branch and re-running), so filing separately rather than folding it into an unrelated PR. **The failure** — 2 of 93 `CurriculumSmokeTests`: ``` Python reference solution failed for lesson 21301: File "/var/folders/.../spikersoft_py_....py", line 3 match p: ^ SyntaxError: invalid syntax ``` Same for lesson 21300 (`match event:`). **Root cause.** `match` is structural pattern matching — **Python 3.10+**. The harness resolves its interpreter as: ```csharp var pythonExe = Environment.GetEnvironmentVariable("SPIKERSOFT_PYTHON") ?? (OperatingSystem.IsWindows() ? "python" : "python3"); ``` On macOS, `python3` on PATH is `/usr/bin/python3` = **Python 3.9.6** (the system interpreter). Lessons 21300/21301 exist precisely to teach `match`, so they cannot parse. Python 3.11 and 3.13 are both installed on this machine — they're just not what bare `python3` resolves to. **This is not a curriculum bug and not a product bug.** The codebase already knows the floor: `CodeExecutionConfiguration.MinimumPythonVersion = "3.10"`, and `PythonInterpreterProbeHostedService` warns at startup with a message that literally cites *"`match` statements, Python 3.10+"*. Production's coderunner Dockerfile installs a current `python3`. The **test harness** is the only place that doesn't apply the floor — it runs whatever the OS hands it. **Two things are wrong, and the second is the one that matters:** 1. The harness doesn't select an interpreter capable of running the curriculum it is asserting. 2. When the interpreter is too old, the failure surfaces as a raw `SyntaxError` on line 3 of a temp file — which reads exactly like *"the lesson's reference solution is broken"*. It cost real time to work out that the curriculum was fine and the interpreter was stale. A version mismatch must say so. **Fix shape:** resolve a Python that meets `MinimumPythonVersion` (probe `python3`, then `python3.13`/`3.12`/`3.11`/`3.10`, take the first that satisfies the floor — `PythonVersionGate.ParseVersion`/`MeetsMinimum` already exist and are already unit-tested). Honour `SPIKERSOFT_PYTHON` when set, but **validate it** and fail loudly if it's below the floor. If no adequate interpreter exists, fail with an actionable message naming the found version and the required one — never a bare `SyntaxError`, and never a silent skip (a skip would hide genuine curriculum breakage, which is the whole point of these tests).

spikerj commented

2026-07-14 21:09:10 +00:00

Resolved in spikersoft-backend PR #287 (merged to master).

Root cause was neither the curriculum nor the product code: the test harness invoked bare python3, which on macOS resolves to the system Python 3.9. Lessons 21300/21301 teach structural pattern matching (match), which is 3.10+. The reference solutions were correct all along — the interpreter running them was too old.

CurriculumPythonInterpreter now probes python3, python3.13/3.12/3.11/3.10 and selects the first that satisfies the product's own CodeExecutionConfiguration.MinimumPythonVersion (3.10) via the existing PythonVersionGate, so the tests validate against the same floor production enforces rather than a second, hardcoded opinion. SPIKERSOFT_PYTHON is honoured but validated — if it points below the floor it throws with an actionable message.

Deliberately not a Skip: a skip here would have quietly stopped testing the match lessons on every dev machine, which is how this stayed invisible.

SpikerSoft.Business.Tests is now 7511 passed, 0 failed.

Closing.

Resolved in spikersoft-backend PR #287 (merged to `master`). Root cause was neither the curriculum nor the product code: the test harness invoked bare `python3`, which on macOS resolves to the system **Python 3.9**. Lessons 21300/21301 teach structural pattern matching (`match`), which is **3.10+**. The reference solutions were correct all along — the interpreter running them was too old. `CurriculumPythonInterpreter` now probes `python3`, `python3.13/3.12/3.11/3.10` and selects the first that satisfies the product's own `CodeExecutionConfiguration.MinimumPythonVersion` (3.10) via the existing `PythonVersionGate`, so the tests validate against the same floor production enforces rather than a second, hardcoded opinion. `SPIKERSOFT_PYTHON` is honoured but validated — if it points below the floor it throws with an actionable message. Deliberately **not** a `Skip`: a skip here would have quietly stopped testing the `match` lessons on every dev machine, which is how this stayed invisible. `SpikerSoft.Business.Tests` is now **7511 passed, 0 failed**. Closing.

spikerj closed this issue

2026-07-14 21:09:14 +00:00

spikerj referenced this issue

2026-07-14 21:35:20 +00:00

[Bug][CI][Deploy] cancel-in-progress on master kills DEPLOY jobs — back-to-back merges cancel each other and fixes silently never ship #590

spikerj referenced this issue

2026-07-27 15:18:36 +00:00

EPIC: Sponsor family card redesign — image-backed destination tiles, achievements strip, parent-gated age #868

spikerj referenced this issue

2026-07-29 15:29:23 +00:00

Fund-management admin page: recurrence/prize-kind enum drift (400s) + domain/action pickers #875

Sign in to join this conversation.