[Spike] Stream code-execution output incrementally over the existing SignalR channel #306

New Issue

2026-07-01T15:29:40Z

spikerj commented

2026-07-01 15:29:40 +00:00

Problem

When code runs on the server path, stdout/stderr are captured to completion and returned as a single CodeExecutionResponse. For longer runs (clang compile+run, x86 emulation, slower Python/JS), the learner stares at a spinner until the whole run finishes, then sees all output at once. Incremental output would be a meaningful UX improvement.

Current state (encouraging — transport already exists)

The frontend runner already uses SignalR for execution: ClangRunnerService holds a HubConnection (libraries/platform/clang-runtime/src/lib/clang-runner.service.ts), with execution-state signals (isExecuting, currentExecutionId, lastResult).
Execution is a distributed worker flow: the API publishes a request; SpikerSoft.EventHandlers.CodeExecution/Services/CodeExecutionWorkerService.cs runs it and returns one response.
Executors run a Process with RedirectStandardOutput/Error and currently buffer to completion (ReadToEndAsync) — e.g. PythonLessonExecutor, RoslynCodeExecutor, etc.
Precedent for line-streaming already in the codebase: HostCommandRunner.StreamShellLinesAsync (used by host monitoring) streams process output line-by-line via BeginOutputReadLine.
Some runtimes already sidestep the server entirely via browser/WASM grading (ClangRuntimeService worker), which is the strongest scaling answer where feasible.

So the transport (SignalR), the async worker pipeline, and a streaming primitive all exist — what's missing is emitting incremental chunks instead of one final blob.

Spike deliverables

Confirm the end-to-end path for incremental output: executor emits chunks → worker relays over the bus → API hub pushes to the client → runner appends to an output signal. Identify every hop that currently assumes a single terminal response.
Decide the grading boundary: streamed stdout is display-only; the pass/fail assertion must still run on the complete captured output. Confirm streaming doesn't change grading correctness (assertions compare full output/locals).
Assess risk on the hot path: backpressure/chunk flooding, output caps (tie to existing max-output limits), cancellation/timeout behavior mid-stream, ordering, and interleaving stdout/stderr.
Scope for one pilot runtime (suggest Python or C# server path) behind a flag, with transparent fallback to the current single-response behavior.
Recommend whether to invest here vs. expanding WASM/browser grading for more runtimes (which removes the backend from the hot path altogether).

Why the eventual change should be non-breaking

Gate streaming behind a flag with fallback to the existing buffered response; runtimes/paths not opted in behave exactly as today.
Grading logic unchanged — streaming only adds a display channel for partial stdout; the final assertion still runs on complete output.
Reuses the existing SignalR hub + worker pipeline; no new transport.

Acceptance criteria (spike)

Written feasibility + risk assessment covering the hops above, output caps, cancellation, and grading-correctness boundary.
Recommendation: stream-output vs. expand-WASM-grading (or both), with a pilot-runtime plan behind a flag.
A throwaway POC proving incremental chunks reach the client for one runtime (does not need to be production-ready).
Follow-up implementation ticket(s) opened from the recommendation.

Priority

Medium/low — pick up when run latency (not content delivery) becomes the learner complaint. Largest scope of the current perf items and touches the execution hot path, hence spike-first.

(Local-hosting only — no CDN.)

## Problem When code runs on the **server** path, stdout/stderr are captured to completion and returned as a single `CodeExecutionResponse`. For longer runs (clang compile+run, x86 emulation, slower Python/JS), the learner stares at a spinner until the whole run finishes, then sees all output at once. Incremental output would be a meaningful UX improvement. ## Current state (encouraging — transport already exists) - The frontend runner **already uses SignalR** for execution: `ClangRunnerService` holds a `HubConnection` (`libraries/platform/clang-runtime/src/lib/clang-runner.service.ts`), with execution-state signals (`isExecuting`, `currentExecutionId`, `lastResult`). - Execution is a **distributed worker** flow: the API publishes a request; `SpikerSoft.EventHandlers.CodeExecution/Services/CodeExecutionWorkerService.cs` runs it and returns one response. - Executors run a `Process` with `RedirectStandardOutput/Error` and currently **buffer to completion** (`ReadToEndAsync`) — e.g. `PythonLessonExecutor`, `RoslynCodeExecutor`, etc. - Precedent for line-streaming already in the codebase: `HostCommandRunner.StreamShellLinesAsync` (used by host monitoring) streams process output line-by-line via `BeginOutputReadLine`. - Some runtimes already sidestep the server entirely via **browser/WASM grading** (`ClangRuntimeService` worker), which is the strongest scaling answer where feasible. So the transport (SignalR), the async worker pipeline, and a streaming primitive all exist — what's missing is emitting incremental chunks instead of one final blob. ## Spike deliverables 1. Confirm the end-to-end path for incremental output: executor emits chunks → worker relays over the bus → API hub pushes to the client → runner appends to an output signal. Identify every hop that currently assumes a single terminal response. 2. Decide the **grading boundary**: streamed stdout is display-only; the pass/fail assertion must still run on the complete captured output. Confirm streaming doesn't change grading correctness (assertions compare full output/locals). 3. Assess risk on the hot path: backpressure/chunk flooding, output caps (tie to existing max-output limits), cancellation/timeout behavior mid-stream, ordering, and interleaving stdout/stderr. 4. Scope for **one pilot runtime** (suggest Python or C# server path) behind a flag, with transparent fallback to the current single-response behavior. 5. Recommend whether to invest here vs. **expanding WASM/browser grading** for more runtimes (which removes the backend from the hot path altogether). ## Why the eventual change should be non-breaking - Gate streaming behind a flag with fallback to the existing buffered response; runtimes/paths not opted in behave exactly as today. - Grading logic unchanged — streaming only adds a display channel for partial stdout; the final assertion still runs on complete output. - Reuses the existing SignalR hub + worker pipeline; no new transport. ## Acceptance criteria (spike) - [ ] Written feasibility + risk assessment covering the hops above, output caps, cancellation, and grading-correctness boundary. - [ ] Recommendation: stream-output vs. expand-WASM-grading (or both), with a pilot-runtime plan behind a flag. - [ ] A throwaway POC proving incremental chunks reach the client for one runtime (does not need to be production-ready). - [ ] Follow-up implementation ticket(s) opened from the recommendation. ## Priority Medium/low — pick up when run **latency** (not content delivery) becomes the learner complaint. Largest scope of the current perf items and touches the execution hot path, hence spike-first. _(Local-hosting only — no CDN.)_

spikerj added the enhancement label 2026-07-01 15:29:40 +00:00

spikerj referenced this issue

2026-07-15 18:21:29 +00:00

[Security][CI] OpenBao Phase 1: Gitea Actions secrets → AppRole fetch; rotate the committed runner token (epic #543) #545

spikerj referenced this issue

2026-07-15 18:49:14 +00:00

[Security][CI] OpenBao Phase 1: Gitea Actions secrets → AppRole fetch; rotate the committed runner token (epic #543) #545

Sign in to join this conversation.