Activity telemetry: code-execution commands misattribute Python/JavaScript runs to csharp-playground domain #96

New Issue

2026-05-10T04:19:56Z

spikerj commented

2026-05-10 04:19:56 +00:00

Problem

ExecuteCodeCommand and ExecuteLessonCodeCommand are decorated with a hardcoded [TrackableActivity("tools.csharp-playground", ...)]:

// SpikerSoft.Business/Domain/CodeExecution/Commands/ExecuteCode/ExecuteCodeCommand.cs:10
[TrackableActivity("tools.csharp-playground", "run-code-success")]
public class ExecuteCodeCommand : IRequest<CodeExecutionSubmissionResponse> { ... }

// SpikerSoft.Business/Domain/CodeExecution/Commands/ExecuteLessonCode/ExecuteLessonCodeCommand.cs:10
[TrackableActivity("tools.csharp-playground", "run-lesson")]
public class ExecuteLessonCodeCommand : IRequest<CodeExecutionSubmissionResponse> { ... }

ActivityTrackingBehavior reads the attribute statically and writes a UserActivity { Domain, Action } row per request:

// SpikerSoft.Business/Behaviors/ActivityTrackingBehavior.cs:24–40
var attr = typeof(TRequest).GetCustomAttribute<TrackableActivityAttribute>();
if (attr is null) return response;
...
var activity = new UserActivity { ..., Domain = attr.Domain, Action = attr.Action, ... };

But the controller (/api/CSharpCodeRunner/{run|lesson}) is the only POST endpoint for code execution, and all three runners (C# / Python / JavaScript) hit it (see csharp-runner.service.ts:64, python-runner.service.ts:35, javascript-runner.service.ts:27). The runners differentiate themselves via metadata.language:

// python-runner.service.ts:211
metadata: { ...(request.metadata ?? {}), language: "python", ... }

// javascript-runner.service.ts:198
metadata: { ...(request.metadata ?? {}), language: "javascript", ... }

The backend already canonicalizes that field via CodeExecutionMetadataHelper.GetLanguageId(metadata) (defaults to "csharp" when absent). The UserLessonProgress durable completion path uses it correctly, so per-lesson completion is unaffected.

What IS broken is the secondary UserActivity row written by ActivityTrackingBehavior: every Python/JavaScript run/submit produces a row tagged tools.csharp-playground, drowning the legitimate C# rows and leaving Python/JavaScript with zero tools.{python,javascript}-playground / run-lesson activity events. Analytics, knowledge-domain progress aggregation, and cross-language usage reports are all silently misled.

Surfaced as a follow-up to #43 / #44 / #45 (per-lesson completion tracking) — those tickets are unaffected at the completion layer; this is purely the analytics layer.

Acceptance

UserActivity rows produced for Python submissions land under tools.python-playground.
UserActivity rows produced for JavaScript submissions land under tools.javascript-playground.
C# rows continue to land under tools.csharp-playground (default when metadata.language is missing or "csharp").
Solution is generic enough to extend to other language-aware commands without bespoke logic in ActivityTrackingBehavior for each command type.
Unit tests cover all three language paths plus the default.

Suggested fix

Introduce a small IActivityDomainOverride interface alongside TrackableActivityAttribute:

public interface IActivityDomainOverride
{
    (string Domain, string Action) ResolveActivity(string defaultDomain, string defaultAction);
}

Have ActivityTrackingBehavior call request.ResolveActivity(attr.Domain, attr.Action) when the request implements the interface, otherwise fall back to the static attribute values. ExecuteCodeCommand and ExecuteLessonCodeCommand implement it by reading Metadata through CodeExecutionMetadataHelper.GetLanguageId. Future cross-cutting commands can opt in the same way.

Owner pointers

SpikerSoft.Business/Behaviors/ActivityTrackingBehavior.cs — the pipeline behavior.
SpikerSoft.Business/Attributes/TrackableActivityAttribute.cs — the attribute (+ new interface).
SpikerSoft.Business/Domain/CodeExecution/Commands/Execute{Code,LessonCode}Command.cs — the two affected commands.
SpikerSoft.Business/Domain/CodeExecution/Validation/CodeExecutionMetadataHelper.cs — the existing language-id parser to reuse.
SpikerSoft.Tests.Unit/... — a FundraiserActivityTrackingTests.cs already shows the test pattern.

## Problem `ExecuteCodeCommand` and `ExecuteLessonCodeCommand` are decorated with a hardcoded `[TrackableActivity("tools.csharp-playground", ...)]`: ```cs // SpikerSoft.Business/Domain/CodeExecution/Commands/ExecuteCode/ExecuteCodeCommand.cs:10 [TrackableActivity("tools.csharp-playground", "run-code-success")] public class ExecuteCodeCommand : IRequest<CodeExecutionSubmissionResponse> { ... } // SpikerSoft.Business/Domain/CodeExecution/Commands/ExecuteLessonCode/ExecuteLessonCodeCommand.cs:10 [TrackableActivity("tools.csharp-playground", "run-lesson")] public class ExecuteLessonCodeCommand : IRequest<CodeExecutionSubmissionResponse> { ... } ``` `ActivityTrackingBehavior` reads the attribute statically and writes a `UserActivity { Domain, Action }` row per request: ```cs // SpikerSoft.Business/Behaviors/ActivityTrackingBehavior.cs:24–40 var attr = typeof(TRequest).GetCustomAttribute<TrackableActivityAttribute>(); if (attr is null) return response; ... var activity = new UserActivity { ..., Domain = attr.Domain, Action = attr.Action, ... }; ``` But the controller (`/api/CSharpCodeRunner/{run|lesson}`) is the only POST endpoint for code execution, and **all three runners** (C# / Python / JavaScript) hit it (see `csharp-runner.service.ts:64`, `python-runner.service.ts:35`, `javascript-runner.service.ts:27`). The runners differentiate themselves via `metadata.language`: ```ts // python-runner.service.ts:211 metadata: { ...(request.metadata ?? {}), language: "python", ... } // javascript-runner.service.ts:198 metadata: { ...(request.metadata ?? {}), language: "javascript", ... } ``` The backend already canonicalizes that field via `CodeExecutionMetadataHelper.GetLanguageId(metadata)` (defaults to `"csharp"` when absent). The `UserLessonProgress` *durable completion* path uses it correctly, so per-lesson completion is unaffected. What IS broken is the secondary `UserActivity` row written by `ActivityTrackingBehavior`: every Python/JavaScript run/submit produces a row tagged `tools.csharp-playground`, drowning the legitimate C# rows and leaving Python/JavaScript with **zero** `tools.{python,javascript}-playground / run-lesson` activity events. Analytics, knowledge-domain progress aggregation, and cross-language usage reports are all silently misled. Surfaced as a follow-up to #43 / #44 / #45 (per-lesson completion tracking) — those tickets are unaffected at the completion layer; this is purely the analytics layer. ## Acceptance - [ ] `UserActivity` rows produced for Python submissions land under `tools.python-playground`. - [ ] `UserActivity` rows produced for JavaScript submissions land under `tools.javascript-playground`. - [ ] C# rows continue to land under `tools.csharp-playground` (default when `metadata.language` is missing or `"csharp"`). - [ ] Solution is generic enough to extend to other language-aware commands without bespoke logic in `ActivityTrackingBehavior` for each command type. - [ ] Unit tests cover all three language paths plus the default. ## Suggested fix Introduce a small `IActivityDomainOverride` interface alongside `TrackableActivityAttribute`: ```cs public interface IActivityDomainOverride { (string Domain, string Action) ResolveActivity(string defaultDomain, string defaultAction); } ``` Have `ActivityTrackingBehavior` call `request.ResolveActivity(attr.Domain, attr.Action)` when the request implements the interface, otherwise fall back to the static attribute values. `ExecuteCodeCommand` and `ExecuteLessonCodeCommand` implement it by reading `Metadata` through `CodeExecutionMetadataHelper.GetLanguageId`. Future cross-cutting commands can opt in the same way. ## Owner pointers - `SpikerSoft.Business/Behaviors/ActivityTrackingBehavior.cs` — the pipeline behavior. - `SpikerSoft.Business/Attributes/TrackableActivityAttribute.cs` — the attribute (+ new interface). - `SpikerSoft.Business/Domain/CodeExecution/Commands/Execute{Code,LessonCode}Command.cs` — the two affected commands. - `SpikerSoft.Business/Domain/CodeExecution/Validation/CodeExecutionMetadataHelper.cs` — the existing language-id parser to reuse. - `SpikerSoft.Tests.Unit/...` — a `FundraiserActivityTrackingTests.cs` already shows the test pattern.

spikerj commented

2026-05-10 04:25:13 +00:00

Fixed.

Approach

Went with the suggested fix verbatim: a small IActivityDomainOverride interface alongside TrackableActivityAttribute, plus a PlaygroundActivityDomain helper that maps metadata.language to the registered playground domain.

Changes

`SpikerSoft.Business/Attributes/TrackableActivityAttribute.cs`

Added the override interface in the same file as the attribute (they're tightly coupled and both small):

public interface IActivityDomainOverride
{
    (string Domain, string Action) ResolveActivity(string defaultDomain, string defaultAction);
}

`SpikerSoft.Business/Behaviors/ActivityTrackingBehavior.cs`

The pipeline behavior now consults the interface when the request implements it; the static [TrackableActivity] values are passed in as defaults so implementations can swap one field without re-knowing the other:

var (domain, action) = (attr.Domain, attr.Action);
if (request is IActivityDomainOverride dynamicResolver)
{
    (domain, action) = dynamicResolver.ResolveActivity(attr.Domain, attr.Action);
}
var activity = new UserActivity { ..., Domain = domain, Action = action, ... };

`SpikerSoft.Business/Domain/CodeExecution/Validation/PlaygroundActivityDomain.cs` (new)

Maps the canonical metadata.language id (parsed by the existing CodeExecutionMetadataHelper.GetLanguageId) to the tools.<language>-playground domain registered in KnowledgeDomainRegistry. Falls back to a caller-provided default for unknown languages, so a future runtime that hasn't been added to the registry continues to land somewhere sensible instead of crashing.

`ExecuteCodeCommand` and `ExecuteLessonCodeCommand`

Both now implement IActivityDomainOverride:

public (string Domain, string Action) ResolveActivity(string defaultDomain, string defaultAction)
    => (PlaygroundActivityDomain.Resolve(Metadata, defaultDomain), defaultAction);

The [TrackableActivity("tools.csharp-playground", ...)] attributes stay in place as the default for legacy callers (and for the C# playground itself, which still doesn't bother sending metadata.language).

Tests added

PlaygroundActivityDomainTests (5 tests) — null/missing metadata defaults to csharp; python/Python/javascript/JavaScript/csharp map to their respective domains; unknown languages fall back to the provided default; custom fallback domains are passed through verbatim (forward-compat).
ActivityTrackingBehaviorTests (2 new tests, joining the 4 pre-existing) — dynamic override actually swaps the recorded (Domain, Action); static-attribute defaults reach the override.
CodeExecutionActivityDomainTests (8 new tests) — both real commands implement IActivityDomainOverride; both resolve correctly for python/javascript/csharp; both default to tools.csharp-playground when metadata.language is missing (preserves legacy behavior).

Verification

dotnet build SpikerSoft.Business/SpikerSoft.Business.csproj — 0 errors (74 pre-existing warnings unrelated to this change).
dotnet test --filter "~ActivityTracking|~PlaygroundActivityDomain|~CodeExecutionActivityDomain|~CodeExecutionMetadataHelper" — 59/59 passed, no regressions on the original ActivityTrackingBehaviorTests / CodeExecutionMetadataHelperTests / FundraiserActivityTrackingTests.
dotnet test --filter "~CodeExecution|~CSharpCodeRunner|~Activity" (broader sanity sweep) — 334/334 passed.

Acceptance check

UserActivity rows for Python submissions land under tools.python-playground.
UserActivity rows for JavaScript submissions land under tools.javascript-playground.
C# rows continue to land under tools.csharp-playground (default when metadata.language is missing or "csharp").
Solution is generic via IActivityDomainOverride — future cross-cutting commands opt in by implementing the interface; no per-command logic in ActivityTrackingBehavior.
Unit tests cover all three language paths plus the default.

Closing.

Fixed. ## Approach Went with the suggested fix verbatim: a small `IActivityDomainOverride` interface alongside `TrackableActivityAttribute`, plus a `PlaygroundActivityDomain` helper that maps `metadata.language` to the registered playground domain. ## Changes ### `SpikerSoft.Business/Attributes/TrackableActivityAttribute.cs` Added the override interface in the same file as the attribute (they're tightly coupled and both small): ```cs public interface IActivityDomainOverride { (string Domain, string Action) ResolveActivity(string defaultDomain, string defaultAction); } ``` ### `SpikerSoft.Business/Behaviors/ActivityTrackingBehavior.cs` The pipeline behavior now consults the interface when the request implements it; the static `[TrackableActivity]` values are passed in as defaults so implementations can swap one field without re-knowing the other: ```cs var (domain, action) = (attr.Domain, attr.Action); if (request is IActivityDomainOverride dynamicResolver) { (domain, action) = dynamicResolver.ResolveActivity(attr.Domain, attr.Action); } var activity = new UserActivity { ..., Domain = domain, Action = action, ... }; ``` ### `SpikerSoft.Business/Domain/CodeExecution/Validation/PlaygroundActivityDomain.cs` (new) Maps the canonical `metadata.language` id (parsed by the existing `CodeExecutionMetadataHelper.GetLanguageId`) to the `tools.<language>-playground` domain registered in `KnowledgeDomainRegistry`. Falls back to a caller-provided default for unknown languages, so a future runtime that hasn't been added to the registry continues to land somewhere sensible instead of crashing. ### `ExecuteCodeCommand` and `ExecuteLessonCodeCommand` Both now implement `IActivityDomainOverride`: ```cs public (string Domain, string Action) ResolveActivity(string defaultDomain, string defaultAction) => (PlaygroundActivityDomain.Resolve(Metadata, defaultDomain), defaultAction); ``` The `[TrackableActivity("tools.csharp-playground", ...)]` attributes stay in place as the default for legacy callers (and for the C# playground itself, which still doesn't bother sending `metadata.language`). ## Tests added - **`PlaygroundActivityDomainTests`** (5 tests) — null/missing metadata defaults to csharp; python/Python/javascript/JavaScript/csharp map to their respective domains; unknown languages fall back to the provided default; custom fallback domains are passed through verbatim (forward-compat). - **`ActivityTrackingBehaviorTests`** (2 new tests, joining the 4 pre-existing) — dynamic override actually swaps the recorded `(Domain, Action)`; static-attribute defaults reach the override. - **`CodeExecutionActivityDomainTests`** (8 new tests) — both real commands implement `IActivityDomainOverride`; both resolve correctly for python/javascript/csharp; both default to `tools.csharp-playground` when `metadata.language` is missing (preserves legacy behavior). ## Verification - `dotnet build SpikerSoft.Business/SpikerSoft.Business.csproj` — 0 errors (74 pre-existing warnings unrelated to this change). - `dotnet test --filter "~ActivityTracking|~PlaygroundActivityDomain|~CodeExecutionActivityDomain|~CodeExecutionMetadataHelper"` — **59/59 passed**, no regressions on the original `ActivityTrackingBehaviorTests` / `CodeExecutionMetadataHelperTests` / `FundraiserActivityTrackingTests`. - `dotnet test --filter "~CodeExecution|~CSharpCodeRunner|~Activity"` (broader sanity sweep) — **334/334 passed**. ## Acceptance check - [x] `UserActivity` rows for Python submissions land under `tools.python-playground`. - [x] `UserActivity` rows for JavaScript submissions land under `tools.javascript-playground`. - [x] C# rows continue to land under `tools.csharp-playground` (default when `metadata.language` is missing or `"csharp"`). - [x] Solution is generic via `IActivityDomainOverride` — future cross-cutting commands opt in by implementing the interface; no per-command logic in `ActivityTrackingBehavior`. - [x] Unit tests cover all three language paths plus the default. Closing.

spikerj closed this issue

2026-05-10 04:25:17 +00:00

spikerj referenced this issue

2026-07-02 02:30:43 +00:00

[Playgrounds P4] Text layer — SpikerSoft-authored prose per section (our own book, in-shell) #327

spikerj referenced this issue

2026-07-02 15:07:20 +00:00

[Playgrounds P3] Books layer — curated book links per subject/section in the shared shell #326

spikerj referenced this issue

2026-07-02 15:07:50 +00:00

[Epic] Playground uniformity & lesson content layers (C / C++ / JavaScript / Python / C#) #323

spikerj referenced this issue