spikerj/spikersoft-issues
1
0
Fork 0
Code Issues 196 Pull Requests Actions Packages Projects Releases Wiki Activity

art_pipe observability activation: propagate JAEGER_ENDPOINT/SEQ_URL into the worker subprocess (#428) #459

New Issue
Closed
opened 2026-07-08 04:15:03 +00:00 by spikerj · 2 comments
spikerj commented 2026-07-08 04:15:03 +00:00
Owner
Copy Link
Copy Source

Now that #430 (OTel spans) and #431 (Seq log shipping) are merged in spikersoft-artpipe, the Python worker honors JAEGER_ENDPOINT / SEQ_URL env — but the .NET ArtPipeProcessor that spawns the subprocess doesn't set them, so both stay dormant in prod.

Scope (backend — this ticket)

Propagate the host's already-configured telemetry endpoints into the worker subprocess via the single shared SubprocessArtPipeStageExecutor.ApplyWorkerEnvironment seam (covers both the subprocess executor and the resident ServeWorkerProcess):

  • ArtPipeConfig: surface JaegerEndpoint / SeqUrl / SeqApiKey, bound in ArtPipeConfig.Bind from the existing root config keys Jaeger:EndPoint, Seq:ServerUrl, Seq:ApiKey (the exact keys per the telemetry-correlation rule — the host already reads them for its own traces/logs).
  • ApplyWorkerEnvironment: set psi.Environment["JAEGER_ENDPOINT"|"SEQ_URL"|"SEQ_API_KEY"] when non-empty.

No infra change: the ArtPipeProcessor container already joins the jaeger / seq-attachable overlays (it emits its own traces/logs), and the subprocess inherits network access.

Effect

  • #431 activates immediately in prod — Seq shipping is stdlib-only (urllib), so SEQ_URL alone lights up worker log↔trace correlation.
  • #430 is primed — JAEGER_ENDPOINT is set; spans start exporting the moment the model venvs carry the OTel packages.

Out of scope (separate follow-up)

Installing the opentelemetry-sdk + OTLP exporter into the per-model venvs — art_pipe is volume-mounted (not vendored in the image), so this belongs in art_pipe's venv_setup/bootstrap and carries a decision (which venvs get the telemetry extra). Filing separately.

Acceptance

  • With Jaeger:EndPoint / Seq:ServerUrl configured, the spawned worker's env carries JAEGER_ENDPOINT / SEQ_URL; empty config → env unset (worker untraced / local-log-only, unchanged).
  • Unit-tested on ApplyWorkerEnvironment + ArtPipeConfig.Bind; dotnet build SpikerSoft.Tests.Unit clean.

Refs #428, #430, #431.

Now that #430 (OTel spans) and #431 (Seq log shipping) are merged in spikersoft-artpipe, the Python worker honors `JAEGER_ENDPOINT` / `SEQ_URL` env — but the .NET `ArtPipeProcessor` that spawns the subprocess doesn't set them, so both stay dormant in prod. ## Scope (backend — this ticket) Propagate the host's already-configured telemetry endpoints into the worker subprocess via the single shared `SubprocessArtPipeStageExecutor.ApplyWorkerEnvironment` seam (covers both the subprocess executor and the resident `ServeWorkerProcess`): - `ArtPipeConfig`: surface `JaegerEndpoint` / `SeqUrl` / `SeqApiKey`, bound in `ArtPipeConfig.Bind` from the **existing** root config keys `Jaeger:EndPoint`, `Seq:ServerUrl`, `Seq:ApiKey` (the exact keys per the `telemetry-correlation` rule — the host already reads them for its own traces/logs). - `ApplyWorkerEnvironment`: set `psi.Environment["JAEGER_ENDPOINT"|"SEQ_URL"|"SEQ_API_KEY"]` when non-empty. No infra change: the ArtPipeProcessor container already joins the `jaeger` / `seq-attachable` overlays (it emits its own traces/logs), and the subprocess inherits network access. ## Effect - **#431 activates immediately** in prod — Seq shipping is stdlib-only (`urllib`), so `SEQ_URL` alone lights up worker log↔trace correlation. - **#430 is primed** — `JAEGER_ENDPOINT` is set; spans start exporting the moment the model venvs carry the OTel packages. ## Out of scope (separate follow-up) Installing the `opentelemetry-sdk` + OTLP exporter into the per-model venvs — art_pipe is volume-mounted (not vendored in the image), so this belongs in art_pipe's `venv_setup`/`bootstrap` and carries a decision (which venvs get the telemetry extra). Filing separately. ## Acceptance - With `Jaeger:EndPoint` / `Seq:ServerUrl` configured, the spawned worker's env carries `JAEGER_ENDPOINT` / `SEQ_URL`; empty config → env unset (worker untraced / local-log-only, unchanged). - Unit-tested on `ApplyWorkerEnvironment` + `ArtPipeConfig.Bind`; `dotnet build SpikerSoft.Tests.Unit` clean. Refs #428, #430, #431.
spikerj commented 2026-07-08 04:18:34 +00:00
Author
Owner
Copy Link
Copy Source

Implementing PR up for review: spikersoft-backend PR #183 — spikerj/spikersoft-backend#183

ArtPipeConfig.Bind now surfaces JaegerEndpoint/SeqUrl/SeqApiKey from the host's root Jaeger:EndPoint/Seq:ServerUrl/Seq:ApiKey, and ApplyWorkerEnvironment (shared by the subprocess executor + resident ServeWorkerProcess) sets JAEGER_ENDPOINT/SEQ_URL/SEQ_API_KEY on the worker subprocess env. #431 activates immediately (Seq shipping is stdlib-only → SEQ_URL alone); #430 primed for when the venvs carry OTel. dotnet build SpikerSoft.Tests.Unit clean; 24 config/env tests pass.

Leaving open until PR #183 merges. The per-model-venv OTel-package install is tracked separately (filing now).

Implementing PR up for review: **spikersoft-backend PR #183** — https://git.spikersoft.com/spikerj/spikersoft-backend/pulls/183 `ArtPipeConfig.Bind` now surfaces `JaegerEndpoint`/`SeqUrl`/`SeqApiKey` from the host's root `Jaeger:EndPoint`/`Seq:ServerUrl`/`Seq:ApiKey`, and `ApplyWorkerEnvironment` (shared by the subprocess executor + resident `ServeWorkerProcess`) sets `JAEGER_ENDPOINT`/`SEQ_URL`/`SEQ_API_KEY` on the worker subprocess env. #431 activates immediately (Seq shipping is stdlib-only → `SEQ_URL` alone); #430 primed for when the venvs carry OTel. `dotnet build SpikerSoft.Tests.Unit` clean; 24 config/env tests pass. Leaving **open** until PR #183 merges. The per-model-venv OTel-package install is tracked separately (filing now).
spikerj commented 2026-07-08 04:27:46 +00:00
Author
Owner
Copy Link
Copy Source

Resolved in spikersoft-backend PR #183 (merged to master as 4a4cded). ArtPipeConfig.Bind surfaces JaegerEndpoint/SeqUrl/SeqApiKey from the host's Jaeger:EndPoint/Seq:ServerUrl/Seq:ApiKey, and ApplyWorkerEnvironment sets JAEGER_ENDPOINT/SEQ_URL/SEQ_API_KEY on the worker subprocess env (both subprocess and resident modes). #431 Seq log shipping now activates in prod from SEQ_URL alone; #430 spans are primed pending the venv OTel packages (#460). Closing.

Resolved in spikersoft-backend **PR #183** (merged to `master` as `4a4cded`). `ArtPipeConfig.Bind` surfaces `JaegerEndpoint`/`SeqUrl`/`SeqApiKey` from the host's `Jaeger:EndPoint`/`Seq:ServerUrl`/`Seq:ApiKey`, and `ApplyWorkerEnvironment` sets `JAEGER_ENDPOINT`/`SEQ_URL`/`SEQ_API_KEY` on the worker subprocess env (both subprocess and resident modes). #431 Seq log shipping now activates in prod from `SEQ_URL` alone; #430 spans are primed pending the venv OTel packages (**#460**). Closing.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agentic
Actionable surfaced by the agentic architecture & security review
 bug
Something is not working
 duplicate
This issue or pull request already exists
 enhancement
New feature
 help wanted
Need some help
 invalid
Something is wrong
 question
More information is needed
 sonarqube
Code quality issue surfaced by SonarQube (bug / code smell)
 wontfix
This won't be fixed
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
spikerj (Joseph Spiker)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: spikerj/spikersoft-issues#459