Joseph Spiker

spikerj closed issue spikerj/spikersoft-issues#490

2026-07-12 00:45:54 +00:00

[Bug][Backend] InfluxDB health check throws ArgumentException (duplicate 'write_successful' key) on query failure — masks real error

spikerj commented on issue spikerj/spikersoft-issues#490

2026-07-12 00:45:53 +00:00

[Bug][Backend] InfluxDB health check throws ArgumentException (duplicate 'write_successful' key) on query failure — masks real error

Resolved in spikersoft-backend PR #213 (merged to master). Both instances of the key-collision class fixed: (1) TestWriteAndQuery's catch now uses indexer assignment and preserves a successful…

spikerj commented on issue spikerj/spikersoft-issues#489

2026-07-12 00:44:54 +00:00

[Bug][Backend] Missing reference data on prod — GeoLite2-City.mmdb + postal-code dataset absent, features degraded

Triage — mostly provisioning, and the GeoIP half already has machinery: SpikerSoft.EventHandlers.Scheduler/TaskRunners/GeoIpUpdateTaskRunner.cs downloads GeoLite2 from MaxMind given a LicenseKey…

spikerj commented on issue spikerj/spikersoft-issues#488

2026-07-12 00:44:54 +00:00

[Bug][Backend][Security] Data Protection keys are ephemeral + unencrypted (container-local) — redeploy logs users out, replicas can't share keys

Triage — implementable once you pick the key store; parking the decision here rather than guessing on auth-critical infrastructure. Two viable shapes given our stack: (a) PersistKeysToStackExchang…

spikerj commented on issue spikerj/spikersoft-issues#487

2026-07-12 00:44:40 +00:00

[Bug][Backend] MediatR (Lucky Penny) commercial license expired — Error logged on every startup, renew before hard enforcement

Resolved in spikersoft-backend PR #212 (merged to master): Joey's renewed Lucky Penny key installed (decoded exp 2027-07-12 — calendar the next renewal for early July 2027). Also fixed along the…

spikerj closed issue spikerj/spikersoft-issues#487

2026-07-12 00:44:40 +00:00

[Bug][Backend] MediatR (Lucky Penny) commercial license expired — Error logged on every startup, renew before hard enforcement

spikerj opened issue spikerj/spikersoft-issues#490

2026-07-12 00:34:49 +00:00

[Bug][Backend] InfluxDB health check throws ArgumentException (duplicate 'write_successful' key) on query failure — masks real error

spikerj commented on issue spikerj/spikersoft-issues#486

2026-07-12 00:33:53 +00:00

[Bug][Prod] SafetyCheck art-pipe resident worker missing 'torch' → ArtPipeProcessor host crash-loops (24x)

Provisioning half fixed in spikersoft-artpipe PR #10: SafetyCheck's install.pip never listed torch — transformers treats it as optional and, uniquely among the models, SafetyCheck vendors no repo…

spikerj opened issue spikerj/spikersoft-issues#489

2026-07-12 00:32:20 +00:00

[Bug][Backend] Missing reference data on prod — GeoLite2-City.mmdb + postal-code dataset absent, features degraded

spikerj opened issue spikerj/spikersoft-issues#488

2026-07-12 00:32:20 +00:00

[Bug][Backend][Security] Data Protection keys are ephemeral + unencrypted (container-local) — redeploy logs users out, replicas can't share keys

spikerj commented on issue spikerj/spikersoft-issues#486

2026-07-12 00:32:20 +00:00

[Bug][Prod] SafetyCheck art-pipe resident worker missing 'torch' → ArtPipeProcessor host crash-loops (24x)

Upstream root cause confirmed from Seq Warnings: ServeWorkerProcessFactory logs 'Model venv python not found at /opt/art_pipe/models/SafetyCheck/venv/bin/python; falling back to python3.…

spikerj commented on issue spikerj/spikersoft-issues#485

2026-07-12 00:28:20 +00:00

[Observability][Backend] Client-abort exceptions (OperationCanceled / 'client reset request stream') logged as Error + 500 — noise in Seq/traces

Resolved in spikersoft-backend PR #211 (merged to master). Client aborts (OperationCanceledException tied to RequestAborted; Kestrel 'client reset the request stream' IOException) now log…

spikerj closed issue spikerj/spikersoft-issues#485

2026-07-12 00:28:20 +00:00

[Observability][Backend] Client-abort exceptions (OperationCanceled / 'client reset request stream') logged as Error + 500 — noise in Seq/traces

spikerj opened issue spikerj/spikersoft-issues#487

2026-07-12 00:26:57 +00:00

[Bug][Backend] MediatR (Lucky Penny) commercial license expired — Error logged on every startup, renew before hard enforcement

spikerj opened issue spikerj/spikersoft-issues#486

2026-07-12 00:26:57 +00:00

[Bug][Prod] SafetyCheck art-pipe resident worker missing 'torch' → ArtPipeProcessor host crash-loops (24x)

spikerj commented on issue spikerj/spikersoft-issues#484

2026-07-12 00:23:40 +00:00

[Bug][Infra] API /healthz returns 503 — TileServerGL check failing (tiles.spikersoft.com 404s, no tile-server deployed)

API half merged (spikersoft-backend PR #210) — /healthz now reports Degraded/200 instead of 503 when only the tile server is missing; CI auto-deploys the API so the liveness symptom clears with…

spikerj commented on issue spikerj/spikersoft-issues#481

2026-07-12 00:23:03 +00:00

[Bug][Infra] jaeger_elasticsearch down ~24h on SERVER (exit 1) — MaxAttempts exhausted, tracing backend offline

Correction after checking Jaeger directly (http://192.168.0.105:16686): tracing is NOT down — /api/services returns 200 and traces are queryable (e.g. trace 2890c9d4...). So Jaeger is serving.

Ro…

spikerj opened issue spikerj/spikersoft-issues#485

2026-07-12 00:23:02 +00:00

[Observability][Backend] Client-abort exceptions (OperationCanceled / 'client reset request stream') logged as Error + 500 — noise in Seq/traces

spikerj commented on issue spikerj/spikersoft-issues#484

2026-07-12 00:17:58 +00:00

[Bug][Infra] API /healthz returns 503 — TileServerGL check failing (tiles.spikersoft.com 404s, no tile-server deployed)

API-side fix in spikersoft-backend PR (fix/healthz-tileserver-degraded-484): the check now honors the registered FailureStatus and is registered as Degraded — /healthz reports Degraded (HTTP 200)…

spikerj commented on issue spikerj/spikersoft-issues#483

2026-07-12 00:17:58 +00:00

[Bug][Infra] InfluxDB rejecting a client every ~10s ("authorization not found") — stale/missing token on a poller

Triage from the repo side (no prod probing from this session): the ~10s cadence exactly matches Traefik's influxdb2 metrics push — traefik/docker-stack.yml pushes to https://influxdb.spikersoft.co…