[Bug][Prod][Infra] DLL decompiler service is built but never deployed — uploads RPC-timeout (no consumer on decompile.dll.upload.requested) #496

New Issue

2026-07-12T03:31:13Z

spikerj commented

2026-07-12 03:31:13 +00:00

Summary

DLL decompilation is non-functional in prod: every upload fails with a 60 s RPC timeout because the decompiler consumer service was built but never deployed — no CI workflow, no registry image, no swarm stack. The handler code + Dockerfile already exist; this ticket is the deployment wiring to make it run. (Concrete substance behind #412's "decompiler" item.)

Evidence (prod, learn.spikersoft.com)

DecompileController   Processing DLL upload: CarePayment.BlobStorageHelper.dll (11776 bytes)
RemoteDispatch (RPC)  publishing UploadDllCommand -> decompile/decompile.dll.upload.requested (await 60000ms)
[Error]               RPC timeout after 60000ms on decompile/decompile.dll.upload.requested
[Error] DecompileController  Error processing DLL upload — System.TimeoutException

docker service ls → no decompiler service in the swarm.
RabbitMQ → no decompile.* queue exists at all (nothing ever bound the exchange).
Registry → no spikersoft-decompile image published.

What exists vs missing

Piece	Status
`SpikerSoft.EventHandlers.Decompile` RPC consumer (`DecompileRpcConsumerHostedService`)	✅ complete — declares `decompile` topic exchange, binds `decompile.dll.upload.requested`→`UploadDllCommand`, MediatR dispatch, replies via direct-reply-to
`SpikerSoft.EventHandlers.Decompile/Dockerfile`	✅ exists
`SpikerSoft.Workers.Decompile`	✅ exists
CI workflow (`spikersoft-backend/.gitea/workflows/spikersoft-decompile.yml`)	❌ missing
Registry image `git.spikersoft.com/spikerj/spikersoft-decompile:latest`	❌ missing
Infra stack `spikersoft-infrastructure/spikersoft-decompile/docker-stack.yml`	❌ missing

Context: the decompiler was refactored out of the API into this event-handler (SpikerSoft.Api/Extensions/ServiceCollectionExtensions.cs:648-651 — "the API no longer references ICSharpCode.Decompiler"), but the build/deploy pipeline was never finished. This is an RPC/in-message flow — orthogonal to the MinIO migration (no bucket/bind-mount staging).

Work

CI workflow — spikersoft-backend/.gitea/workflows/spikersoft-decompile.yml, modeled exactly on spikersoft-embeddings.yml: build -f ./SpikerSoft.EventHandlers.Decompile/Dockerfile -t git.spikersoft.com/spikerj/spikersoft-decompile:latest ., push, then SSH-deploy docker stack deploy --with-registry-auth -c docker-stack.yml spikersoft-decompile, + Discord notify.
Infra stack — spikersoft-infrastructure/spikersoft-decompile/docker-stack.yml, commented gpu-coordinator style (guardrail #5). Stateless worker: pull the image, join the rabbitmq / seq-attachable / jaeger external overlays (and mongo only if the handler needs it), replicas: 1, node.labels.spikersoft-app == true placement, traefik.enable=false, restart on-failure. No bind mounts / no GPU. Add the folder→stack mapping to the README naming table.
Deploy + verify — image builds & pushes; docker service ps spikersoft-decompile Running; decompile.dll.upload.requested queue now has a consumer; re-upload a DLL and confirm it decompiles (no 60 s timeout) and DecompileController returns success.

Acceptance

Uploading CarePayment.BlobStorageHelper.dll returns a decompiled result, not a TimeoutException.
decompile.dll.upload.requested shows consumers>=1 in RabbitMQ.

Related: #412.

## Summary DLL decompilation is non-functional in prod: every upload fails with a 60 s RPC timeout because the decompiler consumer service **was built but never deployed** — no CI workflow, no registry image, no swarm stack. The handler code + Dockerfile already exist; this ticket is the deployment wiring to make it run. (Concrete substance behind #412's "decompiler" item.) ## Evidence (prod, learn.spikersoft.com) ``` DecompileController Processing DLL upload: CarePayment.BlobStorageHelper.dll (11776 bytes) RemoteDispatch (RPC) publishing UploadDllCommand -> decompile/decompile.dll.upload.requested (await 60000ms) [Error] RPC timeout after 60000ms on decompile/decompile.dll.upload.requested [Error] DecompileController Error processing DLL upload — System.TimeoutException ``` - `docker service ls` → **no** decompiler service in the swarm. - RabbitMQ → **no `decompile.*` queue exists at all** (nothing ever bound the exchange). - Registry → **no** `spikersoft-decompile` image published. ## What exists vs missing | Piece | Status | |---|---| | `SpikerSoft.EventHandlers.Decompile` RPC consumer (`DecompileRpcConsumerHostedService`) | ✅ complete — declares `decompile` topic exchange, binds `decompile.dll.upload.requested`→`UploadDllCommand`, MediatR dispatch, replies via direct-reply-to | | `SpikerSoft.EventHandlers.Decompile/Dockerfile` | ✅ exists | | `SpikerSoft.Workers.Decompile` | ✅ exists | | CI workflow (`spikersoft-backend/.gitea/workflows/spikersoft-decompile.yml`) | ❌ missing | | Registry image `git.spikersoft.com/spikerj/spikersoft-decompile:latest` | ❌ missing | | Infra stack `spikersoft-infrastructure/spikersoft-decompile/docker-stack.yml` | ❌ missing | Context: the decompiler was refactored out of the API into this event-handler (`SpikerSoft.Api/Extensions/ServiceCollectionExtensions.cs:648-651` — "the API no longer references ICSharpCode.Decompiler"), but the build/deploy pipeline was never finished. This is an RPC/in-message flow — **orthogonal to the MinIO migration** (no bucket/bind-mount staging). ## Work 1. **CI workflow** — `spikersoft-backend/.gitea/workflows/spikersoft-decompile.yml`, modeled exactly on `spikersoft-embeddings.yml`: build `-f ./SpikerSoft.EventHandlers.Decompile/Dockerfile -t git.spikersoft.com/spikerj/spikersoft-decompile:latest .`, push, then SSH-deploy `docker stack deploy --with-registry-auth -c docker-stack.yml spikersoft-decompile`, + Discord notify. 2. **Infra stack** — `spikersoft-infrastructure/spikersoft-decompile/docker-stack.yml`, commented gpu-coordinator style (guardrail #5). Stateless worker: pull the image, join the `rabbitmq` / `seq-attachable` / `jaeger` external overlays (and `mongo` only if the handler needs it), `replicas: 1`, `node.labels.spikersoft-app == true` placement, `traefik.enable=false`, restart on-failure. No bind mounts / no GPU. Add the folder→stack mapping to the README naming table. 3. **Deploy + verify** — image builds & pushes; `docker service ps spikersoft-decompile` Running; `decompile.dll.upload.requested` queue now has a consumer; re-upload a DLL and confirm it decompiles (no 60 s timeout) and `DecompileController` returns success. ## Acceptance - Uploading `CarePayment.BlobStorageHelper.dll` returns a decompiled result, not a `TimeoutException`. - `decompile.dll.upload.requested` shows `consumers>=1` in RabbitMQ. Related: #412.

spikerj commented

2026-07-12 03:52:00 +00:00

Resolved and verified in prod. Three PRs:

spikersoft-backend #220 — spikersoft-decompile.yml CI (build → push → deploy → notify).
spikersoft-infrastructure #34 — spikersoft-decompile/docker-stack.yml (stateless RPC worker, rabbitmq/seq-attachable/jaeger/redis overlays, spikersoft-app placement).
spikersoft-backend #221 — the actual blocker: the handler runs .WithRedis() but its appsettings.json had no Redis section, so AddEventHandlerRedis threw at startup → the worker crash-looped before binding the queue or logging to Seq (image built + stack deployed, yet zero worker logs + 60s RPC timeout). Added the shared cluster connection string.

Verified (Seq, 03:50–03:51 UTC): worker starts clean (Redis: True, listening :8084), DecompileRpcConsumer listening on decompile.dll.upload.requested, and CarePayment.BlobStorageHelper.dll decompiled end-to-end in ~0.3s (1 namespaces, 2 types, 21 members), cached to Redis, UploadDllCommand completed. Joey confirms uploads now work near-instantly.

Acceptance met: consumers>=1 on decompile.dll.upload.requested, uploads return a result instead of TimeoutException. Closing.

Minor follow-up (non-blocking): the worker logs a MediatR/Lucky Penny 'no valid license key' warning (same class as #487) — it runs fine in dev-allowed mode; wire MediatR:LicenseKey into this handler when convenient.

Resolved and verified in prod. Three PRs: - **spikersoft-backend #220** — `spikersoft-decompile.yml` CI (build → push → deploy → notify). - **spikersoft-infrastructure #34** — `spikersoft-decompile/docker-stack.yml` (stateless RPC worker, rabbitmq/seq-attachable/jaeger/redis overlays, `spikersoft-app` placement). - **spikersoft-backend #221** — the actual blocker: the handler runs `.WithRedis()` but its appsettings.json had no `Redis` section, so `AddEventHandlerRedis` threw at startup → the worker crash-looped before binding the queue or logging to Seq (image built + stack deployed, yet zero worker logs + 60s RPC timeout). Added the shared cluster connection string. **Verified (Seq, 03:50–03:51 UTC):** worker starts clean (`Redis: True`, listening :8084), `DecompileRpcConsumer listening on decompile.dll.upload.requested`, and `CarePayment.BlobStorageHelper.dll` decompiled end-to-end in ~0.3s (`1 namespaces, 2 types, 21 members`), cached to Redis, `UploadDllCommand completed`. Joey confirms uploads now work near-instantly. Acceptance met: consumers>=1 on `decompile.dll.upload.requested`, uploads return a result instead of TimeoutException. Closing. Minor follow-up (non-blocking): the worker logs a MediatR/Lucky Penny 'no valid license key' warning (same class as #487) — it runs fine in dev-allowed mode; wire `MediatR:LicenseKey` into this handler when convenient.

spikerj closed this issue

2026-07-12 03:52:00 +00:00

spikerj referenced this issue

2026-07-21 01:54:35 +00:00

EPIC: Angular 22 + Nx 23 full-leverage modernization (signal forms, resource APIs, affected CI, remote cache) #722

spikerj referenced this issue

2026-07-27 15:18:36 +00:00

EPIC: Sponsor family card redesign — image-backed destination tiles, achievements strip, parent-gated age #868

spikerj referenced this issue

2026-07-29 15:31:35 +00:00

Skill awards can trigger fund payouts (server skill-awarded activity + earned-skill-count provider) #874

Sign in to join this conversation.