[Bug][Backend][Security] Data Protection keys are ephemeral + unencrypted (container-local) — redeploy logs users out, replicas can't share keys #488

New Issue

2026-07-12T00:32:20Z

spikerj commented

2026-07-12 00:32:20 +00:00

Summary

ASP.NET Data Protection keys are stored in an ephemeral, container-local directory and are not encrypted at rest. On any container restart/redeploy the keyring is lost, and across swarm replicas the keys are not shared — both of which invalidate anything protected by those keys (auth cookies, antiforgery tokens, OIDC correlation/nonce, any IDataProtector payloads).

Evidence (Seq, Warning)

[Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository]
  Storing keys in a directory '/home/app/.aspnet/DataProtection-Keys' that may not be
  persisted outside of the container. Protected data will be unavailable when container
  is destroyed. (https://aka.ms/aspnet/dataprotectionwarning)

[Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager]
  No XML encryptor configured. Key {..} may be persisted to storage in unencrypted form.

Impact

Every redeploy logs users out and breaks in-flight flows that rely on protected payloads (antiforgery failures, OIDC login "correlation failed").
Multi-replica risk: replicas generate/hold different keyrings, so a cookie encrypted by replica A fails to decrypt on replica B -> intermittent auth failures under scale-out (not just on restart).
At rest: keys are stored unencrypted.

Proposed fix

Configure a shared, persistent, encrypted key ring in the API/services DI, e.g.:

services.AddDataProtection()
    .SetApplicationName("spikersoft")                 // stable, shared across replicas/services
    // persist to a shared store — Redis cluster already exists here:
    .PersistKeysToStackExchangeRedis(redisConn, "DataProtection-Keys")
    // or a shared volume mounted on all nodes:
    // .PersistKeysToFileSystem(new DirectoryInfo("/var/spikersoft/dp-keys"))
    .ProtectKeysWith...(...);                          // encrypt at rest (cert/KMS)

Given a Redis cluster is already deployed, PersistKeysToStackExchangeRedis is the lowest-friction option and fixes both the persistence and the multi-replica sharing problem.

Verify

After deploy, the FileSystemXmlRepository / No XML encryptor warnings disappear from Seq.
A redeploy no longer forces re-login; keys visible in the shared store.

Priority: medium-high (auth stability + security).

Filed proactively by automated health check (Seq Warning audit).

## Summary ASP.NET **Data Protection keys are stored in an ephemeral, container-local directory and are not encrypted at rest**. On any container restart/redeploy the keyring is lost, and across swarm replicas the keys are not shared — both of which invalidate anything protected by those keys (auth cookies, antiforgery tokens, OIDC correlation/nonce, any `IDataProtector` payloads). ## Evidence (Seq, Warning) ``` [Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository] Storing keys in a directory '/home/app/.aspnet/DataProtection-Keys' that may not be persisted outside of the container. Protected data will be unavailable when container is destroyed. (https://aka.ms/aspnet/dataprotectionwarning) [Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager] No XML encryptor configured. Key {..} may be persisted to storage in unencrypted form. ``` ## Impact - **Every redeploy logs users out** and breaks in-flight flows that rely on protected payloads (antiforgery failures, OIDC login "correlation failed"). - **Multi-replica risk:** replicas generate/hold different keyrings, so a cookie encrypted by replica A fails to decrypt on replica B -> intermittent auth failures under scale-out (not just on restart). - **At rest:** keys are stored unencrypted. ## Proposed fix Configure a shared, persistent, encrypted key ring in the API/services DI, e.g.: ```csharp services.AddDataProtection() .SetApplicationName("spikersoft") // stable, shared across replicas/services // persist to a shared store — Redis cluster already exists here: .PersistKeysToStackExchangeRedis(redisConn, "DataProtection-Keys") // or a shared volume mounted on all nodes: // .PersistKeysToFileSystem(new DirectoryInfo("/var/spikersoft/dp-keys")) .ProtectKeysWith...(...); // encrypt at rest (cert/KMS) ``` Given a Redis cluster is already deployed, `PersistKeysToStackExchangeRedis` is the lowest-friction option and fixes both the persistence and the multi-replica sharing problem. ## Verify - After deploy, the `FileSystemXmlRepository` / `No XML encryptor` warnings disappear from Seq. - A redeploy no longer forces re-login; keys visible in the shared store. Priority: medium-high (auth stability + security). --- _Filed proactively by automated health check (Seq Warning audit)._

spikerj commented

2026-07-12 00:44:54 +00:00

Triage — implementable once you pick the key store; parking the decision here rather than guessing on auth-critical infrastructure. Two viable shapes given our stack: (a) PersistKeysToStackExchangeRedis against the existing redis cluster (fixes BOTH redeploy-logout and replica-sharing; keys survive as long as redis does — note redis is cache-class here and only recently gained persistence, #371) or (b) PersistKeysToFileSystem on a SERVER bind mount (simplest, survives anything, but single-node — fine while the API is pinned to SERVER, breaks if the API ever joins the spikersoft-app burst pool). Both need SetApplicationName("spikersoft"). At-rest encryption (the second warning) needs ProtectKeysWithCertificate — requires a cert/secret you'd have to provision; can land as a follow-up to the persistence fix. Say the word on (a) or (b) and it's a small PR.

spikerj commented

2026-07-12 03:14:50 +00:00

Resolved in spikersoft-backend PR #218 (merged to master, commit c019f3e). New AddDataProtectionConfiguration persists the Data Protection key ring to the existing Redis cluster (RedisXmlRepository, shared across replicas + survives redeploy) with SetApplicationName("spikersoft"), and encrypts keys at rest via ProtectKeysWithCertificate when DataProtection:CertificatePath is provisioned (absent = non-fatal, warning logged, so deploys never brick before the cert exists). 5 xUnit wiring tests. Store=Redis and encryptor-in-PR per decision.

Ops follow-up to fully close the at-rest half: provision the PKCS#12 cert secret and set DataProtection__CertificatePath (+ __CertificatePassword) on the API stack — until then keys persist to Redis unencrypted with a startup warning. Closing the ticket; the cert provisioning is a small config step, not code.

Closing.

Resolved in spikersoft-backend PR #218 (merged to `master`, commit c019f3e). New `AddDataProtectionConfiguration` persists the Data Protection key ring to the existing **Redis cluster** (`RedisXmlRepository`, shared across replicas + survives redeploy) with `SetApplicationName("spikersoft")`, and encrypts keys at rest via `ProtectKeysWithCertificate` when `DataProtection:CertificatePath` is provisioned (absent = non-fatal, warning logged, so deploys never brick before the cert exists). 5 xUnit wiring tests. Store=Redis and encryptor-in-PR per decision. **Ops follow-up to fully close the at-rest half:** provision the PKCS#12 cert secret and set `DataProtection__CertificatePath` (+ `__CertificatePassword`) on the API stack — until then keys persist to Redis unencrypted with a startup warning. Closing the ticket; the cert provisioning is a small config step, not code. Closing.

spikerj closed this issue

2026-07-12 03:14:51 +00:00

spikerj referenced this issue

2026-07-21 01:11:19 +00:00

EPIC: Angular 22 + Nx 23 full-leverage modernization (signal forms, resource APIs, affected CI, remote cache) #722

spikerj referenced this issue

2026-07-26 19:31:01 +00:00

artpipe: 33 GB of immutable weights/venvs are baked into every CI build — make them data, not image layers #859

Sign in to join this conversation.