spikerj/spikersoft-issues
1
0
Fork 0
Code Issues 196 Pull Requests Actions Packages Projects Releases Wiki Activity

[Infra][MinIO] Sensitive documents (passports/travel docs): GridFS → MinIO sole store (slice of epic #413) #773

New Issue
Closed
opened 2026-07-21 18:49:53 +00:00 by spikerj · 3 comments
spikerj commented 2026-07-21 18:49:53 +00:00
Owner
Copy Link
Copy Source

Next slice of epic #413, following the #771 profile-images recipe. Passports and travel documents are encrypted client-of-storage-side by SecureDocumentStorageService (IEncryptionService envelope + ISensitiveDataAuditService audit trail on every access) and the CIPHERTEXT lands in Mongo GridFS bucket sensitive-documents. Per the epic, blobs belong in MinIO — and this slice is low-risk precisely because MinIO only ever holds ciphertext; the encryption envelope and audit logging move over untouched.

Plan (hard cutover, #771 pattern):

  1. Extract the raw byte-store behind SecureDocumentStorageService into a small ISensitiveBlobStore seam (GridFS impl = local-dev fallback, S3 impl = prod via the shared SpikerSoft.Storage registration, bucket sensitive-documents, key = existing GridFS id so no Mongo document rewrites). Encryption/audit stay in the single service — no duplication.
  2. MinIO bucket + api-svc rw grant (live + provision-minio-svc-users.sh parity, infra PR).
  3. One-shot migration tool with per-object SHA-256 read-back verification (ciphertext copied verbatim), idempotent.
  4. Tests: seam unit tests both impls; all existing suites stay green.
  5. After deploy + live verification: drop GridFS sensitive-documents.files/.chunks (operator go-ahead, mongodump archive first — note archive contains ciphertext only).

Remaining GridFS holdouts after this: info-vault files (IGridFsVaultFileService), photograph preview store (#718).

Next slice of epic #413, following the #771 profile-images recipe. Passports and travel documents are encrypted client-of-storage-side by SecureDocumentStorageService (IEncryptionService envelope + ISensitiveDataAuditService audit trail on every access) and the CIPHERTEXT lands in Mongo GridFS bucket `sensitive-documents`. Per the epic, blobs belong in MinIO — and this slice is low-risk precisely because MinIO only ever holds ciphertext; the encryption envelope and audit logging move over untouched. Plan (hard cutover, #771 pattern): 1. Extract the raw byte-store behind SecureDocumentStorageService into a small ISensitiveBlobStore seam (GridFS impl = local-dev fallback, S3 impl = prod via the shared SpikerSoft.Storage registration, bucket `sensitive-documents`, key = existing GridFS id so no Mongo document rewrites). Encryption/audit stay in the single service — no duplication. 2. MinIO bucket + api-svc rw grant (live + provision-minio-svc-users.sh parity, infra PR). 3. One-shot migration tool with per-object SHA-256 read-back verification (ciphertext copied verbatim), idempotent. 4. Tests: seam unit tests both impls; all existing suites stay green. 5. After deploy + live verification: drop GridFS `sensitive-documents.files/.chunks` (operator go-ahead, mongodump archive first — note archive contains ciphertext only). Remaining GridFS holdouts after this: info-vault files (IGridFsVaultFileService), photograph preview store (#718).
spikerj commented 2026-07-21 18:58:24 +00:00
Author
Owner
Copy Link
Copy Source

Implementation up: spikersoft-backend PR #443 + spikersoft-infrastructure PR #135. Ops already executed: bucket sensitive-documents created, api-svc rw granted+verified, 2/2 encrypted envelopes migrated from GridFS with SHA-256 read-back verification. After #443 merges+deploys: spot-check a travel-doc/passport view in the UI, then archive (mongodump) + drop GridFS sensitive-documents.files/.chunks — will hold the drop for operator go-ahead as with #771.

Implementation up: spikersoft-backend PR #443 + spikersoft-infrastructure PR #135. Ops already executed: bucket sensitive-documents created, api-svc rw granted+verified, 2/2 encrypted envelopes migrated from GridFS with SHA-256 read-back verification. After #443 merges+deploys: spot-check a travel-doc/passport view in the UI, then archive (mongodump) + drop GridFS sensitive-documents.files/.chunks — will hold the drop for operator go-ahead as with #771.
spikerj commented 2026-07-21 19:18:26 +00:00
Author
Owner
Copy Link
Copy Source

Deployed (after a manual digest re-apply — the CI deploy silently rolled back on a transient pull failure, filed separately as #774). App started clean on the S3 sensitive-documents path; profile-image serving re-verified. GridFS sensitive-documents.* pre-archived to ~/backups/gridfs-sensitive-documents-20260721. Remaining: UI spot-check of a passport/travel-doc view, then operator go-ahead to drop the two GridFS collections.

Deployed (after a manual digest re-apply — the CI deploy silently rolled back on a transient pull failure, filed separately as #774). App started clean on the S3 sensitive-documents path; profile-image serving re-verified. GridFS sensitive-documents.* pre-archived to ~/backups/gridfs-sensitive-documents-20260721. Remaining: UI spot-check of a passport/travel-doc view, then operator go-ahead to drop the two GridFS collections.
spikerj commented 2026-07-21 19:34:43 +00:00
Author
Owner
Copy Link
Copy Source

Operator verified a passport loads through the MinIO path in the UI. GridFS sensitive-documents.files/.chunks dropped (0 remaining; mongodump archive at ~/backups/gridfs-sensitive-documents-20260721 on the 4090 — ciphertext only). MinIO is now the SOLE store for sensitive documents. Slice complete — remaining epic-#413 GridFS holdouts: info-vault files (IGridFsVaultFileService) and the photograph preview store (#718). Closing.

Operator verified a passport loads through the MinIO path in the UI. GridFS sensitive-documents.files/.chunks dropped (0 remaining; mongodump archive at ~/backups/gridfs-sensitive-documents-20260721 on the 4090 — ciphertext only). MinIO is now the SOLE store for sensitive documents. Slice complete — remaining epic-#413 GridFS holdouts: info-vault files (IGridFsVaultFileService) and the photograph preview store (#718). Closing.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agentic
Actionable surfaced by the agentic architecture & security review
 bug
Something is not working
 duplicate
This issue or pull request already exists
 enhancement
New feature
 help wanted
Need some help
 invalid
Something is wrong
 question
More information is needed
 sonarqube
Code quality issue surfaced by SonarQube (bug / code smell)
 wontfix
This won't be fixed
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
spikerj (Joseph Spiker)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: spikerj/spikersoft-issues#773