spikerj/spikersoft-issues
1
0
Fork 0
Code Issues 196 Pull Requests Actions Packages Projects Releases Wiki Activity

[Infra][MinIO] Migrate quarantine dir to S3 (quarantine bucket, epic #413) #526

New Issue
Closed
opened 2026-07-13 00:40:40 +00:00 by spikerj · 3 comments
spikerj commented 2026-07-13 00:40:40 +00:00
Owner
Copy Link
Copy Source

Parent epic: #413.

Scope: quarantine bucket — /mnt/fusionio/spikersoft/quarantine (/app/quarantine). Note: Art Studio quarantine is already GridFS (GridFsArtAssetQuarantineService) — this ticket covers only the upload/blog quarantine directory used by file-movement, upload-coordinator, blog-media-processor (BlogController.cs:1203-1245 references it).

Steps: bucket + scoped key (write-mostly; consider a lifecycle/retention rule on the bucket) → backend PR: quarantine writes go to S3 behind Storage:UseS3 → backfill existing quarantined objects → dual-run → drop binds with the uploads batch.

Parent epic: #413. **Scope:** `quarantine` bucket — `/mnt/fusionio/spikersoft/quarantine` (`/app/quarantine`). Note: **Art Studio quarantine is already GridFS** (`GridFsArtAssetQuarantineService`) — this ticket covers only the upload/blog quarantine directory used by file-movement, upload-coordinator, blog-media-processor (`BlogController.cs:1203-1245` references it). **Steps:** bucket + scoped key (write-mostly; consider a lifecycle/retention rule on the bucket) → backend PR: quarantine writes go to S3 behind `Storage:UseS3` → backfill existing quarantined objects → dual-run → drop binds with the uploads batch.
spikerj commented 2026-07-13 19:44:49 +00:00
Author
Owner
Copy Link
Copy Source

Backend PR #257: both quarantine writers (FileMovementService root tree + BlogMediaOrchestrator blogs/ subdir) dual-write to the quarantine bucket behind Storage:UseS3 — disk authoritative, octet-stream forced, shared-root key layout. Storage seam gained AddKeyedS3ObjectStore since FileMovement is the first two-bucket service (ebooks + quarantine). Bucket provisioning + backfill + bind-drop ride with the #529 uploads batch.

Backend PR #257: both quarantine writers (FileMovementService root tree + BlogMediaOrchestrator blogs/ subdir) dual-write to the quarantine bucket behind Storage:UseS3 — disk authoritative, octet-stream forced, shared-root key layout. Storage seam gained AddKeyedS3ObjectStore since FileMovement is the first two-bucket service (ebooks + quarantine). Bucket provisioning + backfill + bind-drop ride with the #529 uploads batch.
spikerj commented 2026-07-13 20:32:39 +00:00
Author
Owner
Copy Link
Copy Source

Code MERGED (backend #257). Remaining = ops via the #529 runbook: quarantine bucket + keys (blogs-svc write, file-movement policy widening), backfill, Storage__UseS3 flip (already true on file-movement stack; blog-media rides infra #60), bind drop at phase 2.

Code MERGED (backend #257). Remaining = ops via the #529 runbook: quarantine bucket + keys (blogs-svc write, file-movement policy widening), backfill, Storage__UseS3 flip (already true on file-movement stack; blog-media rides infra #60), bind drop at phase 2.
spikerj commented 2026-07-18 18:00:08 +00:00
Author
Owner
Copy Link
Copy Source

Epic #413 accuracy pass (2026-07-18) — closing: migration scope complete; the one real blocker is #613, which gates #529 phase 2.

Verified against current masters:

  • Code merged: backend #257 — both quarantine writers dual-write behind Storage:UseS3 (QuarantineMirror in blog-media-processor + file-movement's root-tree route, AddKeyedS3ObjectStore two-bucket seam) — on backend master
  • quarantine bucket created in the 2026-07-14 phase 0 (it was missing until then); empty-as-disk
  • Stacks dual-run (file-movement and blog-media-processor both carry Storage__UseS3=true)

The live blocker is #613: file-movement's metadata-svc key gets Access Denied writing quarantine (Seq evidence 2026-07-16). That's a MinIO policy fix, precisely scoped in #613, and it must land before the #529 phase-2 bind drop — otherwise quarantine writes have nowhere to go. EICAR-upload verification after the policy fix is #613's/# 529's acceptance line.

Residual tracked in: #613 (key policy) + #529 (bind drop).

**Epic #413 accuracy pass (2026-07-18) — closing: migration scope complete; the one real blocker is #613, which gates #529 phase 2.** Verified against current masters: - Code merged: backend #257 — both quarantine writers dual-write behind `Storage:UseS3` (`QuarantineMirror` in blog-media-processor + file-movement's root-tree route, `AddKeyedS3ObjectStore` two-bucket seam) — on backend master - `quarantine` bucket created in the 2026-07-14 phase 0 (it was missing until then); empty-as-disk - Stacks dual-run (`file-movement` and `blog-media-processor` both carry `Storage__UseS3=true`) The live blocker is **#613**: file-movement's `metadata-svc` key gets `Access Denied` writing `quarantine` (Seq evidence 2026-07-16). That's a MinIO policy fix, precisely scoped in #613, and it must land before the #529 phase-2 bind drop — otherwise quarantine writes have nowhere to go. EICAR-upload verification after the policy fix is #613's/# 529's acceptance line. Residual tracked in: #613 (key policy) + #529 (bind drop).
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agentic
Actionable surfaced by the agentic architecture & security review
 bug
Something is not working
 duplicate
This issue or pull request already exists
 enhancement
New feature
 help wanted
Need some help
 invalid
Something is wrong
 question
More information is needed
 sonarqube
Code quality issue surfaced by SonarQube (bug / code smell)
 wontfix
This won't be fixed
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
spikerj (Joseph Spiker)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: spikerj/spikersoft-issues#526