spikerj/spikersoft-issues
1
0
Fork 0
Code Issues 196 Pull Requests Actions Packages Projects Releases Wiki Activity

[Infra][MinIO] Profile/team/sponsor images: GridFS → MinIO sole store (profile-images slice of epic #413) #771

New Issue
Closed
opened 2026-07-21 16:40:07 +00:00 by spikerj · 3 comments
spikerj commented 2026-07-21 16:40:07 +00:00
Owner
Copy Link
Copy Source

Slice of epic #413, prompted by #770 review: profile pictures (avatars, profile image galleries, team photos, sponsor images — everything behind IGridFsImageService) still live in Mongo GridFS (profile-images GridFS bucket). Per the epic, all binary media belongs in MinIO.

Plan (app is not live — hard cutover, no dual-read):

  1. Replace GridFsImageService with a MinIO/S3-backed implementation of the same seam via the shared SpikerSoft.Storage registration (#533), bucket profile-images, in-cluster endpoint (no hairpin). Same ids become object keys, so no Mongo document changes.
  2. One-shot migration tool: stream every GridFS file to the bucket (key = GridFS id, content-type + filename preserved), read-back verify (length + hash), idempotent.
  3. MinIO bucket + scoped svc-user provisioning (extend spikersoft-infrastructure openbao/minio scripts), creds via OpenBao secret/services/api.
  4. Fix the #770 avatar fallback correctly: ProfileImageId is a GUID referencing Images[]; the storage key is that entry's GridFsFileId (the #441 draft coalesce would have thrown on ObjectId.Parse).
  5. Tests: migrate existing IGridFsImageService mocks to the new seam, add coverage for the new storage service + avatar-key resolution; whole unit suite green; drop tests only where genuinely obsolete.
  6. After deploy + verification: drop the GridFS profile-images.files/.chunks collections.
Slice of epic #413, prompted by #770 review: profile pictures (avatars, profile image galleries, team photos, sponsor images — everything behind `IGridFsImageService`) still live in Mongo GridFS (`profile-images` GridFS bucket). Per the epic, all binary media belongs in MinIO. Plan (app is not live — hard cutover, no dual-read): 1. Replace `GridFsImageService` with a MinIO/S3-backed implementation of the same seam via the shared `SpikerSoft.Storage` registration (#533), bucket `profile-images`, in-cluster endpoint (no hairpin). Same ids become object keys, so no Mongo document changes. 2. One-shot migration tool: stream every GridFS file to the bucket (key = GridFS id, content-type + filename preserved), read-back verify (length + hash), idempotent. 3. MinIO bucket + scoped svc-user provisioning (extend spikersoft-infrastructure openbao/minio scripts), creds via OpenBao `secret/services/api`. 4. Fix the #770 avatar fallback correctly: `ProfileImageId` is a GUID referencing `Images[]`; the storage key is that entry's `GridFsFileId` (the #441 draft coalesce would have thrown on ObjectId.Parse). 5. Tests: migrate existing `IGridFsImageService` mocks to the new seam, add coverage for the new storage service + avatar-key resolution; whole unit suite green; drop tests only where genuinely obsolete. 6. After deploy + verification: drop the GridFS `profile-images.files/.chunks` collections.
spikerj commented 2026-07-21 16:54:40 +00:00
Author
Owner
Copy Link
Copy Source

Implementation up: spikersoft-backend PR #442 + spikersoft-infrastructure PR #134. Data migration ALREADY EXECUTED against prod: bucket profile-images created, api-svc granted rw (verified read/write/delete with the API's own creds), 3/3 GridFS files copied with SHA-256 read-back verification, idempotency proven on re-run. Remaining after #442 merges+deploys: verify avatars render from MinIO, then drop the GridFS profile-images.files/.chunks collections. Note two further GridFS slices remain for this epic (not in scope here): IGridFsVaultFileService (info-vault files) and the photograph gallery preview store (#718).

Implementation up: spikersoft-backend PR #442 + spikersoft-infrastructure PR #134. Data migration ALREADY EXECUTED against prod: bucket profile-images created, api-svc granted rw (verified read/write/delete with the API's own creds), 3/3 GridFS files copied with SHA-256 read-back verification, idempotency proven on re-run. Remaining after #442 merges+deploys: verify avatars render from MinIO, then drop the GridFS profile-images.files/.chunks collections. Note two further GridFS slices remain for this epic (not in scope here): IGridFsVaultFileService (info-vault files) and the photograph gallery preview store (#718).
spikerj commented 2026-07-21 18:41:30 +00:00
Author
Owner
Copy Link
Copy Source

Deploy verified: avatars now serve from MinIO in prod (both test users 200, byte-exact vs migrated objects). GridFS profile-images.files/.chunks archived to ~/backups/gridfs-profile-images-20260721 on the 4090 (mongodump, 3 files + 10 chunks). ONLY remaining step: drop the two GridFS collections — held for explicit operator go-ahead (destructive).

Deploy verified: avatars now serve from MinIO in prod (both test users 200, byte-exact vs migrated objects). GridFS profile-images.files/.chunks archived to ~/backups/gridfs-profile-images-20260721 on the 4090 (mongodump, 3 files + 10 chunks). ONLY remaining step: drop the two GridFS collections — held for explicit operator go-ahead (destructive).
spikerj commented 2026-07-21 18:46:33 +00:00
Author
Owner
Copy Link
Copy Source

GridFS profile-images.files/.chunks dropped (operator-approved; 0 profile-images collections remain). Avatar serving re-verified live post-drop (200, byte-exact from MinIO). mongodump archive retained at ~/backups/gridfs-profile-images-20260721 on the 4090. MinIO is now the SOLE store for profile/team/sponsor images — slice complete. Remaining GridFS slices for epic #413 noted earlier: info-vault files (IGridFsVaultFileService) + photograph preview store (#718). Closing.

GridFS profile-images.files/.chunks dropped (operator-approved; 0 profile-images collections remain). Avatar serving re-verified live post-drop (200, byte-exact from MinIO). mongodump archive retained at ~/backups/gridfs-profile-images-20260721 on the 4090. MinIO is now the SOLE store for profile/team/sponsor images — slice complete. Remaining GridFS slices for epic #413 noted earlier: info-vault files (IGridFsVaultFileService) + photograph preview store (#718). Closing.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agentic
Actionable surfaced by the agentic architecture & security review
 bug
Something is not working
 duplicate
This issue or pull request already exists
 enhancement
New feature
 help wanted
Need some help
 invalid
Something is wrong
 question
More information is needed
 sonarqube
Code quality issue surfaced by SonarQube (bug / code smell)
 wontfix
This won't be fixed
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
spikerj (Joseph Spiker)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: spikerj/spikersoft-issues#771