spikerj/spikersoft-issues
1
0
Fork 0
Code Issues 196 Pull Requests Actions Packages Projects Releases Wiki Activity

[Bug][Backend][MinIO] Cutover blocker: UploadOrchestrator reads BookPageImage.AbsoluteImagePath with raw File.Exists — dropping the /app/ebooks bind silently kills book image descriptions (epic #413, blocks #528/#529) #598

New Issue
Closed
opened 2026-07-14 21:56:28 +00:00 by spikerj · 1 comment
spikerj commented 2026-07-14 21:56:28 +00:00
Owner
Copy Link
Copy Source

Found while verifying #528. Not a live bug today — a landmine that detonates on cutover day, which is exactly when it is most expensive.

The problem

SpikerSoft.Data/Mongos/BookPageImage.cs:48 persists an absolute filesystem path into Mongo:

AbsoluteImagePath = "/app/ebooks/books/9781234567890/images/page-001-image-00.png"

Its consumers read it straight off the filesystem, bypassing IObjectStore entirely:

  • SpikerSoft.EventHandlers.UploadCoordinator/Services/UploadOrchestrator.cs:891 — if (File.Exists(image.AbsoluteImagePath))
  • UploadOrchestrator.cs:814 — passes it as imagePath into the image-description pipeline
  • UploadOrchestrator.cs:749 — error message

Writers: MetadataExtractor/Services/MetadataExtractionService.cs:499 and :882.

Why it matters

Epic #413's goal is to remove the /mnt/fusionio/spikersoft/ebooks:/app/ebooks bind. The moment that bind is gone:

  • File.Exists("/app/ebooks/...") returns false, not an error.
  • Book page images get marked missing.
  • Image descriptions never generate.
  • Nothing throws. Nothing logs an error. It just quietly stops working.

This is the same silent-degradation class as #553 and #557: the failure mode is a false negative, not an exception, so it will not show up as a red service — only as books that mysteriously have no image descriptions.

The #528 media-serving middleware does not cover this. That middleware maps URL prefixes (/ebooks/...) onto bucket keys; AbsoluteImagePath is a worker-side filesystem read on a completely different code path.

Fix

Route the AbsoluteImagePath consumers through IObjectStore, mapping path→key with the shared ObjectKeys.TryMapKey(path, pathPrefix: "/app/ebooks") shim (#533) — the same trick every other migrated service already uses. The ebooks bucket + IObjectStore are already wired into FileMovement/MetadataExtractor, so this is plumbing, not new infrastructure.

Alternatively, stop persisting absolute paths at all and store the object key, deriving the local path only where a local file is genuinely required.

Acceptance

  • No File.Exists / File.Open against /app/ebooks anywhere in UploadCoordinator.
  • xunit coverage proving image-description dispatch still resolves images when the bind is absent and the object is only in the bucket.

Blocks: the ebooks-bind removal in #528 / the #529 coordinated cutover. Should be fixed before Phase 2, not discovered during it.

Found while verifying #528. **Not a live bug today — a landmine that detonates on cutover day**, which is exactly when it is most expensive. ## The problem `SpikerSoft.Data/Mongos/BookPageImage.cs:48` persists an **absolute filesystem path** into Mongo: ``` AbsoluteImagePath = "/app/ebooks/books/9781234567890/images/page-001-image-00.png" ``` Its consumers read it **straight off the filesystem**, bypassing `IObjectStore` entirely: - `SpikerSoft.EventHandlers.UploadCoordinator/Services/UploadOrchestrator.cs:891` — `if (File.Exists(image.AbsoluteImagePath))` - `UploadOrchestrator.cs:814` — passes it as `imagePath` into the image-description pipeline - `UploadOrchestrator.cs:749` — error message Writers: `MetadataExtractor/Services/MetadataExtractionService.cs:499` and `:882`. ## Why it matters Epic #413's goal is to remove the `/mnt/fusionio/spikersoft/ebooks:/app/ebooks` bind. The moment that bind is gone: - `File.Exists("/app/ebooks/...")` returns **false**, not an error. - Book page images get marked **missing**. - Image descriptions **never generate**. - Nothing throws. Nothing logs an error. It just quietly stops working. This is the same silent-degradation class as #553 and #557: the failure mode is a false negative, not an exception, so it will not show up as a red service — only as books that mysteriously have no image descriptions. The #528 media-serving middleware does **not** cover this. That middleware maps *URL prefixes* (`/ebooks/...`) onto bucket keys; `AbsoluteImagePath` is a *worker-side filesystem read* on a completely different code path. ## Fix Route the `AbsoluteImagePath` consumers through `IObjectStore`, mapping path→key with the shared `ObjectKeys.TryMapKey(path, pathPrefix: "/app/ebooks")` shim (#533) — the same trick every other migrated service already uses. The ebooks bucket + `IObjectStore` are already wired into FileMovement/MetadataExtractor, so this is plumbing, not new infrastructure. Alternatively, stop persisting absolute paths at all and store the object key, deriving the local path only where a local file is genuinely required. ## Acceptance - No `File.Exists` / `File.Open` against `/app/ebooks` anywhere in UploadCoordinator. - xunit coverage proving image-description dispatch still resolves images when the bind is absent and the object is only in the bucket. **Blocks:** the ebooks-bind removal in #528 / the #529 coordinated cutover. Should be fixed *before* Phase 2, not discovered during it.
spikerj commented 2026-07-14 22:39:47 +00:00
Author
Owner
Copy Link
Copy Source

Resolved in spikersoft-backend PR #297 (merged to master as bc55a9ff). UploadOrchestrator no longer answers image existence with a bare File.Exists on /app/ebooks/... — it goes through IBookImageLocator, which is filesystem-first and falls back to the ebooks bucket (the same contract the Python image-description worker already uses on the bind-less 4090 lane). Removing the ebooks bind will no longer fail healthy books as 'corrupt'. 19/19 tests, UnitTests.slnf clean.

Ops prerequisite still required before enabling (Storage__UseS3 already true for other reasons): the uploads-svc key must gain read on ebooks (runbook Phase 0 step 2), or lookups 403 and books fail as corrupt — the exact bug this prevents. Closing the code work; the key-policy step is tracked under #529 Phase 0.

Resolved in spikersoft-backend PR #297 (merged to `master` as bc55a9ff). `UploadOrchestrator` no longer answers image existence with a bare `File.Exists` on `/app/ebooks/...` — it goes through `IBookImageLocator`, which is filesystem-first and falls back to the `ebooks` bucket (the same contract the Python image-description worker already uses on the bind-less 4090 lane). Removing the ebooks bind will no longer fail healthy books as 'corrupt'. 19/19 tests, `UnitTests.slnf` clean. **Ops prerequisite still required before enabling** (`Storage__UseS3` already true for other reasons): the `uploads-svc` key must gain **read on `ebooks`** (runbook Phase 0 step 2), or lookups 403 and books fail as corrupt — the exact bug this prevents. Closing the code work; the key-policy step is tracked under #529 Phase 0.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agentic
Actionable surfaced by the agentic architecture & security review
 bug
Something is not working
 duplicate
This issue or pull request already exists
 enhancement
New feature
 help wanted
Need some help
 invalid
Something is wrong
 question
More information is needed
 sonarqube
Code quality issue surfaced by SonarQube (bug / code smell)
 wontfix
This won't be fixed
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
spikerj (Joseph Spiker)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: spikerj/spikersoft-issues#598