[Backend][Search] API book search is a permanent no-op — implement SearchBooksRemoteQuery RPC to the embeddings worker so the live Redis vector index is reachable from the edge #568

New Issue

2026-07-14T14:28:44Z

spikerj commented

2026-07-14 14:28:44 +00:00

QA Team — filed 2026-07-14 from the README audit, at spikerj's direction (decision: yes, ticket the RPC path). The audit found the code comment promising this as "a follow-up phase" with no issue number — this is that issue.

Current state (verified):

SpikerSoft.Api registers NoOpVectorSearchService — every vector/book search request from the edge silently returns nothing.
The real index is live and healthy: Redis 8.8's built-in Query Engine serves book_chunks_idx — confirmed on the production cluster (FT._LIST returns it, 2026-07-14). RedisVectorSearchService (in SpikerSoft.Business.Ai.Workers) is the working implementation, but it is only reachable from GPU worker hosts because it drags LLamaSharp/CUDA natives (embedding the query requires the nomic-embed GGUF) — which is exactly why the API can't call it directly.
Net effect: books are embedded, chunks are indexed, quiz-gen can use them as RAG context — but no user-facing search can reach any of it.

Ask: implement the planned SearchBooksRemoteQuery as an IRemoteCommand<T> RPC (RabbitMQ direct-reply-to, the same pattern as RemoteTwilioVerifyService/#260 and the lesson-regrade client) from the API to the embeddings worker, which owns an embedder and can query the index. Then swap NoOpVectorSearchService for the RPC client in the API's DI.

Watch out for: #565 (the in-memory fallback in RedisVectorSearchService caps its candidate pool before scoring — fix that first or the RPC will inherit a broken degraded path).

**QA Team** — filed 2026-07-14 from the README audit, at spikerj's direction (decision: yes, ticket the RPC path). The audit found the code comment promising this as "a follow-up phase" with no issue number — this is that issue. **Current state (verified):** - `SpikerSoft.Api` registers **`NoOpVectorSearchService`** — every vector/book search request from the edge silently returns nothing. - The real index is **live and healthy**: Redis 8.8's built-in Query Engine serves `book_chunks_idx` — confirmed on the production cluster (`FT._LIST` returns it, 2026-07-14). `RedisVectorSearchService` (in SpikerSoft.Business.Ai.Workers) is the working implementation, but it is only reachable from GPU worker hosts because it drags LLamaSharp/CUDA natives (embedding the query requires the nomic-embed GGUF) — which is exactly why the API can't call it directly. - Net effect: books are embedded, chunks are indexed, quiz-gen can use them as RAG context — but **no user-facing search can reach any of it**. **Ask:** implement the planned `SearchBooksRemoteQuery` as an `IRemoteCommand<T>` RPC (RabbitMQ direct-reply-to, the same pattern as `RemoteTwilioVerifyService`/#260 and the lesson-regrade client) from the API to the **embeddings worker**, which owns an embedder and can query the index. Then swap `NoOpVectorSearchService` for the RPC client in the API's DI. **Watch out for:** #565 (the in-memory fallback in RedisVectorSearchService caps its candidate pool before scoring — fix that first or the RPC will inherit a broken degraded path).

spikerj referenced this issue

2026-07-14 20:16:58 +00:00

[Bug][Prod][AI] Book embeddings are WRITE-ONLY — nothing in production ever reads them; RedisVectorSearchService is registered nowhere #585

spikerj referenced this issue

2026-07-14 20:19:07 +00:00

[Bug][Prod][AI] Book embeddings are WRITE-ONLY — nothing in production ever reads them; RedisVectorSearchService is registered nowhere #585

spikerj referenced this issue

2026-07-14 21:12:06 +00:00

[Bug][Backend][Books] Chapter extraction is orphaned end-to-end — nothing ever populates Book.Chapters, and GET /books/{id}/chapters returns [] for every book, always #593

spikerj commented

2026-07-14 21:47:28 +00:00

Implemented in spikersoft-backend PR #292 (open, awaiting merge).

SearchBooksRemoteQuery : IRemoteCommand<BookChunkSearchResponse> → direct-reply-to → the embeddings worker, NoOpVectorSearchService swapped for the RPC client. Exactly the shape you asked for.

Your "watch out for" was already handled: #565 is merged, so the RPC does not inherit the broken degraded path.

Two things the ticket couldn't have known, and they're the interesting part

1. The obvious way to give the worker an embedder re-creates #553.

RedisVectorSearchService loaded its own nomic-embed with a hardcoded GpuLayerCount = 99, no gpu-coordinator lease, no unload-on-idle. The ticket says the worker "owns an embedder and can query the index" — true, but the search service insisted on loading a second one. Registering it in the embeddings worker would have put an undeclared model on the very card the worker already holds a lease for. That is a tenant taking VRAM it never booked — #553, re-created by the natural wiring.

So the embed step is now injected (IQueryEmbedder). The worker supplies a LeasedQueryEmbedder that goes through the model it already has loaded under lease. Business.Ai.Workers keeps the self-loading one, unchanged, for GPU hosts doing RAG.

2. nomic-embed is asymmetric, and getting it wrong fails silently.

Chunks are indexed as search_document: …. A query must be embedded as search_query: …. The worker's GenerateEmbedding hardcodes the document prefix. Calling it for a query doesn't throw — it returns a vector from the wrong part of the space, and search quality just quietly drops. Added GenerateQueryEmbedding on the same leased model, and a test that fails if the embedder ever reaches for the document method. Without that test, the obvious implementation compiles, runs, returns vectors, and is wrong.

Structural note

With the model gone, the search service is pure managed Redis code, so it moved to the light SpikerSoft.Business.Ai. That's what lets the embeddings worker use it without dragging Aspose.PDF and the SpikerSoft.Business monolith into its image — which referencing Business.Ai.Workers would have done.

Also deleted NoOpVectorSearchService outright. Now that it's unregistered, an IVectorSearchService that silently returns empty is a landmine one DI line from killing search again — same dead-but-armed shape as #573.

Verification

11 new tests. Build clean; Business 7517, API 1288, Common 609, Embeddings 60 — 0 failed. No infra change needed: the embeddings worker is already on both the rabbitmq and redis overlays.

Implemented in spikersoft-backend PR #292 (open, awaiting merge). `SearchBooksRemoteQuery : IRemoteCommand<BookChunkSearchResponse>` → direct-reply-to → the embeddings worker, `NoOpVectorSearchService` swapped for the RPC client. Exactly the shape you asked for. Your "watch out for" was already handled: **#565 is merged**, so the RPC does not inherit the broken degraded path. ## Two things the ticket couldn't have known, and they're the interesting part **1. The obvious way to give the worker an embedder re-creates #553.** `RedisVectorSearchService` loaded **its own** nomic-embed with a hardcoded `GpuLayerCount = 99`, **no gpu-coordinator lease, no unload-on-idle**. The ticket says the worker "owns an embedder and can query the index" — true, but the *search service* insisted on loading a second one. Registering it in the embeddings worker would have put an undeclared model on the very card the worker already holds a lease for. That is a tenant taking VRAM it never booked — #553, re-created by the natural wiring. So the embed step is now injected (`IQueryEmbedder`). The worker supplies a `LeasedQueryEmbedder` that goes through the model it **already has loaded under lease**. `Business.Ai.Workers` keeps the self-loading one, unchanged, for GPU hosts doing RAG. **2. nomic-embed is asymmetric, and getting it wrong fails silently.** Chunks are indexed as `search_document: …`. A query must be embedded as `search_query: …`. The worker's `GenerateEmbedding` **hardcodes the document prefix**. Calling it for a query doesn't throw — it returns a vector from the wrong part of the space, and search quality just quietly drops. Added `GenerateQueryEmbedding` on the same leased model, and a test that fails if the embedder ever reaches for the document method. Without that test, the obvious implementation compiles, runs, returns vectors, and is wrong. ## Structural note With the model gone, the search service is pure managed Redis code, so it moved to the **light** `SpikerSoft.Business.Ai`. That's what lets the embeddings worker use it without dragging **Aspose.PDF and the `SpikerSoft.Business` monolith** into its image — which referencing `Business.Ai.Workers` would have done. Also deleted `NoOpVectorSearchService` outright. Now that it's unregistered, an `IVectorSearchService` that silently returns empty is a landmine one DI line from killing search again — same dead-but-armed shape as #573. ## Verification 11 new tests. Build clean; Business 7517, API 1288, Common 609, Embeddings 60 — **0 failed**. **No infra change needed**: the embeddings worker is already on both the `rabbitmq` and `redis` overlays.

spikerj referenced this issue

2026-07-14 21:47:47 +00:00

[Bug][Prod][AI] Book embeddings are WRITE-ONLY — nothing in production ever reads them; RedisVectorSearchService is registered nowhere #585

spikerj commented

2026-07-14 21:55:36 +00:00

Merged to master in spikersoft-backend PR #292.

Book search now goes from the API to the embeddings worker over direct-reply-to RPC (SearchBooksRemoteQuery), NoOpVectorSearchService is deleted, and the query is embedded on the worker's already-leased model using the correct nomic search_query: prefix.

The vector index the embeddings worker has been filling all along is finally readable from the edge. No infra change was needed — the worker was already on both the rabbitmq and redis overlays.

Closing.

Merged to `master` in spikersoft-backend PR #292. Book search now goes from the API to the embeddings worker over direct-reply-to RPC (`SearchBooksRemoteQuery`), `NoOpVectorSearchService` is deleted, and the query is embedded on the worker's **already-leased** model using the correct nomic `search_query:` prefix. The vector index the embeddings worker has been filling all along is finally readable from the edge. No infra change was needed — the worker was already on both the `rabbitmq` and `redis` overlays. Closing.

spikerj closed this issue

2026-07-14 21:55:39 +00:00

spikerj referenced this issue

2026-07-14 22:48:54 +00:00

[Bug][Prod][AI] Book embeddings are WRITE-ONLY — nothing in production ever reads them; RedisVectorSearchService is registered nowhere #585

spikerj referenced this issue

2026-07-24 18:58:41 +00:00

Epic: Sponsor page — unified card grid, declutter donate buttons, and family shared-trip funds #827

spikerj referenced this issue

2026-07-24 18:58:41 +00:00

Angular master e2e-anonymous RED since #827 PR #564: /sponsor/donate-family/:familyKey unclassified in route-manifest → publish skipped, frontend not deploying #828

spikerj referenced this issue