[Playgrounds P5b] Video media pipeline — upload, virus scan, transcode + thumbnail, storage/serving (event-driven) #337
Assignees
spikerj (Joseph Spiker)
Clear assignees
spikerj (Joseph Spiker)
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: spikerj/spikersoft-issues#337
Reference in New Issue
Block a user
Sub-ticket of P5 (#328), epic #323. Depends on #336 (model + endpoints).
What: the upload + processing pipeline for community-submitted lesson videos, reusing the event-driven media pattern from
SpikerSoft.EventHandlers.BlogMediaProcessor(RabbitMQ exchange/queue consumers, workflow-state doc, ClamAV scan, move-to-final + static serving).Scope:
IFileProcessingService(newVideos:StagingPath), accepted extensions + size/length limits documentedlesson.video.lifecycleexchange,lesson.video.receivedrouting key) mirroringBlogMediaEventPublisherLessonVideo.url+thumbnailUrl+durationSeconds/blog-pictures/...→/lesson-videos/...) inWebApplicationExtensionsBlogMediaWorkflowState)Note: the current blog pipeline is local-filesystem + ImageSharp (images only, no transcode). Video adds ffmpeg transcode/thumbnail — confirm ffmpeg availability in the worker image or add it. Storage/serving decision (S3/MinIO vs local) to be made here.
Acceptance: a submitted raw video is scanned, transcoded to a web-safe MP4 with a poster thumbnail, and served from the final store;
LessonVideofields are populated; failures land the workflow in a terminal error state without exposing the video.Architecture decision (storage/serving + transcode) — resolves the open question in the description
Storage: local bind-mount on
SERVER, NOT S3/MinIO. Investigation of the swarm confirms all media is host bind-mounted under/mnt/fusionio/spikersoft/...(the enterprise 3TB Fusion SSD on nodeSERVER) — there is no GlusterFS or object store in any stack (spikersoft-backend/deploy/docker-stack.yml). The API and every media worker (file-movement,upload-coordinator,blog-media-processor,security-scanner,metadata-extractor) are all pinned tonode.hostname == SERVERand mount the same fusionio paths; that co-location is how they share files today. GlusterFS (/mntacross DreamStream1–7) is intentionally out of the media topology. Introducing MinIO/S3 would contradict every existing media path for no benefit here.Design — mirror
SpikerSoft.EventHandlers.BlogMediaProcessor:SpikerSoft.EventHandlers.LessonVideoProcessor, pinned tonode.hostname == SERVER, mounting:/mnt/fusionio/spikersoft/uploads/lesson-videos -> /app/uploads/lesson-videos/mnt/fusionio/spikersoft/lesson-videos -> /app/lesson-videosIFileProcessingService(newVideos:StagingPath = /app/uploads/lesson-videos) and publisheslesson.video.receivedon a newlesson.video.lifecycletopic exchange (mirrorsBlogMediaEventPublisher).LessonVideo.Url/ThumbnailUrl/DurationSecondsand status (staysPendingfor the #338 moderation queue). Workflow-state doc mirrorsBlogMediaWorkflowState.+faststart) + JPG poster — not HLS. Plays via native<video>, which the #338 moderation UI and the #339 playground tab already use. Adequate for short lesson clips; HLS can be revisited if long videos become common.apt-get install -y ffmpeg), invoked viaSystem.Diagnostics.Process(no native bindings)./lesson-videosviaPhysicalFileProviderinWebApplicationExtensions, mirroring/blog-pictures.spikersoft-infrastructure/spikersoft-lesson-video-processor/docker-stack.yml.Net effect on the SERVER-fusion vs Gluster split: because the worker is pinned to
SERVERalongside the API, it reads the same fusionio SSD the API wrote to; the Gluster pool is never involved (correct — it isn't part of the media topology). RabbitMQ is pinned offSERVER, but that's messaging only, not file IO.Status: decision recorded; implementation deferred — proceeding with #339 (playground Videos tab) first, then returning to build this pipeline.
Done — merged.
Delivered against scope:
POST /api/Lessons/{n}/videos/upload+ staging via newLessonVideoStagingService(Videos:StagingPath;.mp4/.webm/.mov/.m4v, 200 MB cap).lesson.video.lifecycleexchange,lesson.video.receivedrouting key) mirroringBlogMediaEventPublisher.SpikerSoft.EventHandlers.LessonVideoProcessor: received → scan (ClamAV) → transcode (ffmpeg H.264/AAC MP4+faststart) → poster frame → move; populatesurl/thumbnailUrl/durationSeconds. A failed scan rejects the video; other failures land the workflow in a terminal error state without exposing the video./lesson-videos/...inWebApplicationExtensions.LessonVideoWorkflowStatedoc + indexes (lesson-video-workflows), mirroringBlogMediaWorkflowState.Storage/serving decision: kept local-filesystem host bind mounts on
SERVER(consistent with the blog pipeline) rather than introducing S3/MinIO. ffmpeg is bundled into the worker runtime image.Follow-up (not blocking): the SPA still submits videos by URL (P5d); wiring the in-browser file-upload flow to
/videos/uploadis a small frontend follow-up if desired.