[Feature][Calendar] Subscription feeds — periodic .ics refresh via scheduler worker #803

New Issue

2026-07-22T22:47:08Z

spikerj commented

2026-07-22 22:47:08 +00:00

Part of the unified scheduling epic #781. Builds on the one-shot import in #802 (shares the parser + import service).

One-shot upload (#802) is a snapshot. This ticket adds the periodic half: a user pastes a secret webcal/.ics feed URL (Outlook, Google, and iCloud can all publish one) and we re-poll it on a schedule, keeping their imported calendar fresh — no OAuth, read-only.

Delivery vehicle: the SpikerSoft.EventHandlers.Scheduler worker, which is now actually deployed and running in prod (verified #540). A new RefreshCalendarSubscriptions task runner sits alongside GeoIpUpdate.

Scope

Entity: CalendarSubscription (calendar-subscriptions collection): CreatedBy, FeedUrl, Source, Enabled, ETag?, LastModified?, LastPolledUtc?, LastStatus, last import counts.
API: POST/GET/DELETE api/calendar/subscriptions (MediatR). URL validated http(s) + length.
Task runner (RefreshCalendarSubscriptionsTaskRunner : ITaskRunner, [SchedulableTask]): iterate enabled subscriptions → SSRF-guarded conditional GET (If-None-Match/If-Modified-Since, reusing the SsrfEgressPolicy the HttpCallback runner already applies — fetching arbitrary user-supplied URLs must not reach internal hosts) → on 200, parse via #802's IcsImportParser → CalendarImportService.UpsertByUidAsync(sub.CreatedBy, "ics-subscription:{id}", …); on 304, skip. Persist ETag/LastModified/counts back to the subscription.
A single recurring RefreshCalendarSubscriptions ScheduledTask drives the cadence (created via the existing ScheduledTasks admin path); documented in the PR.

Known follow-ups (out of scope)

Worker-path Redis cache invalidation (worker writes straight to Mongo; API domain cache is TTL-bounded) — noted for a fast-follow.
Deletion reconciliation (a UID that vanishes from the feed) — v1 leaves imported events in place.

Tests (tight)

Task runner: 304→skip (no writes), 200→parse+upsert with counts persisted, SSRF-blocked host→failure (no import), disabled subscription skipped; [SchedulableTask] registration test; subscription command/query handler tests (create with URL validation / list-mine / delete-mine ownership).

Ships together with #802 in one WIP PR.

Part of the unified scheduling epic #781. Builds on the one-shot import in #802 (shares the parser + import service). One-shot upload (#802) is a snapshot. This ticket adds the **periodic** half: a user pastes a secret `webcal`/`.ics` feed URL (Outlook, Google, and iCloud can all publish one) and we re-poll it on a schedule, keeping their imported calendar fresh — no OAuth, read-only. Delivery vehicle: the `SpikerSoft.EventHandlers.Scheduler` worker, which is now actually deployed and running in prod (verified #540). A new `RefreshCalendarSubscriptions` task runner sits alongside `GeoIpUpdate`. ## Scope - **Entity:** `CalendarSubscription` (`calendar-subscriptions` collection): `CreatedBy`, `FeedUrl`, `Source`, `Enabled`, `ETag?`, `LastModified?`, `LastPolledUtc?`, `LastStatus`, last import counts. - **API:** `POST/GET/DELETE api/calendar/subscriptions` (MediatR). URL validated `http(s)` + length. - **Task runner** (`RefreshCalendarSubscriptionsTaskRunner : ITaskRunner`, `[SchedulableTask]`): iterate enabled subscriptions → **SSRF-guarded conditional GET** (`If-None-Match`/`If-Modified-Since`, reusing the `SsrfEgressPolicy` the HttpCallback runner already applies — fetching arbitrary user-supplied URLs must not reach internal hosts) → on `200`, parse via #802's `IcsImportParser` → `CalendarImportService.UpsertByUidAsync(sub.CreatedBy, "ics-subscription:{id}", …)`; on `304`, skip. Persist ETag/LastModified/counts back to the subscription. - A single recurring `RefreshCalendarSubscriptions` `ScheduledTask` drives the cadence (created via the existing ScheduledTasks admin path); documented in the PR. ## Known follow-ups (out of scope) - Worker-path Redis cache invalidation (worker writes straight to Mongo; API domain cache is TTL-bounded) — noted for a fast-follow. - Deletion reconciliation (a UID that vanishes from the feed) — v1 leaves imported events in place. ## Tests (tight) Task runner: `304`→skip (no writes), `200`→parse+upsert with counts persisted, SSRF-blocked host→failure (no import), disabled subscription skipped; `[SchedulableTask]` registration test; subscription command/query handler tests (create with URL validation / list-mine / delete-mine ownership). **Ships together with #802 in one WIP PR.**

spikerj referenced this issue

2026-07-22 23:14:11 +00:00

[Feature][Calendar] Import .ics calendars — one-shot file upload (Microsoft / Google / Apple) #802

spikerj commented

2026-07-22 23:14:15 +00:00

Implemented in spikersoft-backend PR #456 (WIP, ships with #802).

CalendarSubscription entity + calendar-subscriptions collection; POST/GET/DELETE api/calendar/subscriptions (webcal:// normalised to https://); RefreshCalendarSubscriptionsTaskRunner in the Scheduler worker doing SSRF-guarded conditional GET → shared parser → shared upsert, with per-feed failure isolation. Tests cover 304-skip, 200-import, error isolation, disabled-skip, and [SchedulableTask] registration.

⚠️ Post-deploy gate: the runner is inert until a recurring RefreshCalendarSubscriptions ScheduledTask exists to drive it (same trap as #540 — deployed but never triggered). Create one via the ScheduledTasks admin path after deploy. Open question in the PR: whether to add a startup seed that guarantees the task exists.

Implemented in spikersoft-backend **PR #456** (WIP, ships with #802). `CalendarSubscription` entity + `calendar-subscriptions` collection; `POST/GET/DELETE api/calendar/subscriptions` (webcal:// normalised to https://); `RefreshCalendarSubscriptionsTaskRunner` in the Scheduler worker doing SSRF-guarded conditional GET → shared parser → shared upsert, with per-feed failure isolation. Tests cover 304-skip, 200-import, error isolation, disabled-skip, and `[SchedulableTask]` registration. **⚠️ Post-deploy gate:** the runner is inert until a recurring `RefreshCalendarSubscriptions` `ScheduledTask` exists to drive it (same trap as #540 — deployed but never triggered). Create one via the ScheduledTasks admin path after deploy. Open question in the PR: whether to add a startup seed that guarantees the task exists.

spikerj commented

2026-07-22 23:23:47 +00:00

Update: the post-deploy gate is gone — made it a guarantee instead of a manual step.

CalendarSubscriptionScheduleSeeder (worker startup, hosted service) now ensures the recurring RefreshCalendarSubscriptions task exists and is firing. Because the recurrence self-perpetuates through a durable RabbitMQ delayed message, the seeder is careful never to double-schedule a healthy chain — the pure CalendarSubscriptionSchedulePlan decides: create-if-absent, leave-alone-if-healthy/running, re-arm-if-stalled (self-heals a dead chain), respect operator pause. Tunable via Scheduler:CalendarSubscriptions:{AutoSeed,RefreshIntervalMinutes}; seed failures logged, never fatal.

So deploying the worker is now sufficient — no admin action required. Pushed to PR #456 (Scheduler.Tests now 27 green, incl. the full seeder decision matrix + no-double-schedule-on-restart).

Update: the post-deploy gate is **gone** — made it a guarantee instead of a manual step. `CalendarSubscriptionScheduleSeeder` (worker startup, hosted service) now ensures the recurring `RefreshCalendarSubscriptions` task exists and is firing. Because the recurrence self-perpetuates through a durable RabbitMQ delayed message, the seeder is careful never to double-schedule a healthy chain — the pure `CalendarSubscriptionSchedulePlan` decides: create-if-absent, leave-alone-if-healthy/running, re-arm-if-stalled (self-heals a dead chain), respect operator pause. Tunable via `Scheduler:CalendarSubscriptions:{AutoSeed,RefreshIntervalMinutes}`; seed failures logged, never fatal. So deploying the worker is now sufficient — no admin action required. Pushed to PR #456 (Scheduler.Tests now 27 green, incl. the full seeder decision matrix + no-double-schedule-on-restart).

spikerj commented

2026-07-22 23:34:20 +00:00

Resolved in spikersoft-backend PR #456 (merged to master, ee541921). Subscription feeds shipped: CalendarSubscription entity + POST/GET/DELETE api/calendar/subscriptions (webcal:// normalised), and RefreshCalendarSubscriptionsTaskRunner in the Scheduler worker doing SSRF-guarded conditional GET → shared parser → shared upsert with per-feed failure isolation. Self-arming via CalendarSubscriptionScheduleSeeder — no manual post-deploy step; won't double-schedule a healthy chain and self-heals a stalled one. Documented follow-ups (worker-path cache invalidation, feed-deletion reconciliation) noted above. Closing.

Note: verify after the next Scheduler deploy that a RefreshCalendarSubscriptions task appears (the seeder creates it ~10s after boot) and fires — the code guarantees it, but this is the first real exercise of the seed path.

Resolved in spikersoft-backend PR #456 (merged to `master`, `ee541921`). Subscription feeds shipped: `CalendarSubscription` entity + `POST/GET/DELETE api/calendar/subscriptions` (webcal:// normalised), and `RefreshCalendarSubscriptionsTaskRunner` in the Scheduler worker doing SSRF-guarded conditional GET → shared parser → shared upsert with per-feed failure isolation. Self-arming via `CalendarSubscriptionScheduleSeeder` — no manual post-deploy step; won't double-schedule a healthy chain and self-heals a stalled one. Documented follow-ups (worker-path cache invalidation, feed-deletion reconciliation) noted above. Closing. Note: verify after the next Scheduler deploy that a `RefreshCalendarSubscriptions` task appears (the seeder creates it ~10s after boot) and fires — the code guarantees it, but this is the first real exercise of the seed path.

spikerj closed this issue

2026-07-22 23:34:20 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: spikerj/spikersoft-issues#803