Stacker download analytics: per-download store, admin/finance dashboards, statusboard page #1206

Open
opened 2026-10-13 05:19:39 +00:00 by spikerj · 2 comments
Owner

We know how many Stacker licenses we've sold but not how many copies were downloaded. Today the only trace is the StackerDownload Seq log line per HTTP response (DownloadMeter.cs): no IP, geo, user agent, referrer, user or website-vs-updater split, no durable store, nothing on any dashboard.

Plan

  • Backend (Domain/ReleaseFeeds): DownloadMeter captures IP, UA, referrer, src/utm_*, optional download ticket / bearer user, then queues it. A hosted writer adds GeoIP and a visitor hash and stores it in release-downloads (kept) plus release-download-addresses (raw IP, 90-day TTL). Admin endpoints: api/stacker/downloads/admin/{analytics,recent}, a stackerDownloads section on api/admin/dashboard, and a Seq backfill (counts only, since 2026-09-11).
  • Stacker app: the updater sends User-Agent: SpikerSoft-Stacker/<ver> (<os>; <arch>) and ?src=updater.
  • Angular: /stacker links carry src/utm/ticket; a staff-dashboard tile; an /admin/stacker-downloads page with geo map and live table; download tiles on the /finance/sales Stacker tab; privacy copy update.
  • Statusboard: a new Downloads page (KPIs, per-version and per-country bars, live feed).

Decisions: raw IP kept 90 days; hashed visitor + geo kept; statusboard stays on the TX2; counts-only backfill from Seq.

We know how many Stacker licenses we've sold but not how many copies were downloaded. Today the only trace is the `StackerDownload` Seq log line per HTTP response (`DownloadMeter.cs`): no IP, geo, user agent, referrer, user or website-vs-updater split, no durable store, nothing on any dashboard. ## Plan - **Backend** (`Domain/ReleaseFeeds`): `DownloadMeter` captures IP, UA, referrer, `src`/`utm_*`, optional download ticket / bearer user, then queues it. A hosted writer adds GeoIP and a visitor hash and stores it in `release-downloads` (kept) plus `release-download-addresses` (raw IP, 90-day TTL). Admin endpoints: `api/stacker/downloads/admin/{analytics,recent}`, a `stackerDownloads` section on `api/admin/dashboard`, and a Seq backfill (counts only, since 2026-09-11). - **Stacker app**: the updater sends `User-Agent: SpikerSoft-Stacker/<ver> (<os>; <arch>)` and `?src=updater`. - **Angular**: `/stacker` links carry `src`/utm/ticket; a staff-dashboard tile; an `/admin/stacker-downloads` page with geo map and live table; download tiles on the `/finance/sales` Stacker tab; privacy copy update. - **Statusboard**: a new Downloads page (KPIs, per-version and per-country bars, live feed). Decisions: raw IP kept 90 days; hashed visitor + geo kept; statusboard stays on the TX2; counts-only backfill from Seq.
Author
Owner

Update: the TX2 → Xavier move changes one decision here (spikerj/spikersoft-infrastructure#621, Phase 1 of the GlusterFS retirement, 2026-10-13).

  • "Statusboard stays on the TX2" no longer holds. The wall display (statusboard tui on tty1) now runs on xavier-2; the monitor moved from dreamstream2 and has been live since ~05:50Z. The TX2s are being drained and retired with GlusterFS.
    • The collector is unchanged: it still runs on the x86 manager (statusboard stack) on :9300.
    • The security note in spikerj/spikersoft-infrastructure#631 still applies: the snapshot carries download IPs, cities and ISPs, so it stays LAN-only.
  • Deploying the Downloads page (0.2.0): after spikerj/spikersoft-infrastructure#631 merges and the collector is deployed, re-run the display installer on xavier-2 so the TUI binary is 0.2.0:
    cd /mnt/infrastructure && sudo git pull && sudo bash statusboard/host/install-display.sh
  • Other moves the backend/Angular halves touch, none needing code changes:
    • Seq is on xavier-6. Same URL, with history before and after the move intact, so the StackerDownload backfill (--until = deploy time) reads the same store.
    • Keycloak is on xavier-7. The same realm, clients and roles were copied from its database, so the spikersoft-statusboard client-credentials login and the staff/finance policies behave the same.
    • RabbitMQ is on xavier-5, and OpenBao on xavier-1/3/4. Analytics:VisitorHashKey and the ticket key read from Bao as before.
    • CI deploys now run from xavier-1's local /mnt/infrastructure. Workflows are unchanged.
    • MongoDB stays on SERVER, so release-downloads and the TTL collection are unaffected.
    • Redis moves next, one node at a time with its data, and the cluster stays up. The dashboard aggregate cache will not be flushed.
**Update: the TX2 → Xavier move changes one decision here** (spikerj/spikersoft-infrastructure#621, Phase 1 of the GlusterFS retirement, 2026-10-13). - **"Statusboard stays on the TX2" no longer holds.** The wall display (`statusboard tui` on tty1) now runs on **xavier-2**; the monitor moved from dreamstream2 and has been live since ~05:50Z. The TX2s are being drained and retired with GlusterFS. - The collector is unchanged: it still runs on the x86 manager (`statusboard` stack) on `:9300`. - The security note in spikerj/spikersoft-infrastructure#631 still applies: the snapshot carries download IPs, cities and ISPs, so it stays LAN-only. - **Deploying the Downloads page (0.2.0):** after spikerj/spikersoft-infrastructure#631 merges and the collector is deployed, re-run the display installer on xavier-2 so the TUI binary is 0.2.0: `cd /mnt/infrastructure && sudo git pull && sudo bash statusboard/host/install-display.sh` - **Other moves the backend/Angular halves touch, none needing code changes:** - **Seq** is on xavier-6. Same URL, with history before and after the move intact, so the `StackerDownload` backfill (`--until` = deploy time) reads the same store. - **Keycloak** is on xavier-7. The same realm, clients and roles were copied from its database, so the `spikersoft-statusboard` client-credentials login and the staff/finance policies behave the same. - **RabbitMQ** is on xavier-5, and **OpenBao** on xavier-1/3/4. `Analytics:VisitorHashKey` and the ticket key read from Bao as before. - **CI deploys** now run from xavier-1's local `/mnt/infrastructure`. Workflows are unchanged. - **MongoDB** stays on SERVER, so `release-downloads` and the TTL collection are unaffected. - **Redis** moves next, one node at a time with its data, and the cluster stays up. The dashboard aggregate cache will not be flushed.
Author
Owner

Audit 2026-10-14. All four halves are merged and live:

part PR verified
Backend store + endpoints spikersoft-backend#1388 deployed; endpoints answer 401 anonymously; rows being written since 2026-10-13T16:55Z
Stacker updater UA / src=updater spikersoft-stacker#975 in v2026.10.13f+; updater shows as a source in live data → closed spikersoft-stacker#976
Angular page, tile, finance, tagged links spikersoft-angular#1587 in the production bundle → closed spikersoft-angular#1588
Statusboard Downloads page spikersoft-infrastructure#631 collector 0.2.3 serving downloads; wall-display binary on xavier-2 not checked

Left before this epic closes:

  1. Seq backfill not run (no seq-backfill rows; all-time = 7). Window: --since 2026-09-11T00:00:00Z --until 2026-10-13T16:55:59Z. Tracked on spikersoft-backend#1390.
  2. Confirm the xavier-2 wall shows page 8 (spikersoft-infrastructure#634).
**Audit 2026-10-14.** All four halves are merged and live: | part | PR | verified | |---|---|---| | Backend store + endpoints | spikersoft-backend#1388 | deployed; endpoints answer 401 anonymously; rows being written since 2026-10-13T16:55Z | | Stacker updater UA / `src=updater` | spikersoft-stacker#975 | in v2026.10.13f+; `updater` shows as a source in live data → **closed** spikersoft-stacker#976 | | Angular page, tile, finance, tagged links | spikersoft-angular#1587 | in the production bundle → **closed** spikersoft-angular#1588 | | Statusboard Downloads page | spikersoft-infrastructure#631 | collector 0.2.3 serving `downloads`; wall-display binary on xavier-2 not checked | Left before this epic closes: 1. **Seq backfill not run** (no `seq-backfill` rows; all-time = 7). Window: `--since 2026-09-11T00:00:00Z --until 2026-10-13T16:55:59Z`. Tracked on spikersoft-backend#1390. 2. Confirm the xavier-2 wall shows page 8 (spikersoft-infrastructure#634).
Sign in to join this conversation.