Add unit test coverage for #287 host remediation + security monitoring services #300

New Issue

2026-06-30T20:46:07Z

spikerj commented

2026-06-30 20:46:07 +00:00

Background

PR #35 (spikerj/spikersoft-backend) shipped SpikerSoft.EventHandlers.NodeAgent, SpikerSoft.EventHandlers.SecurityMonitor, and SpikerSoft.EventHandlers.SystemRemediation — ~38 files including security-sensitive detection logic (SSH brute force, sudo abuse, successful login, kernel anomaly, firewall drop detectors), regex-based log parsing, and remediation rule/execution logic — with zero unit tests. This was found during a general backend test-coverage audit, not a reported bug.

This isn't unique to #287 — none of the EventHandlers.* worker services in this codebase have dedicated tests historically — but #287 added the highest-value, most testable, and most security-sensitive surface to that gap, so it's worth closing deliberately rather than leaving it with the rest.

Done so far (first PR)

LogParse (SSH/firewall IP + username regex extraction) — full coverage including IPv4/IPv6 edge cases.
SudoAbuseDetector, KernelAnomalyDetector — fully pure, no mocking needed.
SshBruteForceDetector, FirewallDropDetector — the two Redis-backed sliding-window threshold detectors, mocked at the IDatabase level (same pattern as the existing RedisRateLimitServiceTests).

42 new tests, 0 regressions (full suite: 9814 passed / 4 pre-existing Python 3.9 environment failures, unrelated).

Remaining work (follow-up, not yet scheped)

SuccessfulLoginDetector, HostChangeDetector (SecurityMonitor) — root-login alert, baseline-vs-new-source logic, listener/user/authorized_keys diffing.
SecurityStateStore itself — the Redis fail-open behavior (catches exceptions, returns safe defaults) is currently untested directly.
SystemRemediation: RemediationRuleEngine, RemediationExecutor — rule-matching and command execution have real blast radius if they misfire; currently fully untested.
NodeAgent: HostHealthProbe, JournalTailService, JournalCursorStore — mostly I/O-bound (journalctl, filesystem) so will need light abstraction/mocking to test meaningfully.
RabbitMQ consumers (SecurityMonitorConsumer, SystemRemediationConsumer, RemediationCommandConsumer) — wiring/dispatch, lower priority than the detection/remediation logic itself.

Not blocking, but worth picking up before or alongside #297 (ops console), since that UI surfaces these alerts/incidents directly.

## Background PR #35 (spikerj/spikersoft-backend) shipped `SpikerSoft.EventHandlers.NodeAgent`, `SpikerSoft.EventHandlers.SecurityMonitor`, and `SpikerSoft.EventHandlers.SystemRemediation` — ~38 files including security-sensitive detection logic (SSH brute force, sudo abuse, successful login, kernel anomaly, firewall drop detectors), regex-based log parsing, and remediation rule/execution logic — with **zero unit tests**. This was found during a general backend test-coverage audit, not a reported bug. This isn't unique to #287 — none of the `EventHandlers.*` worker services in this codebase have dedicated tests historically — but #287 added the highest-value, most testable, and most security-sensitive surface to that gap, so it's worth closing deliberately rather than leaving it with the rest. ## Done so far (first PR) - `LogParse` (SSH/firewall IP + username regex extraction) — full coverage including IPv4/IPv6 edge cases. - `SudoAbuseDetector`, `KernelAnomalyDetector` — fully pure, no mocking needed. - `SshBruteForceDetector`, `FirewallDropDetector` — the two Redis-backed sliding-window threshold detectors, mocked at the `IDatabase` level (same pattern as the existing `RedisRateLimitServiceTests`). 42 new tests, 0 regressions (full suite: 9814 passed / 4 pre-existing Python 3.9 environment failures, unrelated). ## Remaining work (follow-up, not yet scheped) - `SuccessfulLoginDetector`, `HostChangeDetector` (SecurityMonitor) — root-login alert, baseline-vs-new-source logic, listener/user/authorized_keys diffing. - `SecurityStateStore` itself — the Redis fail-open behavior (catches exceptions, returns safe defaults) is currently untested directly. - `SystemRemediation`: `RemediationRuleEngine`, `RemediationExecutor` — rule-matching and command execution have real blast radius if they misfire; currently fully untested. - `NodeAgent`: `HostHealthProbe`, `JournalTailService`, `JournalCursorStore` — mostly I/O-bound (journalctl, filesystem) so will need light abstraction/mocking to test meaningfully. - RabbitMQ consumers (`SecurityMonitorConsumer`, `SystemRemediationConsumer`, `RemediationCommandConsumer`) — wiring/dispatch, lower priority than the detection/remediation logic itself. Not blocking, but worth picking up before or alongside #297 (ops console), since that UI surfaces these alerts/incidents directly.

spikerj commented

2026-07-01 04:01:29 +00:00

Resolved in spikersoft-backend PR #43 (merged to master). Added 72 more unit tests covering SuccessfulLoginDetector, HostChangeDetector, SecurityStateStore (fail-open behavior), RemediationRuleEngine, and RemediationExecutor (whitelist, rate limiting, idempotency, argument validation guard rails). Full suite: 9886 passed, 0 regressions.

Still open: NodeAgent's I/O-bound services (HostHealthProbe, JournalTailService, JournalCursorStore) and the RabbitMQ consumers (SecurityMonitorConsumer, SystemRemediationConsumer, RemediationCommandConsumer) are not yet covered. Leaving this open until those land.

Resolved in spikersoft-backend PR #43 (merged to `master`). Added 72 more unit tests covering `SuccessfulLoginDetector`, `HostChangeDetector`, `SecurityStateStore` (fail-open behavior), `RemediationRuleEngine`, and `RemediationExecutor` (whitelist, rate limiting, idempotency, argument validation guard rails). Full suite: 9886 passed, 0 regressions. Still open: `NodeAgent`'s I/O-bound services (`HostHealthProbe`, `JournalTailService`, `JournalCursorStore`) and the RabbitMQ consumers (`SecurityMonitorConsumer`, `SystemRemediationConsumer`, `RemediationCommandConsumer`) are not yet covered. Leaving this open until those land.

spikerj commented

2026-07-01 07:51:58 +00:00

Resolved in spikersoft-backend PR #44 (merged to master). Added 45 more unit tests covering HostHealthProbe's command-output parsers and JournalTailService's script-building/journald-JSON parsing (both made internal via a new InternalsVisibleTo, no behavior change), plus JournalCursorStore exercised against a real temp file. Full suite: 9964 passed, 0 regressions.

Opened PR #45 for the one remaining item — the RabbitMQ consumer wiring/dispatch logic (SecurityMonitorConsumer, SystemRemediationConsumer, RemediationCommandConsumer) — which will close out #300 entirely once merged.

Resolved in spikersoft-backend PR #44 (merged to `master`). Added 45 more unit tests covering `HostHealthProbe`'s command-output parsers and `JournalTailService`'s script-building/journald-JSON parsing (both made `internal` via a new `InternalsVisibleTo`, no behavior change), plus `JournalCursorStore` exercised against a real temp file. Full suite: 9964 passed, 0 regressions. Opened PR #45 for the one remaining item — the RabbitMQ consumer wiring/dispatch logic (`SecurityMonitorConsumer`, `SystemRemediationConsumer`, `RemediationCommandConsumer`) — which will close out #300 entirely once merged.

spikerj commented

2026-07-01 08:05:51 +00:00

Resolved in spikersoft-backend PR #45 (merged to master). Added the final 45 tests covering the RabbitMQ consumer wiring/dispatch logic (SecurityMonitorConsumer, SystemRemediationConsumer, RemediationCommandConsumer) via the same InternalsVisibleTo-based accessibility pattern used in PR #42/#43/#44. Full suite: 9964+ passed, 0 regressions.

All of #300's originally-scoped host-monitoring/security-remediation test coverage is now in place across PRs #42, #43, #44, and #45. Closing.

Resolved in spikersoft-backend PR #45 (merged to `master`). Added the final 45 tests covering the RabbitMQ consumer wiring/dispatch logic (`SecurityMonitorConsumer`, `SystemRemediationConsumer`, `RemediationCommandConsumer`) via the same `InternalsVisibleTo`-based accessibility pattern used in PR #42/#43/#44. Full suite: 9964+ passed, 0 regressions. All of #300's originally-scoped host-monitoring/security-remediation test coverage is now in place across PRs #42, #43, #44, and #45. Closing.

spikerj closed this issue

2026-07-01 08:06:01 +00:00

spikerj referenced this issue

2026-07-15 16:19:15 +00:00

[Security][CI] OpenBao Phase 1: Gitea Actions secrets → AppRole fetch; rotate the committed runner token (epic #543) #545

spikerj referenced this issue

2026-07-15 18:02:30 +00:00

[Security][CI] OpenBao Phase 1: Gitea Actions secrets → AppRole fetch; rotate the committed runner token (epic #543) #545

Sign in to join this conversation.