[Bug] Redis Cluster healthcheck says "5 healthy, 0 unhealthy node(s)" while cluster_state is fail with 3 disconnected gossip nodes #372

New Issue

2026-07-05T02:59:40Z

spikerj commented

2026-07-05 02:59:40 +00:00

During the 2026-07-05 Redis incident the healthcheck's status line read Redis cluster status: fail — 5 healthy, 0 unhealthy node(s) while node_details showed three Connected: false gossip entries (:0@0 ghosts) and 5461 failed slots. The healthy/unhealthy counters appear to count only configured endpoints that accept a TCP connection, not cluster-membership health — so the summary contradicted the payload and understated the incident.

Fix: derive node counts from CLUSTER NODES connectivity (gossip view) rather than endpoint reachability, or report both explicitly (reachable_endpoints vs cluster_nodes_connected), and surface cluster_slots_fail > 0 in the description prominently. The Unhealthy overall verdict was correct (cluster_state drove it) — this is about the human-readable summary lying. Small change in the API's Redis health check implementation + unit test with a ghost-node CLUSTER NODES fixture.

During the 2026-07-05 Redis incident the healthcheck's status line read `Redis cluster status: fail — 5 healthy, 0 unhealthy node(s)` while `node_details` showed three `Connected: false` gossip entries (`:0@0` ghosts) and 5461 failed slots. The healthy/unhealthy counters appear to count only configured endpoints that accept a TCP connection, not cluster-membership health — so the summary contradicted the payload and understated the incident. **Fix:** derive node counts from `CLUSTER NODES` connectivity (gossip view) rather than endpoint reachability, or report both explicitly (`reachable_endpoints` vs `cluster_nodes_connected`), and surface `cluster_slots_fail > 0` in the description prominently. The `Unhealthy` overall verdict was correct (cluster_state drove it) — this is about the human-readable summary lying. Small change in the API's Redis health check implementation + unit test with a ghost-node CLUSTER NODES fixture.

spikerj added the bug label 2026-07-05 02:59:40 +00:00

spikerj commented

2026-07-05 06:46:12 +00:00

Resolved in spikersoft-backend PR #84 (merged to master). The "Redis Cluster" healthcheck summary now derives from CLUSTER NODES gossip membership, not endpoint TCP-reachability: "fail — 3 master(s)/2 replica(s) connected, 3 gossip node(s) disconnected, 5461/16384 slots failing" instead of the contradictory "5 healthy, 0 unhealthy." New pure BuildClusterSummary seam, ghost-node + healthy fixtures, verdict logic unchanged, all data keys preserved + gossip/slot keys added. 32/32 tests. Closing.

Resolved in spikersoft-backend PR #84 (merged to `master`). The "Redis Cluster" healthcheck summary now derives from `CLUSTER NODES` gossip membership, not endpoint TCP-reachability: `"fail — 3 master(s)/2 replica(s) connected, 3 gossip node(s) disconnected, 5461/16384 slots failing"` instead of the contradictory "5 healthy, 0 unhealthy." New pure `BuildClusterSummary` seam, ghost-node + healthy fixtures, verdict logic unchanged, all data keys preserved + gossip/slot keys added. 32/32 tests. Closing.

spikerj closed this issue

2026-07-05 06:46:13 +00:00

spikerj referenced this issue

2026-07-05 08:27:26 +00:00

Code-execution Python probe should assert a minimum interpreter version (curriculum uses 3.10+ `match`) #379

spikerj referenced this issue

2026-07-17 18:40:09 +00:00

CurriculumPythonInterpreter throws ArgumentNullException on unparseable Python version (CS8604) #673

Sign in to join this conversation.