[Observability][Backend] Client-abort exceptions (OperationCanceled / 'client reset request stream') logged as Error + 500 — noise in Seq/traces #485

New Issue

2026-07-12T00:23:02Z

spikerj commented

2026-07-12 00:23:02 +00:00

Summary

Client-driven request aborts/cancellations are logged at Error level and returned as HTTP 500 (and marked as errors on the Jaeger span). These are not server faults — the client disconnected or canceled mid-request — but they pollute the Error stream in Seq (the same stream used for alerting/triage) and inflate the API's error/500 metrics. Low functional impact, real observability impact.

Evidence (Seq, last 6h)

SpikerSoft.Business.Behaviors.TracingBehavior — Query failed: GetLinkCommentsQuery (4 events), exception:

System.OperationCanceledException: The operation was canceled.
  at MongoDB.EntityFrameworkCore.Query.QueryingEnumerable... MoveNextAsync()
  at ...ToListAsync[TSource](IQueryable, CancellationToken)

SpikerSoft.Api.Middleware.GlobalExceptionHandlerMiddleware — Unhandled exception ... POST /api/authentication/check-email (UserId: anonymous), exception:
```
System.IO.IOException: The client reset the request stream.
  at ...HttpRequestPipeReader.ReadAsync(...)   (request body read aborted)
```

Both are the caller going away — OperationCanceledException tied to HttpContext.RequestAborted, and the Kestrel "client reset the request stream" IOException.

Root cause (code)

SpikerSoft.Api/Middleware/GlobalExceptionHandlerMiddleware.cs:33 — catch (Exception ex) handles everything the same way: LogError (line ~75), sets Response.StatusCode = 500 (line ~86), and marks the current Activity/span as error (activity.SetStatus(Error), error=true). No special-casing for cancellation/client-abort.
SpikerSoft.Business/Behaviors/TracingBehavior.cs:86 — catch (Exception ex) -> logger.LogError(ex, "{OperationType} failed: ...") (line 101). No cancellation special-casing.

Proposed fix

In both catch blocks, detect client-abort and treat it as non-error:

if (ex is OperationCanceledException or TaskCanceledException
    && context.RequestAborted.IsCancellationRequested)
{
    _logger.LogInformation("Request aborted by client: {Path}", context.Request.Path);
    context.Response.StatusCode = 499;      // client closed request (nginx convention)
    return;                                  // do NOT mark the span as error
}
// also treat IOException "client reset the request stream" (Kestrel BadHttpRequest/connection reset) as client-abort

For TracingBehavior, log OperationCanceledException where the token is canceled at Debug/Information and skip the activity.SetStatus(Error) so canceled requests don't show as failed traces.

Impact of fixing

Cleaner Seq Error stream (fewer false positives during triage/alerting).
Accurate 5xx/error-rate metrics and Jaeger span success rates.

Priority: low (cosmetic/observability; ~6 events/6h currently), but worth doing since we rely on the error stream to find real issues.

Filed proactively by automated health check (Seq centralized error audit).

## Summary Client-driven request **aborts/cancellations are logged at `Error` level and returned as HTTP 500** (and marked as errors on the Jaeger span). These are not server faults — the client disconnected or canceled mid-request — but they pollute the Error stream in Seq (the same stream used for alerting/triage) and inflate the API's error/500 metrics. Low functional impact, real observability impact. ## Evidence (Seq, last 6h) - `SpikerSoft.Business.Behaviors.TracingBehavior` — `Query failed: GetLinkCommentsQuery` (4 events), exception: ``` System.OperationCanceledException: The operation was canceled. at MongoDB.EntityFrameworkCore.Query.QueryingEnumerable... MoveNextAsync() at ...ToListAsync[TSource](IQueryable, CancellationToken) ``` - `SpikerSoft.Api.Middleware.GlobalExceptionHandlerMiddleware` — `Unhandled exception ... POST /api/authentication/check-email` (UserId: anonymous), exception: ``` System.IO.IOException: The client reset the request stream. at ...HttpRequestPipeReader.ReadAsync(...) (request body read aborted) ``` Both are the caller going away — `OperationCanceledException` tied to `HttpContext.RequestAborted`, and the Kestrel "client reset the request stream" IOException. ## Root cause (code) - `SpikerSoft.Api/Middleware/GlobalExceptionHandlerMiddleware.cs:33` — `catch (Exception ex)` handles everything the same way: `LogError` (line ~75), sets `Response.StatusCode = 500` (line ~86), and marks the current Activity/span as error (`activity.SetStatus(Error)`, `error=true`). No special-casing for cancellation/client-abort. - `SpikerSoft.Business/Behaviors/TracingBehavior.cs:86` — `catch (Exception ex)` -> `logger.LogError(ex, "{OperationType} failed: ...")` (line 101). No cancellation special-casing. ## Proposed fix In both catch blocks, detect client-abort and treat it as non-error: ```csharp if (ex is OperationCanceledException or TaskCanceledException && context.RequestAborted.IsCancellationRequested) { _logger.LogInformation("Request aborted by client: {Path}", context.Request.Path); context.Response.StatusCode = 499; // client closed request (nginx convention) return; // do NOT mark the span as error } // also treat IOException "client reset the request stream" (Kestrel BadHttpRequest/connection reset) as client-abort ``` For `TracingBehavior`, log `OperationCanceledException` where the token is canceled at `Debug`/`Information` and skip the `activity.SetStatus(Error)` so canceled requests don't show as failed traces. ## Impact of fixing - Cleaner Seq Error stream (fewer false positives during triage/alerting). - Accurate 5xx/error-rate metrics and Jaeger span success rates. Priority: **low** (cosmetic/observability; ~6 events/6h currently), but worth doing since we rely on the error stream to find real issues. --- _Filed proactively by automated health check (Seq centralized error audit)._

spikerj commented

2026-07-12 00:28:20 +00:00

Resolved in spikersoft-backend PR #211 (merged to master). Client aborts (OperationCanceledException tied to RequestAborted; Kestrel 'client reset the request stream' IOException) now log Information + 499 with a client.abort span tag in GlobalExceptionHandlerMiddleware, and TracingBehavior filters caller-canceled OCEs to Information + mediatr.cancelled without failing the span. The boundary is preserved and tested both ways — cancellations NOT caused by the caller still log Error/500. 6 new xUnit tests; CI auto-deploys the API, so the GetLinkCommentsQuery and check-email noise drops out of the Seq Error stream on the next rollout. Closing.

spikerj closed this issue

2026-07-12 00:28:20 +00:00

spikerj referenced this issue

2026-07-21 00:00:13 +00:00

feature(sponsor): family card member spotlight + globe.gl choropleth dream globe on the family page #752

spikerj referenced this issue

2026-07-26 00:50:31 +00:00

Drop GridFS naming from Art Studio: rename to objectKey, migrate legacy docs, remove the #852 shim #853

spikerj referenced this issue

2026-07-29 15:36:24 +00:00

Drop GridFS naming from Art Studio: rename to objectKey, migrate legacy docs, remove the #852 shim #853

Sign in to join this conversation.