Circuit Breaker

Overview

Circuit Breaker is a resilience control that temporarily stops calls to a failing or overloaded dependency after defined failure conditions are reached. It prevents repeated calls during a fault; retries repeat calls, and the two mechanisms must be coordinated to avoid amplifying outages.

The principal risks include breaking healthy traffic because thresholds are too sensitive, hiding persistent defects behind fallbacks, synchronized recovery probes, stale breaker state, and retry logic bypassing the breaker. Operational review of Circuit Breaker should separate successful recovery from repeated attempts that only increase latency or load.

A production implementation should define failure thresholds, open duration, half-open probes, operation-specific policies, safe fallbacks, state visibility, and isolation between providers or tenants. Metrics for Circuit Breaker should separate first-attempt success, recovered operations, exhausted attempts, duplicate prevention, and added latency.

Useful measures include breaker openings, rejected calls, half-open success rate, dependency recovery time, fallback usage, and customer-impact duration. Circuit Breaker is closely connected to Automatic Retry, Backpressure, and RPC Failover. A Circuit Breaker decision should use documented method semantics and provider error classes rather than message text alone.

Manual intervention for Circuit Breaker should begin only after automated status checks cannot determine a safe final outcome. Configuration changes to Circuit Breaker should be versioned and tested under burst, timeout, dependency failure, and restart conditions.

The Circuit Breaker policy should define which failures are retryable, how long the decision remains valid, and how duplicate effects are prevented. For Circuit Breaker, a missing response should remain uncertain until status lookup or reconciliation confirms whether the original operation executed.