Circuit Breakers in Node.js: When Slow Dependencies Become System Failures

The failure that isn't yours

In my article on graceful degradation, the throughline is that a product doesn't have to die just because a piece of it does. This article is about the mechanism that makes that possible when the piece that dies is a dependency you call, a payment gateway, a recommendation service, a third-party API, the search cluster.

Here's the trap. Your service is healthy. Your code is fine. But you call a downstream that just started answering in eight seconds instead of eighty milliseconds. Every request that touches it now waits. Each one parks a connection and holds an event-loop callback while that slow call decides whether to answer. Multiply that by your traffic and you've run out of both. Your latency climbs, your memory climbs, your health check goes red, and the load balancer pulls you out of rotation. You are now "down," and you never had a bug. A sick dependency reached across the wire and made its illness yours.

The counterintuitive part: a slow dependency is more dangerous than a dead one. A dead one fails fast, connection refused, you move on in a millisecond. A slow one holds your resources hostage while it decides whether to answer. The circuit breaker exists to turn slow-and-uncertain into fast-and-decided.

Retries make it worse, not better

The instinct when a call fails is to retry. Under a partial outage, that instinct is gasoline. The dependency is already struggling, and now every caller is hitting it three times instead of once. You've tripled the load on the exact thing that's failing, which guarantees it stays failing. This is a retry storm, and it's how a five-second blip becomes a thirty-minute incident.

To be clear, retries aren't the enemy, they're essential. The problem is unbounded retries into a failing dependency. Bounded by exponential backoff, a hard attempt cap, and a breaker underneath, retries are how you ride out a transient blip. Strip those bounds away and the exact same mechanism becomes the storm.

  WITHOUT A BREAKER
  dep slows down
       │
       ▼
  callers wait 8s each
       │
       ▼
  + retries pile on 3x load
       │
       ▼
  ✗ your service falls over too

  WITH A BREAKER
  dep slows down
       │
       ▼
  breaker trips after N failures
       │
       ▼
  calls fail instantly, fall back
       │
       ▼
  ✓ dep gets breathing room to recover

A circuit breaker flips the logic: when a dependency is clearly unwell, stop calling it for a while. Fail immediately, serve a fallback, and give the downstream room to breathe instead of a mob to fight.

What a circuit breaker actually does

The name comes from electrical panels, and the metaphor is exact. When current surges, the breaker trips and cuts the circuit before the wiring catches fire. You don't rewire the house, you flip the switch back once things are safe.

Structurally, it's a wrapper that sits between your service and the dependency, and it decides, per call, whether that call is even allowed to happen:

                        ┌─────────────┐
   Your service ─────▶ │   BREAKER   │───▶  Payment API
                        └─────────────┘       (the dependency)

      closed    → call passes through to Payment API
      open      → fail fast, serve the fallback (dependency not called)
      half-open → let a single probe through, then decide

A software breaker wraps a call and lives in one of three states:

• Closed — normal. Calls pass through. Failures are counted.
• Open — tripped. Calls fail instantly without touching the dependency, for a cooldown window. This is the whole point: you stop hammering a service that's already down.
• Half-open — probing. After the cooldown, let one trial call through. If it succeeds, close the breaker and resume. If it fails, open again and wait longer.

        failures ≥ threshold
  CLOSED ───────────────────────▶ OPEN
    ▲                              │
    │ probe succeeds               │ cooldown elapsed
    │                              ▼
    └──────────────────────── HALF-OPEN
              probe fails ──────────┘

That half-open state is what makes the breaker self-healing. It doesn't need a human to reset it, and it doesn't blindly slam the full traffic back onto a service that might still be fragile. It tests the water with one toe.

Building one in Node.js

The mechanism is small enough to write in a screen of TypeScript. Doing it once by hand is the best way to understand what a library is doing for you later.

type State = "closed" | "open" | "half-open";
 
interface BreakerOptions {
  failureThreshold: number; // consecutive failures before it trips
  resetTimeout: number; // ms to stay open before probing
  callTimeout: number; // ms before a slow call counts as a failure
}
 
class CircuitBreaker<T> {
  private state: State = "closed";
  private failures = 0;
  private openedAt = 0;
 
  constructor(
    private readonly action: () => Promise<T>,
    private readonly fallback: () => Promise<T>,
    private readonly opts: BreakerOptions,
  ) {}
 
  async call(): Promise<T> {
    if (this.state === "open") {
      // Still cooling down? Fail fast, don't touch the dependency.
      if (Date.now() - this.openedAt < this.opts.resetTimeout) {
        return this.fallback();
      }
      this.state = "half-open"; // cooldown elapsed, allow one probe
    }
 
    try {
      const result = await this.withTimeout(this.action());
      this.onSuccess();
      return result;
    } catch {
      this.onFailure();
      return this.fallback();
    }
  }
 
  private onSuccess() {
    this.failures = 0;
    this.state = "closed"; // a good probe closes the circuit
  }
 
  private onFailure() {
    this.failures++;
    // Trip on threshold, OR immediately if the probe in half-open failed.
    if (
      this.state === "half-open" ||
      this.failures >= this.opts.failureThreshold
    ) {
      this.state = "open";
      this.openedAt = Date.now();
    }
  }
 
  private withTimeout(p: Promise<T>): Promise<T> {
    return new Promise((resolve, reject) => {
      const timer = setTimeout(
        () => reject(new Error("call timed out")),
        this.opts.callTimeout,
      );
      p.then(resolve, reject).finally(() => clearTimeout(timer));
    });
  }
}

Notice the two things this gives you that a bare try/catch never could. When the breaker is open, the dependency is never even called, so a failing service gets zero traffic from you and a chance to recover. And every call is bounded by callTimeout, so the slow dependency, the dangerous one, is converted into a fast failure your caller can act on.

Wiring it up reads clean at the call site:

const paymentsBreaker = new CircuitBreaker(
  () => paymentsApi.charge(order),
  async () => {
    // Fallback: queue the charge to reconcile later instead of blocking checkout.
    await outbox.enqueue({ type: "charge", order });
    return { status: "queued" as const };
  },
  { failureThreshold: 5, resetTimeout: 10_000, callTimeout: 2_000 },
);
 
const result = await paymentsBreaker.call();

The nuances that decide if it actually helps

A breaker is easy to add and easy to add wrong. The details below are where a protective breaker and a decorative one diverge.

The timeout is the real trigger, not the error. Most outages I've watched weren't clean errors, they were latency. If your breaker only counts thrown exceptions and not slow responses, it never trips on the failure mode that hurts most. The callTimeout above is not optional, it's the point. A breaker without an aggressive timeout is a smoke detector with the battery out.

Not every error is a breaker error. A 404, a validation 400, a "card declined", these are the dependency working correctly and telling you "no." Count those as failures and your breaker will trip on a healthy service during a spike of legitimate declines, turning a normal Tuesday into a self-inflicted outage. Only count failures that mean "the dependency is unwell", timeouts, connection errors, 5xx, never the business outcomes you asked for.

One breaker per dependency, never one global. The whole idea is isolation, the same bulkhead thinking behind read-only modes. If search and payments share a breaker, a search outage stops your payments, which is the cascade you were trying to prevent, just rebuilt with extra steps. Give each downstream its own breaker with its own thresholds, because "unhealthy" means something different for each.

A breaker without a fallback just relocates the failure. Tripping the circuit turns a slow failure into a fast one, that's real, but if "fast failure" still means a 500 to your user, you've only sped up the disappointment. The breaker earns its keep in the fallback: serve stale cache, degrade the feature, queue the write, hide the section. Failing fast is step one. Failing usefully is the goal.

Consecutive-count is a teaching model; use a rolling window in production. Counting consecutive failures, like the class above, is simple to reason about but brittle: a single lucky success resets the count and masks a service that's failing 40% of calls. Real breakers trip on an error rate over a rolling window (say, "open if >50% of the last 20 calls failed"), which is what production libraries implement.

Don't hand-roll this in production

Writing one by hand is how you understand it. Shipping one by hand is how you acquire a subtle bug in your recovery path at the worst possible time. In Node, reach for opossum: it gives you the rolling-window stats, half-open probing, per-call timeouts, and, crucially, metrics events you can pipe to your observability stack. A breaker you can't see is a breaker you can't trust. You want a dashboard that shows, per dependency, when circuits open and how often, because an open circuit is one of the earliest, clearest signals that something downstream is breaking, often before your own error rate moves.

The trade-offs, because this isn't free

• A breaker can fail closed on a false positive. Tune thresholds too tight and a brief hiccup trips the circuit, cutting off a dependency that was actually fine. Too loose and it never trips when you need it. There's no universal number, you set it per dependency and adjust from real metrics.
• Fallbacks are real code that rots. Every fallback is a second path that has to stay correct, and it only ever runs during incidents, exactly when you're not looking. An untested fallback is a bug you've scheduled for your worst day. Exercise it on purpose in staging.
• State lives per-process. In a fleet of instances, each has its own breaker, so ten pods discover the outage ten times. That's usually fine, but if you need coordinated tripping, that's extra machinery (shared state, and its own failure modes) you should add only when you've proven you need it.

Why this makes your infrastructure cheaper

The cost angle, the one that runs through everything I write: a breaker lets you size for the normal case instead of the catastrophe. Without one, a single slow dependency forces you to over-provision everything upstream, more instances, bigger connection pools, longer queues, all to absorb the backpressure of a downstream that occasionally misbehaves. You pay, 24/7, for headroom whose only job is to survive someone else's bad day.

With a breaker, that bad day gets contained instead of absorbed. Calls fail fast, resources release immediately, and the blast radius stops at one feature instead of your whole service. You don't buy capacity to wait out a slow dependency, you shed the call and move on. Resilience and cost efficiency turn out to be the same decision viewed from two angles, and the circuit breaker is one of the cleanest places you'll ever see it.

Is your breaker protecting you, or decorating you?

1. Does it trip on slow responses, not just thrown errors? (If there's no timeout, the answer is no.)
2. Does it ignore business errors like 404 and 400, and count only "the dependency is unwell" failures?
3. Does each dependency have its own breaker, so one outage can't cascade into another?
4. When it's open, does the fallback serve something useful, or just a faster 500?
5. Does it trip on an error rate over a window, not a fragile consecutive count?
6. Can you see it, an open circuit on a dashboard, before your users can?

If you can't answer #6, start there. A circuit breaker's best feature isn't that it fails fast. It's that it tells you, early and unambiguously, exactly which downstream is about to ruin your afternoon, while you still have time to do something about it.