How Do I Implement a Circuit Breaker?

Key takeaways

A circuit breaker contains the damage from an API that's acting up. It tracks calls, stops sending them when the API appears unhealthy, and allows a limited probe after a cooldown period.
Retries and timeouts don't fully stop cascading failures. If hundreds of customer integrations hit a slow API, retries can cascade into storms while workers and connections are held.
Only qualifying failures should trip the breaker. Timeouts, DNS failures, connection resets, and 5xx responses should trip it. A 429 is throttling handled at the provider's limit scope, honoring Retry-After, and 400, 401, 403, 404, 409, and 422 probably shouldn't.
Multi-tenant integrations need breakers scoped by vendor and customer. A vendor-level circuit breaker catches server-side failures, but a per-customer circuit breaker isolates an issue with a single customer.
An open circuit must defer work but never drop it. Persist the webhook or scheduled sync and replay it after recovery. A timed-out write might have succeeded for the third-party API, so a proper replay should use the vendor's idempotency keys or deduplication.

It's 2 AM, and your customer's CRM API has started timing out. Nothing is technically down, but requests take 30 seconds and then fail. The integration retries, the job queue backs up, and (depending on your infrastructure) other customers' integrations may also be running behind.

You probably didn't ship a bug. A dependency you don't control got flaky, and your integration had no way to contain the failure.

This is a common failure mode in B2B SaaS integrations, and one useful defense is the circuit breaker pattern. A circuit breaker tracks calls to a dependency, stops sending calls when failures indicate it may be unhealthy, and allows a limited probe after a cooldown. It won't make a flaky API reliable, but it can keep that API's problems from spreading. Below, we'll cover how circuit breakers work, a small TypeScript implementation, and what changes when you run integrations for many customers at once.

Why retries and timeouts are insufficient

Retries are useful for temporary failures: a dropped connection, a 503 during a vendor deploy, or a slow request. A bounded retry with backoff is great for recovering from short interruptions like these.

However, retries can become liabilities during a longer failure. Picture an API that starts timing out. One integration waits five seconds, retries twice, and reports a failure. But, multiply that by 400 customer integrations hitting the same third-party API at once, and we may have:

Retry storms. Many clients retrying at once can keep a recovering API pinned down. Backoff with jitter spreads retries out, but it doesn't decide when to stop.
Resource exhaustion. Each in-flight request holds a connection, worker slot, and memory. Slow failures hold them longest.
Vendor penalties. Sustained traffic may trigger stricter rate limits, temporary lockouts, or other account restrictions.

A slow dependency can be more dangerous than a dead one. A dead API often fails fast and releases resources. A slow API holds calls until a timeout fires, which can tie up workers and create a larger incident.

Each resilience mechanism answers a different question. Only a circuit breaker asks whether calls to a dependency should be paused:

Mechanism	What it protects
Timeout	A single request that takes too long
Retry	A temporary failure that may succeed on another attempt
Backoff with jitter	Retry traffic that is too heavy or synchronized
Rate limiter	Exceeding the vendor's permitted request volume
Circuit breaker	Repeated qualifying failures from a dependency

The term comes from the circuit breaker in an electrical panel. When a circuit draws too much current, the breaker trips and cuts power before the wiring overheats and something burns down. You fix the problem, reset the breaker, and the circuit is fine.

A software circuit breaker wraps calls to a dependency and uses three states:

Closed (normal operation). Requests flow through. The circuit breaker records qualifying failures.
Open (tripped). Requests fail immediately without touching the API for a cooldown period. Your app gets a fast, predictable error instead of a slow one, and the vendor gets breathing room.
Half-open (testing). After the cooldown, the circuit breaker allows a limited number of trial requests. A successful probe closes the circuit breaker; a qualifying failure reopens it and restarts the cooldown.

The key here is that the circuit breaker can turn an unknown, slow failure into a known, fast one. Fast failures don't hold connections or pile up in queues, and they signal your code to defer work or use an appropriate fallback.

The pattern was popularized by Michael Nygard in Release It!; Martin Fowler's circuit-breaker primer is also a helpful reference.

Deciding what counts as a failure

Classify errors at the HTTP-client boundary, and make the classification specific to the provider and operation.

Response or error	Counts toward tripping	Why
Timeout, DNS failure, connection reset	Usually	The dependency may be unreachable or too slow
500, 502, 503, 504	Usually	Often indicates a server-side failure; confirm provider semantics
429 Too Many Requests	Usually not in the availability breaker	Handle as throttling at the scope the provider limits; honor Retry-After when supplied
401, 403	Usually no	Often indicates credentials or authorization that need attention
400, 404, 409, 422	Usually no	Often indicates a request, record, or business-rule problem

These are starting points only. For example, some providers use 4xx responses for transient conditions, and a 5xx response may apply to a specific endpoint. A 429 means the caller exceeded a limit, but the provider defines how it identifies the caller and counts requests. Retry-After, when present, can specify when to try again. See RFC 6585 and the provider's API documentation.

For 4xx errors caused by bad records, you'll need to log the issue, trigger an appropriate error, and only continue if the integration's data guarantees will allow. An improperly formed record shouldn't trip a shared circuit breaker.

For 429s, use a rate-limit pause or scheduler keyed to the right limit. This could be a credential, customer, endpoint, or even a provider scope. And don't immediately treat every 429 as a vendor-wide outage (it may not be). Finally, don't replace a provider's Retry-After function with a generic cooldown.

Building a circuit breaker step by step

The following is a compact, single-process example and not a production-ready library. It guards against stale in-flight calls changing a newer circuit breaker state, but it does not provide rolling-window statistics, metrics, distributed coordination, or provider-specific rate-limit handling. For production, evaluate a maintained library such as Opossum or Cockatiel.

Step 1: Define state and configuration

Use settings to define how many qualifying failures are necessary to trip the circuit breaker (set it to open), how long it stays tripped, and how many trial calls it permits while half-open. Use an explicit classifier so unknown errors are not automatically treated as dependency failures.

123456789101112131415161718192021
type State = "CLOSED" | "OPEN" | "HALF_OPEN";

export interface CircuitBreakerOptions {
  failureThreshold: number;
  cooldownMs: number;
  halfOpenMaxCalls: number;
  isFailure: (err: unknown) => boolean;
}

interface CircuitBreakerSnapshot {
  state: State;
  failures: number;
  openedAt: number;
}

export class CircuitOpenError extends Error {
  constructor(public retryAt?: number) {
    super("Circuit is open; request was not sent");
    this.name = "CircuitOpenError";
  }
}

Step 2: Wrap calls and gate half-open probes

call() is the only path to the dependency. The synchronous prelude reserves half-open capacity before awaiting the request. Each call records the circuit breaker generation it started in, so a later result from an older generation can't overwrite a newer state.

12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152
export class CircuitBreaker {
  private state: State = "CLOSED";
  private failures = 0;
  private openedAt = 0;
  private generation = 0;
  private trialsInFlight = new Map<number, number>();

  constructor(private opts: CircuitBreakerOptions) {
    if (!Number.isInteger(opts.failureThreshold) || opts.failureThreshold < 1) {
      throw new Error("failureThreshold must be a positive integer");
    }
    if (!Number.isFinite(opts.cooldownMs) || opts.cooldownMs <= 0) {
      throw new Error("cooldownMs must be positive");
    }
    if (!Number.isInteger(opts.halfOpenMaxCalls) || opts.halfOpenMaxCalls < 1) {
      throw new Error("halfOpenMaxCalls must be a positive integer");
    }
  }

  async call<T>(fn: () => Promise<T>): Promise<T> {
    if (this.state === "OPEN") {
      const retryAt = this.openedAt + this.opts.cooldownMs;
      if (Date.now() < retryAt) throw new CircuitOpenError(retryAt);
      this.setState("HALF_OPEN");
    }

    const isTrial = this.state === "HALF_OPEN";
    const callGeneration = this.generation;

    if (isTrial) {
      const inFlight = this.trialsInFlight.get(callGeneration) ?? 0;
      if (inFlight >= this.opts.halfOpenMaxCalls) {
        throw new CircuitOpenError();
      }
      this.trialsInFlight.set(callGeneration, inFlight + 1);
    }

    try {
      const result = await fn();
      this.onSuccess(callGeneration, isTrial);
      return result;
    } catch (err) {
      if (this.opts.isFailure(err)) {
        this.onFailure(callGeneration, isTrial);
      } else {
        this.onNonFailure(callGeneration, isTrial);
      }
      throw err;
    } finally {
      if (isTrial) this.releaseTrial(callGeneration);
    }
  }

Step 3: Handle state transitions

Only results from the current generation can change state. A successful call resets the failure count. A non-qualifying error during half-open (such as a 400 response) shows that the dependency responded, so it closes the circuit breaker but lets the original error continue to the caller.

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354
  private onSuccess(callGeneration: number, isTrial: boolean) {
    if (callGeneration !== this.generation) return;

    if (isTrial) {
      this.failures = 0;
      this.setState("CLOSED");
    } else if (this.state === "CLOSED") {
      this.failures = 0;
    }
  }

  private onNonFailure(callGeneration: number, isTrial: boolean) {
    if (callGeneration !== this.generation) return;

    if (isTrial) {
      this.failures = 0;
      this.setState("CLOSED");
    }
  }

  private onFailure(callGeneration: number, isTrial: boolean) {
    if (callGeneration !== this.generation) return;

    if (isTrial) {
      this.openedAt = Date.now();
      this.setState("OPEN");
      return;
    }

    if (this.state !== "CLOSED") return;
    this.failures++;

    if (this.failures >= this.opts.failureThreshold) {
      this.openedAt = Date.now();
      this.setState("OPEN");
    }
  }

  private setState(next: State) {
    if (this.state !== next) {
      this.state = next;
      this.generation++;
    }
  }

  private releaseTrial(callGeneration: number) {
    const remaining = (this.trialsInFlight.get(callGeneration) ?? 1) - 1;
    if (remaining <= 0) this.trialsInFlight.delete(callGeneration);
    else this.trialsInFlight.set(callGeneration, remaining);
  }

  get currentState() {
    return this.state;
  }

Step 4: Add snapshot support for persisted state

For a circuit breaker recreated on each execution, persist the state that matters between calls. Don't persist in-flight probe counts: they belong to the current process or execution.

123456789101112131415161718192021
  snapshot(): CircuitBreakerSnapshot {
    return {
      state: this.state,
      failures: this.failures,
      openedAt: this.openedAt,
    };
  }

  static restore(
    snapshot: CircuitBreakerSnapshot | undefined,
    opts: CircuitBreakerOptions,
  ) {
    const breaker = new CircuitBreaker(opts);
    if (snapshot) {
      breaker.state = snapshot.state;
      breaker.failures = snapshot.failures;
      breaker.openedAt = snapshot.openedAt;
    }
    return breaker;
  }
}

Step 5: Classify failures and use the circuit breaker

Wrap known transport failures in a specific type and don't treat every unknown exception as proof that the dependency is unhealthy.

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859
export class HttpError extends Error {
  constructor(public status: number) {
    super(`HTTP ${status}`);
    this.name = "HttpError";
  }
}

export class DependencyTransportError extends Error {
  constructor(message: string, options?: ErrorOptions) {
    super(message, options);
    this.name = "DependencyTransportError";
  }
}

export function isDependencyFailure(err: unknown): boolean {
  if (err instanceof DependencyTransportError) return true;
  if (err instanceof HttpError) return err.status >= 500;
  return false;
}

const crmBreaker = new CircuitBreaker({
  failureThreshold: 5,
  cooldownMs: 30_000,
  halfOpenMaxCalls: 1,
  isFailure: isDependencyFailure,
});

async function fetchContacts(token: string) {
  return crmBreaker.call(async () => {
    let res: Response;
    try {
      res = await fetch("https://api.example-crm.com/v1/contacts", {
        headers: { Authorization: `Bearer ${token}` },
        signal: AbortSignal.timeout(5_000),
      });
    } catch (err) {
      throw new DependencyTransportError("CRM request failed or timed out", {
        cause: err,
      });
    }

    if (!res.ok) throw new HttpError(res.status);
    return res.json();
  });
}

try {
  const contacts = await fetchContacts(token);
  // ...process contacts
} catch (err) {
  if (err instanceof CircuitOpenError) {
    // Defer the job durably; retryAt may be absent when half-open probes are busy.
  } else if (err instanceof HttpError && err.status === 429) {
    // Pause the affected rate-limit scope using provider guidance, including
    // Retry-After when supplied. Do not use the shared availability breaker.
  } else {
    throw err;
  }
}

The exact HTTP status classification is provider-specific. In particular, configure any transient 4xx codes only after checking the provider's documented semantics.

Two things to remember for implementation:

Create the circuit breaker once and reuse it within a process. One that is built per request has no shared failure history.
A timeout ends your client's wait but it does not prove the vendor didn't process a write. Retrying an ambiguous write can create duplicates, so use an idempotency key where possible.

Tuning the settings

A consecutive-failure threshold is simple, but the above example counts qualifying failures since the last successful dependency call while non-qualifying errors are neutral. For busier dependencies, a rolling failure-rate window can be more helpful. You'll want to include a minimum request volume so a tiny sample doesn't look more important than it is.

A cooldown from 15 to 60 seconds is a good starting point. If the cooldown is too short, then you get probes too often. If it is too long, it may delay recovery after the API is ready to work properly again. Set request deadlines based on the API's latency distribution and the work being done and not an arbitrary value.

Treat every number here as a suggestion. In the real work, you'll tune things against each API's latency, failure rate, rate-limit behavior, and workload.

Use a library

You don't have to write your own circuit breaker. Rolling windows, concurrent probes, telemetry, and state transitions take more care than our example might suggest. Check out libraries such as Opossum or Cockatiel for Node.js, Resilience4j for Java, Polly for .NET, or pybreaker for Python.

Combining circuit breakers with other resilience patterns

A circuit breaker is one layer in a resilience stack, not a replacement for the others. For a single API call, a useful arrangement is:

Timeout. Bounds how long an attempt can take.
Retry with exponential backoff and jitter. Absorbs transient blips. Keep attempts bounded and make side-effecting operations safe to repeat.
Circuit breaker around the retry operation. A call that exhausts its retry budget counts as one circuit breaker outcome, rather than each internal attempt counting separately.
Fallback or deferral. Decides what happens when the circuit breaker is open.

Two more patterns will help:

Bulkheads. Give dependencies separate connection pools or concurrency limits so a slow CRM can't starve billing or ERP calls.
Fallbacks that fit the integration. For reads, serve cached data and mark it as stale. For writes, queue the records and replay them after recovery. For user-facing features, pause the sync, and let the user know that it will retry (instead of leaving a spinner on the screen).

Choose your fallbacks very carefully: a cached contact name may be fine on a dashboard, but a account balance or permission check may not be. And don't retry requests that the circuit breaker has rejected. An open circuit means you ought to defer or use a safe fallback – not start another retry loop.

123456
const result = await breaker.call(() =>
  retryWithBackoff(
    () => fetchWithTimeout(url, 5_000),
    { attempts: 3 },
  ),
);

Circuit breakers for multi-tenant integrations

Setting up circuit breakers for B2B SaaS integrations adds another level of complexity. You might run the same CRM integration for hundreds of customers, each with its own configuration (credentials, data mapping, etc.). That changes the way you need to scope the circuit breakers.

Scope circuit breakers appropriately

Deciding what the circuit breaker protects is essential to determining scope:

The third-party API is failing. A vendor-specific circuit breaker can help when correlated server-side failures affect many customers on the same service or endpoint.
One customer's connection has failed. Perhaps their self-hosted instance is misconfigured, or their credentials have exhausted a quota.

Many systems need both scopes, for example salesforce and salesforce:customer-212. For example, circuit breakers tripping for numerous customers because of the same server-side failure may indicate a vendor-wide problem. In contrast, a bunch of 429s may indicate a shared rate-limit scope or unusually high client traffic.

Endpoints can also be separate failure domains. If a bulk-export endpoint returns 503s but contact lookups work, a shared circuit breaker could block healthy calls. Group the endpoints that fail together, such as bulk jobs, real-time reads, and reporting. At the same time, don't create so many circuit breakers that each of them has too little traffic to provide a useful signal.

An in-memory circuit breaker works within a single long-running process and does not coordinate independent workers or short-lived serverless executions. If cross-worker coordination matters for your scenario, use a shared store with atomic state transitions or type of coordination. A shared key-value snapshot can still lose updates under concurrent writes.

If the shared store becomes unavailable, decide whether it should fail open, fail closed, or fall back to local circuit breakers. A local fallback may limit traffic per worker, but it does not support vendor-wide coordination.

Never drop data while the circuit is open

For data-sync integrations, failing fast is only the first step. When a webhook arrives or a scheduled sync fires while the circuit breaker is open, persist the work or checkpoint it so you can retry it after recovery. Customers usually tolerate a delayed sync but they aren’t as happy with a missing invoice.

Replay introduces additional risks. Queued writes may run out of order, and a timed-out request may still have succeeded on the vendor's side. Use idempotency keys or deduplication so replay doesn't create duplicates, and reconcile the systems as needed.

Make state changes visible

Record the circuit breaker state changes:

How often does a circuit breaker open?
How long does it stay open?
How many requests does it reject?
Do probes succeed?
How much deferred work is waiting?

An open circuit often means the breaker is working as intended. You’ll only want to alert someone when a critical vendor-wide circuit breaker stays open beyond a defined window.

A customer-facing message such as "Your HubSpot integration is paused because HubSpot is returning errors. We'll retry automatically." can save support time.

How Prismatic helps

If you build integrations on Prismatic, the platform offers several resilience capabilities that custom circuit breaker logic can complement.

Retries and error handling

In the low-code integration designer, a step can fail, ignore an error, or retry with configurable attempts and delay, including exponential backoff. When a step ignores its final error, its result includes an error property that can be used for branching. Automatic execution retry can retry configured asynchronous executions up to 10 times. Replay is a manual re-run using the same input. Because flow-level retry starts the flow again, make earlier writes safe to repeat.

Per-customer instances

A Prismatic instance is an integration as configured for a specific customer. This provides logical per-customer isolation.

Persisted state by scope

Flow state is scoped to one flow. Cross-flow state is shared across flows in one instance; Integration state is shared across instances of the same integration. That makes the scopes useful for storing small pieces of per-flow, per-customer, or integration-wide state. But persisted state is not a database or an atomic distributed circuit breaker. Instead, it is written when an execution completes successfully, and concurrent executions can overwrite one another's state.

Custom logic in code

With code-first integrations or custom connectors, you can implement a circuit breaker and use persisted state between executions. In the following example, durableWorkQueue.defer represents an application-owned durable queue, not a Prismatic API.

1234567891011121314151617181920212223242526272829
onExecution: async (context, params) => {
  const key = "crm-breaker";
  const breaker = CircuitBreaker.restore(
    context.crossFlowState[key] as CircuitBreakerSnapshot | undefined,
    {
      failureThreshold: 5,
      cooldownMs: 60_000,
      halfOpenMaxCalls: 1,
      isFailure: isDependencyFailure,
    },
  );

  try {
    const contacts = await breaker.call(() => fetchContacts(token));
    await syncContacts(contacts);
  } catch (err) {
    // Persist the work durably before returning success. Do not advance the
    // sync checkpoint until the work has actually completed.
    await durableWorkQueue.defer({
      payload: params,
      retryAt: err instanceof CircuitOpenError ? err.retryAt : undefined,
    });
    context.crossFlowState[key] = breaker.snapshot();
    return { data: { deferred: true } };
  }

  context.crossFlowState[key] = breaker.snapshot();
  return { data: null };
};

The above example handles failures or returns success only after work has been durably deferred. That allows successful-execution state persistence, but it also means Prismatic's flow-level automatic retry won't run for that execution. If you want the execution to fail in such a way that Prismatic can retry it, store circuit breaker coordination outside the success-only persisted-state path. Concurrent writes to persisted state can still overwrite each other, so don't rely on it for cross-worker coordination.

Monitoring

Prismatic provides execution logs, step results, and alerts for execution events. These provide a basis for you to investigate failures and notify your team. However, this data doesn’t mean you can provide a plain-language explanation or a particular timeline for every failed run.

Monitoring and testing your circuit breaker

A circuit breaker that opens and closes may protect your app, but it should also let you know when it does that. At a minimum, you’ll want to track each circuit breaker's state, how often it opens, how long it stays open, how many requests it rejects, whether recovery probes succeed, and how much deferred work is waiting.

Test the state machine and its integration with the work queue:

Unit-test transitions. Use fake timers and controllable promises to test opening, fail-fast rejection, cooldown, probe success, and probe failure.
Test concurrent completions. Start a slow call while the circuit breaker is closed, open it with other calls, then let the slow call succeed. Confirm that it does not close the newer open state.
Test classifications. Confirm that 400/401 errors reach the caller without tripping the circuit breaker, and that a non-qualifying error arriving at half-open does not leave it stuck.
Test probe limits. Start multiple calls during half-open and confirm that only the configured number reach the dependency.
Inject faults. Use a mock server or fault-injection proxy such as WireMock or Toxiproxy to return 503s or add latency. Confirm that the circuit breaker opens and work resumes after the fault is cleared.
Verify side effects. Confirm that deferred work survives, alerts fire, fallbacks serve appropriate data, and replayed records arrive without duplicates. If multiple workers are involved, test concurrent state updates and other behavior when the shared store is unavailable.

Circuit breakers help prevent outages

Flaky third-party APIs are something we all must deal with. Cascading failures don't need to be. A circuit breaker can turn a slow, unpredictable situation into a fast, predictable failure, protecting your app and your customers.

A circuit breaker won't fix the third-party API or guarantee that interrupted work gets done. But when it is paired with proper failure classification, scoping, durable deferral, and idempotent replay/retries, it can prevent outages in your system.

Start a free trial to see how Prismatic's error handling, retry and replay, and per-customer instances support integrations when third-party APIs get a little bit sideways.

How Do I Implement a Circuit Breaker to Protect My App from Flaky Third-Party APIs?

Why retries and timeouts are insufficient

Deciding what counts as a failure

Building a circuit breaker step by step

Step 1: Define state and configuration

Step 2: Wrap calls and gate half-open probes

Step 3: Handle state transitions

Step 4: Add snapshot support for persisted state

Step 5: Classify failures and use the circuit breaker

Tuning the settings

Use a library

Combining circuit breakers with other resilience patterns

Circuit breakers for multi-tenant integrations

Scope circuit breakers appropriately

Never drop data while the circuit is open

Make state changes visible

How Prismatic helps

Retries and error handling

Per-customer instances

Persisted state by scope

Custom logic in code

Monitoring

Monitoring and testing your circuit breaker

Circuit breakers help prevent outages

Related Resources

Ready to make your product extensible?

How Do I Implement a Circuit Breaker to Protect My App from Flaky Third-Party APIs?

Why retries and timeouts are insufficient

Deciding what counts as a failure

Building a circuit breaker step by step

Step 1: Define state and configuration

Step 2: Wrap calls and gate half-open probes

Step 3: Handle state transitions

Step 4: Add snapshot support for persisted state

Step 5: Classify failures and use the circuit breaker

Tuning the settings

Use a library

Combining circuit breakers with other resilience patterns

Circuit breakers for multi-tenant integrations

Scope circuit breakers appropriately

Share the state across workers

Never drop data while the circuit is open

Make state changes visible

How Prismatic helps

Retries and error handling

Per-customer instances

Persisted state by scope

Custom logic in code

Monitoring

Monitoring and testing your circuit breaker

Circuit breakers help prevent outages

Related Resources

Ready to make your product extensible?