Microservices: Circuit Breaker pattern for improving resilience

Microservices architecture distributes applications across multiple services, introducing challenges in their interactions. Failures—whether due to network issues, container problems, or service overloads—are inevitable and can disrupt entire operation chains. The Circuit Breaker pattern addresses these challenges by isolating faulty services to prevent cascading failures, ensuring other services remain operational.

Implementation of circuit breaker for every external service in a distributed system.

This use case demonstrates how implementing this pattern can enhance resilience and minimize disruptions in your integrations. By the end of this article, you'll learn how Digibee can help you implement the Circuit Breaker pattern ensuring your systems remain resilient in the face of failures.

Why Circuit Breakers matter

A Circuit Breaker is a design pattern that helps manage failures gracefully. Just like an electrical circuit breaker prevents damage by cutting off the flow of electricity when an overload occurs, a software circuit breaker temporarily stops requests to a service when it detects repeated failures.

For businesses, this approach is invaluable. By preventing cascading failures, the Circuit Breaker gives systems the time needed to recover without overwhelming resources. This leads to a more stable user experience and reduces the risk of revenue loss due to downtime.

How it works

A Circuit Breaker monitors the success and failure rates of requests to a service and switches between three states:

• Closed Circuit: All requests pass through, assuming the service is healthy.

• Open Circuit: Requests are blocked after failures exceed a predefined threshold*, avoiding further strain on the failing service.

• Half-Open Circuit: A limited number of test requests are allowed to check if the service has recovered.

Circuit Breaker states

By using a Circuit Breaker, you can:

• Prevent overloading failing services by halting unnecessary requests.

• Provide fallback responses* to minimize user impact.

• Give the failing service time to recover before attempting to reconnect.

Use cases

Below are scenarios where circuit breakers significantly improve system reliability:

Scenario 1: Flight booking system

In a flight booking system, the Booking Service relies on the Flight Information Service to fetch available flight data. If the Flight Information Service experiences downtime or slow response times, the Booking Service can become unresponsive.

Backend process

1. The Booking Service sends requests to the Flight Information Service.

2. If the service fails repeatedly, the Circuit Breaker opens, preventing further requests to the failing service.

3. The Booking Service either returns a fallback response or informs the user that the service is unavailable.

Scenario 2: E-commerce order processing

An e-commerce platform depends on multiple microservices. If the Payment Service is experiencing issues, the Order Service can use a Circuit Breaker to prevent further payment requests from being sent until the service recovers.

Backend process

1. The Order Service sends payment requests to the Payment Service.

2. If the Payment Service encounters failures, the Circuit Breaker opens, and the Order Service either responds with a temporary error message or uses a fallback method to complete the order.

Applying theory into practice

Imagine a pipeline designed to integrate with external services, like an Address Service API, to validate addresses based on the postal code entered during user registration. The pipeline must meet the following requirements:

• Fetch full address details via the external service.

• Use a Circuit Breaker to handle server failures, preventing excessive retries.

For this use case, the focus will be on Circuit Breaker implementation, rather than on the pipeline itself.

Simplified representation of a circuit breaker flow: illustrating how requests to an Address Service API are managed during normal operation, failure scenarios, and circuit breaker recovery.

Step-by-step breakdown

1. Pipeline configuration

• Set up a REST trigger with the POST method to receive user data (postal code).

2. Postal code storage

• Save the data to execution memory (Session Management) and retrieve it before calling the Address API.

3. Check Circuit Breaker status

• The next step queries the Circuit Breaker status in a temporary database (such as an Object Store). It specifically checks whether the Circuit Breaker is open or closed.

  • If closed, the flow proceeds with the normal pipeline operations.

  • If not closed (indicating that recent failures exceeded the threshold), it blocks further requests to prevent cascading failures.

4. Handling circuit closure (Block Execution - onProcess)

• If the circuit is closed and an API call to the Address Service is made, the response is processed and returned to the user.

• If an error occurs (service unavailability, etc.), the pipeline is routed to the onException subflow to handle the failure.

5. Handling Circuit Closure (Block Execution - onException)

• The failure count is tracked in an Object Store (or any other temporary database).

  • If the failure threshold is not exceeded, the failure count is incremented by 1.

  • If the failure threshold is exceeded (for example, 15 failures), the Circuit Breaker is triggered, and the status is updated to open to prevent further requests to the failing service. Thus, in the next request, the flow will be redirected to the 'open' circuit path, where the pipeline will block further attempts to interact with the address service until the circuit is closed again.

6. Circuit Breaker Opening

A JSON Generator connector calculates the time since the circuit was opened against the current execution timestamp. If it exceeds X minutes, reset the circuit to close and allow new requests.

7. Success or Failure Post Time-Elapsed

If the circuit has been open for more than X minutes, the circuit is closed (reset) and, during the next pipeline execution, the Circuit Breaker will attempt to reopen by sending a new request to the service.

In the above screenshot, you can see the Circuit Breaker pattern implemented within the pipeline.

To access an example of a revised Circuit Breaker pattern used in this use case, click here to be redirected to the Circuit Breaker challenge in Digibee Academy and look for the answer sheet there.

Final thoughts

The Circuit Breaker pattern is an important consideration when using a microservice architecture. This approach helps integrations maintain availability and resilience.

Explore more possibilities in our Documentation Portal, take on the Circuit Breaker Challenge, or enroll in the Metrics and Monitoring course and webinar at Digibee Academy. Additionally, visit our Blog for more use cases and insights.

Have feedback or suggestions for future articles? Let us know through our feedback form.