Returns a new circuit breaker that wraps the state of the source and that will fire the given callback upon the circuit breaker transitioning to the Closed state.
Returns a new circuit breaker that wraps the state of the source and that will fire the given callback upon the circuit breaker transitioning to the Closed state.
Useful for gathering stats.
NOTE: calling this method multiple times will create a circuit breaker that will call multiple callbacks, thus the callback given is cumulative with other specified callbacks.
is to be executed when the state evolves into Closed
a new circuit breaker wrapping the state of the source
Returns a new circuit breaker that wraps the state of the source and that will fire the given callback upon the circuit breaker transitioning to the HalfOpen state.
Returns a new circuit breaker that wraps the state of the source and that will fire the given callback upon the circuit breaker transitioning to the HalfOpen state.
Useful for gathering stats.
NOTE: calling this method multiple times will create a circuit breaker that will call multiple callbacks, thus the callback given is cumulative with other specified callbacks.
is to be executed when the state evolves into HalfOpen
a new circuit breaker wrapping the state of the source
Returns a new circuit breaker that wraps the state of the source and that will fire the given callback upon the circuit breaker transitioning to the Open state.
Returns a new circuit breaker that wraps the state of the source and that will fire the given callback upon the circuit breaker transitioning to the Open state.
Useful for gathering stats.
NOTE: calling this method multiple times will create a circuit breaker that will call multiple callbacks, thus the callback given is cumulative with other specified callbacks.
is to be executed when the state evolves into Open
a new circuit breaker wrapping the state of the source
Returns a new circuit breaker that wraps the state of the source
and that upon a task being rejected will execute the given
callback
.
Returns a new circuit breaker that wraps the state of the source
and that upon a task being rejected will execute the given
callback
.
Useful for gathering stats.
NOTE: calling this method multiple times will create a circuit breaker that will call multiple callbacks, thus the callback given is cumulative with other specified callbacks.
is to be executed when tasks get rejected
a new circuit breaker wrapping the state of the source
A factor to use for resetting the resetTimeout when in the
HalfOpen
state, in case the attempt for Close
fails.
The maximum count for allowed failures before opening the circuit breaker.
The maximum timespan the circuit breaker is allowed to use as a resetTimeout when applying the exponentialBackoffFactor.
Returns a new Task that upon execution will execute the given task, but with the protection of this circuit breaker.
The timespan to wait in the Open
state before attempting
a close of the circuit breaker (but without the backoff
factor applied).
The timespan to wait in the Open
state before attempting
a close of the circuit breaker (but without the backoff
factor applied).
If we have a specified exponentialBackoffFactor then the actual reset timeout applied will be this value multiplied repeatedly with that factor, a value that can be found by querying the state.
Returns the current TaskCircuitBreaker.State, meant for debugging purposes.
The
TaskCircuitBreaker
is used to provide stability and prevent cascading failures in distributed systems.Purpose
As an example, we have a web application interacting with a remote third party web service. Let's say the third party has oversold their capacity and their database melts down under load. Assume that the database fails in such a way that it takes a very long time to hand back an error to the third party web service. This in turn makes calls fail after a long period of time. Back to our web application, the users have noticed that their form submissions take much longer seeming to hang. Well the users do what they know to do which is use the refresh button, adding more requests to their already running requests. This eventually causes the failure of the web application due to resource exhaustion. This will affect all users, even those who are not using functionality dependent on this third party web service.
Introducing circuit breakers on the web service call would cause the requests to begin to fail-fast, letting the user know that something is wrong and that they need not refresh their request. This also confines the failure behavior to only those users that are using functionality dependent on the third party, other users are no longer affected as there is no resource exhaustion. Circuit breakers can also allow savvy developers to mark portions of the site that use the functionality unavailable, or perhaps show some cached content as appropriate while the breaker is open.
How It Works
The circuit breaker models a concurrent state machine that can be in any of these 3 states:
TaskCircuitBreaker
startsfailures
counterfailures
counter reaches themaxFailures
count, the breaker is tripped intoOpen
stateExecutionRejectedException
resetTimeout
, the circuit breaker enters a HalfOpen state, allowing one task to go through for testing the connectionOpen
has expired is allowed through without failing fast, just before the circuit breaker is evolved into theHalfOpen
stateHalfOpen
fail-fast with an exception just as in Open stateClosed
state, with theresetTimeout
and thefailures
count also reset to initial valuesOpen
state (theresetTimeout
is multiplied by the exponential backoff factor)Usage
When attempting to close the circuit breaker and resume normal operations, we can also apply an exponential backoff for repeated failed attempts, like so:
In this sample we attempt to reconnect after 10 seconds, then after 20, 40 and so on, a delay that keeps increasing up to a configurable maximum of 10 minutes.
Credits
This Monix data type was inspired by the availability of Akka's Circuit Breaker.