object Balancers
Constructor methods for various load balancers. The methods take balancer
specific parameters and return a LoadBalancerFactory that allows you
to easily inject a balancer into the Finagle client stack via the
withLoadBalancer
method.
configuring a client with a load balancer
$Protocol.client .withLoadBalancer(Balancers.aperture()) .newClient(...)
- See also
The user guide for more details.
- Alphabetic
- By Inheritance
- Balancers
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##(): Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- val MaxEffort: Int
This is the default for the fixed number of retries an LB implementation is willing to make if an unavailable (Status != Open) node is returned from the underlying pick.
This is the default for the fixed number of retries an LB implementation is willing to make if an unavailable (Status != Open) node is returned from the underlying pick.
For randomized LBs (P2C* and Aperture*) this can be interpreted as a probability. For example, imagine that half of the replica set is down, the probability of picking two unavailable nodes is 0.25. If we repeat that process for 5 times, the total probability of seeing 5 unavailable nodes in a row, will be (0.25 ^ 5) = 0.1%. This means that if half of the cluster is down, the LB will be making a bad choice (when better choice may have been available) for 0.1% of requests.
Please, note that this doesn't mean that 0.1% of requests will be failed by P2C operating on a half-dead cluster since Finagle clients have additional layers of requeues above the load balancer.
- def aperture(smoothWin: Duration = 15.seconds, lowLoad: Double = 0.875, highLoad: Double = 1.125, minAperture: Int = 1, maxEffort: Int = MaxEffort, rng: Rng = Rng.threadLocal, useDeterministicOrdering: Option[Boolean] = None): LoadBalancerFactory
The aperture load-band balancer balances load to the smallest subset ("aperture") of services so that the concurrent load to each service, measured over a window specified by
smoothWin
, stays within the load band delimited bylowLoad
andhighLoad
.The aperture load-band balancer balances load to the smallest subset ("aperture") of services so that the concurrent load to each service, measured over a window specified by
smoothWin
, stays within the load band delimited bylowLoad
andhighLoad
.The default load band configuration attempts to create a 1:1 relationship between a client's offered load and the aperture size. Given a homogeneous replica set – this optimizes for at least one concurrent request per node and gives the balancer sufficient data to compare load across nodes.
Among the benefits of aperture balancing are:
- A client uses resources commensurate to offered load. In particular, it does not have to open sessions with every service in a large cluster. This is especially important when offered load and cluster capacity are mismatched. 2. It balances over fewer, and thus warmer, services. This enhances the efficacy of the fail-fast mechanisms, etc. This also means that clients pay the penalty of session establishment less frequently. 3. It increases the efficacy of least-loaded balancing which, in order to work well, requires concurrent load. The load-band balancer effectively arranges load in a manner that ensures a higher level of per-service concurrency.
- smoothWin
The time window to use when calculating the rps observed by this load balancer instance. The smoothed rps value is then used to determine the size of the aperture (along with the
lowLoad
andhighLoad
parameters).- lowLoad
The lower threshold on average load (as calculated by rps over smooth window / # of endpoints). Once this threshold is reached, the aperture is narrowed. Put differently, if there is an average of
lowLoad
requests across the endpoints, then we are not concentrating our concurrency well, so we narrow the aperture.- highLoad
The upper threshold on average load (as calculated by rps / # of instances). Once this threshold is reached, the aperture is widened. Put differently, if there is an average of
highLoad
requests across the endpoints, then we are over subscribing the endpoints and need to widen the aperture.- minAperture
The minimum aperture allowed. Note, this value is checked to ensure that it is not larger than the number of endpoints.
- maxEffort
This is the fixed number of retries the LB is willing to make if an unavailable node (Status != Open) is returned from the underlying pick. See the constant MaxEffort for more details on how we pick the default.
- rng
The PRNG used for flipping coins. Override for deterministic tests.
- useDeterministicOrdering
Enables the aperture instance to make use of the coordinate in com.twitter.finagle.loadbalancer.aperture.ProcessCoordinate to calculate an order for the endpoints. In short, this allows coordination for apertures across process boundaries to avoid load concentration when deployed in a distributed system.
- See also
The user guide for more details.
- def aperturePeakEwma(smoothWin: Duration = 15.seconds, lowLoad: Double = 0.875, highLoad: Double = 1.125, minAperture: Int = 1, maxEffort: Int = MaxEffort, rng: Rng = Rng.threadLocal, useDeterministicOrdering: Option[Boolean] = None): LoadBalancerFactory
Like aperture but but using the Peak EWMA (exponentially weight moving average) load metric.
Like aperture but but using the Peak EWMA (exponentially weight moving average) load metric.
Peak EWMA uses a moving average over an endpoint's round-trip time (RTT) that is highly sensitive to peaks. This average is then weighted by the number of outstanding requests. Effectively, increasing our resolution per-request and providing a higher fidelity measurement of server responsiveness compared to the standard least loaded. It is designed to react to slow endpoints more quickly than least-loaded by penalizing them when they exhibit slow response times. This load metric operates under the assumption that a loaded endpoint takes time to recover and so it is generally safe for the advertised load to incorporate an endpoint's history. However, this assumption breaks down in the presence of long polling clients.
- smoothWin
The time window to use when calculating the rps observed by this load balancer instance. The smoothed rps value is used to determine the size of aperture (along with the lowLoad and highLoad parameters). In the context of peakEwma, this value is also used to average over the latency observed per endpoint. It's unlikely that you would want to measure the latency on a different time scale than rps, so we couple the two.
- lowLoad
The lower threshold on average load (as calculated by rps over smooth window / # of endpoints). Once this threshold is reached, the aperture is narrowed. Put differently, if there is an average of
lowLoad
requests across the endpoints, then we are not concentrating our concurrency well, so we narrow the aperture.- highLoad
The upper threshold on average load (as calculated by rps / # of instances). Once this threshold is reached, the aperture is widened. Put differently, if there is an average of
highLoad
requests across the endpoints, then we are over subscribing the endpoints and need to widen the aperture.- minAperture
The minimum aperture allowed. Note, this value is checked to ensure that it is not larger than the number of endpoints.
- maxEffort
This is the fixed number of retries the LB is willing to make if an unavailable node (Status != Open) is returned from the underlying pick. See the constant MaxEffort for more details on how we pick the default.
- rng
The PRNG used for flipping coins. Override for deterministic tests.
- useDeterministicOrdering
Enables the aperture instance to make use of the coordinate in com.twitter.finagle.loadbalancer.aperture.ProcessCoordinate to calculate an order for the endpoints. In short, this allows coordination for apertures across process boundaries to avoid load concentration when deployed in a distributed system.
- See also
The user guide for more details.
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[java.lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def finalize(): Unit
- Attributes
- protected[java.lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable])
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def heap(rng: Random = new Random): LoadBalancerFactory
An efficient strictly least-loaded balancer that maintains an internal heap to select least-loaded endpoints.
An efficient strictly least-loaded balancer that maintains an internal heap to select least-loaded endpoints.
- See also
The user guide for more details.
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- def p2c(maxEffort: Int = MaxEffort, rng: Rng = Rng.threadLocal): LoadBalancerFactory
An O(1), concurrent, least-loaded fair load balancer.
An O(1), concurrent, least-loaded fair load balancer. This uses the ideas behind "power of 2 choices" [1].
- maxEffort
This is the fixed number of retries the LB is willing to make if an unavailable node (Status != Open) is returned from the underlying pick. See the constant MaxEffort for more details on how we pick the default.
- rng
The PRNG used for flipping coins. Override for deterministic tests. [1] Michael Mitzenmacher. 2001. The Power of Two Choices in Randomized Load Balancing. IEEE Trans. Parallel Distrib. Syst. 12, 10 (October 2001), 1094-1104.
- def p2cPeakEwma(decayTime: Duration = 10.seconds, maxEffort: Int = MaxEffort, rng: Rng = Rng.threadLocal): LoadBalancerFactory
Like p2c but using the Peak EWMA (exponentially weight moving average) load metric.
Like p2c but using the Peak EWMA (exponentially weight moving average) load metric.
Peak EWMA uses a moving average over an endpoint's round-trip time (RTT) that is highly sensitive to peaks. This average is then weighted by the number of outstanding requests. Effectively, increasing our resolution per-request and providing a higher fidelity measurement of server responsiveness compared to the standard least loaded. It is designed to react to slow endpoints more quickly than least-loaded by penalizing them when they exhibit slow response times. This load metric operates under the assumption that a loaded endpoint takes time to recover and so it is generally safe for the advertised load to incorporate an endpoint's history. However, this assumption breaks down in the presence of long polling clients.
- decayTime
The window of latency observations.
- maxEffort
This is the fixed number of retries the LB is willing to make if an unavailable node (Status != Open) is returned from the underlying pick. See the constant MaxEffort for more details on how we pick the default.
- rng
The PRNG used for flipping coins. Override for deterministic tests.
- See also
The user guide for more details.
- def roundRobin(maxEffort: Int = MaxEffort): LoadBalancerFactory
A simple round robin balancer that chooses the next endpoint in the list for each request.
A simple round robin balancer that chooses the next endpoint in the list for each request.
WARNING: Unlike other balancers available in finagle, this does not take latency into account and will happily direct load to slow or oversubscribed services. We recommend using one of the other load balancers for typical production use.
- maxEffort
This is the fixed number of retries the LB is willing to make if an unavailable node (Status != Open) is returned from the underlying pick. See the constant MaxEffort for more details on how we pick the default.
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()