Package com.yahoo.messagebus

Class DynamicThrottlePolicy

java.lang.Object
com.yahoo.messagebus.StaticThrottlePolicy
com.yahoo.messagebus.DynamicThrottlePolicy
All Implemented Interfaces:
ThrottlePolicy
public class DynamicThrottlePolicy extends StaticThrottlePolicy
This is an implementation of the ThrottlePolicy that offers dynamic limits to the number of pending messages a SourceSession is allowed to have. Pending means the number of sent messages that have not been replied to yet.

The algorithm works by increasing the number of messages allowed to be pending, the window size, until this no longer increases throughput. At this point, the algorithm is driven by synthetic attraction towards latencies which satisfy log10(1 / latency) % 1 = e, for some constant 0 < e < 1. Weird? Most certainly!

The effect is that the window size the algorithm produces, for a saturated ideal server, has a level for each power of ten with an attractor the window size tends towards while on this level, determined by the e above. The setEfficiencyThreshold(double) determines the e constant. When e is set so the attractor is close to the start of the interval, this has an inhibiting effect on the algorithm, and it is basically reduced to "increase window size until this no longer increases throughput enough that it defeats the random noise". As the attractor moves towards the end of the intervals, the algorithm becomes increasingly eager in increasing its window size past what it measures as effective — if moved to the very end of the interval, the algorithm explodes. The default setting has the attractor at log10(2) of the way from start to end of these levels.

Because the algorithm stops increasing the window size when increases in throughput drown in random variations, the setWindowSizeIncrement(double) directly influences the efficient work domain of the algorithm. With the default setting of 20, it seems to struggle to increase past window sizes of a couple thousand. Using a static increment (and a backoff factor) seems to be necessary to effectively balance the load different, competing policies are allowed to produce.

When there are multiple policies that compete against each other, they will increase load until saturating the server. If keeping all other policies but one fixed, this one policy would still see an increase in throughput with increased window size, even with a saturated server, as it would be given a greater share of the server's resources. However, since all algorithms adjust their windows concurrently, they will all reduce the throughput of the other algorithms. The result is that the commonly see the server as saturated, and commonly reach the behaviour where some increases in window size lead to measurable throughput gains, while others don't.

Now the weighting (setWeight(double) comes into play: with equals weights, two algorithms would have a break-even between being governed by the attractors (above), which eventually limits window size, and increases due to positive measurements, at the same point along the window size axis. With smaller weights, i.e., smaller increases to window size, this break-even occurs where the curve is steeper, i.e., where the client has a smaller share of the server. Thus, competing algorithms with different weights end up with a resource distribution roughly proportional to weight.

Author:
Simon Thoresen Hult, jonmv

  • Constructor Details

    • DynamicThrottlePolicy

      public DynamicThrottlePolicy()
      Constructs a new instance of this policy and sets the appropriate default values of member data.

    • DynamicThrottlePolicy

      public DynamicThrottlePolicy(com.yahoo.concurrent.Timer timer)
      Constructs a new instance of this class using the given clock to calculate efficiency.
      Parameters:
      timer - the timer to use

  • Method Details

    • getWindowSizeIncrement

      public double getWindowSizeIncrement()

    • getWindowSizeBackOff

      public double getWindowSizeBackOff()

    • setMaxThroughput

      public DynamicThrottlePolicy setMaxThroughput(double maxThroughput)

    • canSend

      public boolean canSend(Message message, int pendingCount)
      Description copied from interface: ThrottlePolicy
      Returns whether or not the given message can be sent according to the current state of this policy.
      Specified by:
      canSend in interface ThrottlePolicy
      Overrides:
      canSend in class StaticThrottlePolicy
      Parameters:
      message - the message to evaluate
      pendingCount - the current number of pending messages
      Returns:
      true to send the message

    • processMessage

      public void processMessage(Message message)
      Description copied from interface: ThrottlePolicy
      This method is called once for every message that was accepted by ThrottlePolicy.canSend(Message, int) and sent.
      Specified by:
      processMessage in interface ThrottlePolicy
      Overrides:
      processMessage in class StaticThrottlePolicy
      Parameters:
      message - the message being sent

    • processReply

      public void processReply(Reply reply)
      Description copied from interface: ThrottlePolicy
      This method is called once for every reply that is received.
      Specified by:
      processReply in interface ThrottlePolicy
      Overrides:
      processReply in class StaticThrottlePolicy
      Parameters:
      reply - the reply received

    • setEfficiencyThreshold

      public DynamicThrottlePolicy setEfficiencyThreshold(double efficiencyThreshold)
      Determines where on each latency level the attractor sits. 2 is at the very end, and makes this to *boom*. 0.2 is at the very start, and makes the algorithm more conservative. Probably fine to stay away from this.

    • setWindowSizeIncrement

      public DynamicThrottlePolicy setWindowSizeIncrement(double windowSizeIncrement)
      Sets the step size used when increasing window size.
      Parameters:
      windowSizeIncrement - the step size to set
      Returns:
      this, to allow chaining

    • setWindowSizeDecrementFactor

      public DynamicThrottlePolicy setWindowSizeDecrementFactor(double decrementFactor)
      Sets the relative step size when decreasing window size.
      Parameters:
      decrementFactor - the step size to set
      Returns:
      this, to allow chaining

    • setWindowSizeBackOff

      public DynamicThrottlePolicy setWindowSizeBackOff(double windowSizeBackOff)
      Sets the factor of window size to back off to when the algorithm determines that efficiency is not increasing. Capped to [0, 1]
      Parameters:
      windowSizeBackOff - the back off to set
      Returns:
      this, to allow chaining

    • setResizeRate

      public DynamicThrottlePolicy setResizeRate(double resizeRate)
      Sets the rate at which the window size is updated. The larger the value, the less responsive the resizing becomes. However, the smaller the value, the less accurate the measurements become. Capped to [2, )
      Parameters:
      resizeRate - the rate to set
      Returns:
      this, to allow chaining

    • setWeight

      public DynamicThrottlePolicy setWeight(double weight)
      Sets the weight for this client. The larger the value, the more resources will be allocated to this clients. Resources are shared between clients roughly proportionally to the set weights. Must be a positive number.
      Parameters:
      weight - the weight to set
      Returns:
      this, to allow chaining

    • setMaxWindowSize

      public DynamicThrottlePolicy setMaxWindowSize(double max)
      Sets the maximum number of pending operations allowed at any time, in order to avoid using too much resources.
      Parameters:
      max - the max to set
      Returns:
      this, to allow chaining

    • getMaxWindowSize

      public double getMaxWindowSize()
      Get the maximum number of pending operations allowed at any time.
      Returns:
      the maximum number of operations

    • setMinWindowSize

      public DynamicThrottlePolicy setMinWindowSize(double min)
      Sets the minimum number of pending operations allowed at any time, in order to keep a level of performance.
      Parameters:
      min - the min to set
      Returns:
      this, to allow chaining

    • getMinWindowSize

      public double getMinWindowSize()
      Get the minimum number of pending operations allowed at any time.
      Returns:
      the minimum number of operations

    • setMaxPendingCount

      public DynamicThrottlePolicy setMaxPendingCount(int maxCount)
      Description copied from class: StaticThrottlePolicy
      Sets the maximum number of pending messages allowed.
      Overrides:
      setMaxPendingCount in class StaticThrottlePolicy
      Parameters:
      maxCount - The max count
      Returns:
      this, to allow chaining

    • getMaxPendingCount

      public int getMaxPendingCount()
      Returns the maximum number of pending messages allowed.
      Overrides:
      getMaxPendingCount in class StaticThrottlePolicy
      Returns:
      the max limit