Package com.yahoo.messagebus

Class DynamicThrottlePolicy

  • All Implemented Interfaces:
    ThrottlePolicy
    public class DynamicThrottlePolicy
extends StaticThrottlePolicy
    This is an implementation of the ThrottlePolicy that offers dynamic limits to the number of pending messages a SourceSession is allowed to have. Pending means the number of sent messages that have not been replied to yet.

    The algorithm works by increasing the number of messages allowed to be pending, the window size, until this no longer increases throughput. At this point, the algorithm is driven by synthetic attraction towards latencies which satisfy log10(1 / latency) % 1 = e, for some constant 0 < e < 1. Weird? Most certainly!

    The effect is that the window size the algorithm produces, for a saturated ideal server, has a level for each power of ten with an attractor the window size tends towards while on this level, determined by the e above. The setEfficiencyThreshold(double) determines the e constant. When e is set so the attractor is close to the start of the interval, this has an inhibiting effect on the algorithm, and it is basically reduced to "increase window size until this no longer increases throughput enough that it defeats the random noise". As the attractor moves towards the end of the intervals, the algorithm becomes increasingly eager in increasing its window size past what it measures as effective — if moved to the very end of the interval, the algorithm explodes. The default setting has the attractor at log10(2) of the way from start to end of these levels.

    Because the algorithm stops increasing the window size when increases in throughput drown in random variations, the setWindowSizeIncrement(double) directly influences the efficient work domain of the algorithm. With the default setting of 20, it seems to struggle to increase past window sizes of a couple thousand. Using a static increment (and a backoff factor) seems to be necessary to effectively balance the load different, competing policies are allowed to produce.

    When there are multiple policies that compete against each other, they will increase load until saturating the server. If keeping all other policies but one fixed, this one policy would still see an increase in throughput with increased window size, even with a saturated server, as it would be given a greater share of the server's resources. However, since all algorithms adjust their windows concurrently, they will all reduce the throughput of the other algorithms. The result is that the commonly see the server as saturated, and commonly reach the behaviour where some increases in window size lead to measurable throughput gains, while others don't.

    Now the weighting (setWeight(double) comes into play: with equals weights, two algorithms would have a break-even between being governed by the attractors (above), which eventually limits window size, and increases due to positive measurements, at the same point along the window size axis. With smaller weights, i.e., smaller increases to window size, this break-even occurs where the curve is steeper, i.e., where the client has a smaller share of the server. Thus, competing algorithms with different weights end up with a resource distribution roughly proportional to weight.

    Author:
    Simon Thoresen Hult, jonmv

    • Constructor Detail

      • DynamicThrottlePolicy

        public DynamicThrottlePolicy()
        Constructs a new instance of this policy and sets the appropriate default values of member data.

      • DynamicThrottlePolicy

        public DynamicThrottlePolicy​(com.yahoo.concurrent.Timer timer)
        Constructs a new instance of this class using the given clock to calculate efficiency.
        Parameters:
        timer - the timer to use

    • Method Detail

      • getWindowSizeIncrement

        public double getWindowSizeIncrement()

      • getWindowSizeBackOff

        public double getWindowSizeBackOff()

      • canSend

        public boolean canSend​(Message message,
                       int pendingCount)
        Description copied from interface: ThrottlePolicy
        Returns whether or not the given message can be sent according to the current state of this policy.
        Specified by:
        canSend in interface ThrottlePolicy
        Overrides:
        canSend in class StaticThrottlePolicy
        Parameters:
        message - the message to evaluate
        pendingCount - the current number of pending messages
        Returns:
        true to send the message

      • setEfficiencyThreshold

        public DynamicThrottlePolicy setEfficiencyThreshold​(double efficiencyThreshold)
        Determines where on each latency level the attractor sits. 2 is at the very end, and makes this to *boom*. 0.2 is at the very start, and makes the algorithm more conservative. Probably fine to stay away from this.

      • setWindowSizeIncrement

        public DynamicThrottlePolicy setWindowSizeIncrement​(double windowSizeIncrement)
        Sets the step size used when increasing window size.
        Parameters:
        windowSizeIncrement - the step size to set
        Returns:
        this, to allow chaining

      • setWindowSizeDecrementFactor

        public DynamicThrottlePolicy setWindowSizeDecrementFactor​(double decrementFactor)
        Sets the relative step size when decreasing window size.
        Parameters:
        decrementFactor - the step size to set
        Returns:
        this, to allow chaining

      • setWindowSizeBackOff

        public DynamicThrottlePolicy setWindowSizeBackOff​(double windowSizeBackOff)
        Sets the factor of window size to back off to when the algorithm determines that efficiency is not increasing. Capped to [0, 1]
        Parameters:
        windowSizeBackOff - the back off to set
        Returns:
        this, to allow chaining

      • setResizeRate

        public DynamicThrottlePolicy setResizeRate​(double resizeRate)
        Sets the rate at which the window size is updated. The larger the value, the less responsive the resizing becomes. However, the smaller the value, the less accurate the measurements become. Capped to [2, )
        Parameters:
        resizeRate - the rate to set
        Returns:
        this, to allow chaining

      • setWeight

        public DynamicThrottlePolicy setWeight​(double weight)
        Sets the weight for this client. The larger the value, the more resources will be allocated to this clients. Resources are shared between clients roughly proportionally to the set weights. Must be a positive number.
        Parameters:
        weight - the weight to set
        Returns:
        this, to allow chaining

      • setMaxWindowSize

        public DynamicThrottlePolicy setMaxWindowSize​(double max)
        Sets the maximum number of pending operations allowed at any time, in order to avoid using too much resources.
        Parameters:
        max - the max to set
        Returns:
        this, to allow chaining

      • getMaxWindowSize

        public double getMaxWindowSize()
        Get the maximum number of pending operations allowed at any time.
        Returns:
        the maximum number of operations

      • setMinWindowSize

        public DynamicThrottlePolicy setMinWindowSize​(double min)
        Sets the minimum number of pending operations allowed at any time, in order to keep a level of performance.
        Parameters:
        min - the min to set
        Returns:
        this, to allow chaining

      • getMinWindowSize

        public double getMinWindowSize()
        Get the minimum number of pending operations allowed at any time.
        Returns:
        the minimum number of operations

      • getMaxPendingCount

        public int getMaxPendingCount()
        Returns the maximum number of pending messages allowed.
        Overrides:
        getMaxPendingCount in class StaticThrottlePolicy
        Returns:
        the max limit