com.yahoo.search.searchers

Class RateLimitingSearcher

java.lang.Object

com.yahoo.component.AbstractComponent

com.yahoo.component.chain.ChainedComponent

com.yahoo.processing.Processor

com.yahoo.search.Searcher

com.yahoo.search.searchers.RateLimitingSearcher

All Implemented Interfaces:

com.yahoo.component.Component, Comparable<com.yahoo.component.Component>

public class RateLimitingSearcher
extends Searcher

A simple rate limiter.

This takes these query parameter arguments:

• rate.id - (String) the id of the client from rate limiting perspective
• rate.cost - (Double) the cost Double of this query. This is read after executing the query and hence can be set by downstream searchers inspecting the result to allow differencing the cost of various queries. Default is 1.
• rate.quota - (Double) the cost per second a particular id is allowed to consume in this system.
• rate.idDimension - (String) the name of the rate-id dimension used when logging metrics. If this is not specified, the metric will be logged without dimensions.
• rate.dryRun - (Boolean) emit metrics on rejected requests but don't actually reject them

Whenever quota is exceeded for an id this searcher will reject queries from that id by returning a result containing a status 429 error.

If rate.id or rate.quota is not set in Query.properties this searcher will do nothing.

Metrics: This will emit the count metric requestsOverQuota with the dimension [rate.idDimension=rate.id] counting rejected requests.

Ordering: This searcher Provides rateLimiting

Author:

bratseth

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
private static class	RateLimitingSearcher.AvailableCapacity This keeps track of the current "capacity" (total cost) available to each client (rate id) across all threads.
private static class	RateLimitingSearcher.CapacityAllocation

Field Summary

Fields

Modifier and Type	Field and Description
private ThreadLocal<Map<String,Double>>	allocatedCapacity Capacity already allocated to this thread
private RateLimitingSearcher.AvailableCapacity	availableCapacity Shared capacity across all threads.
private double	capacityIncrement How much capacity to allocate to a thread each time it runs out.
static com.yahoo.processing.request.CompoundName	costKey
static com.yahoo.processing.request.CompoundName	dryRunKey
static com.yahoo.processing.request.CompoundName	idDimensionKey
static com.yahoo.processing.request.CompoundName	idKey
private int	nodeCount Used to divide quota by nodes.
private com.yahoo.metrics.simple.Counter	overQuotaCounter For emitting metrics
static com.yahoo.processing.request.CompoundName	quotaKey
static String	RATE_LIMITING Constant containing the name this Provides - "rateLimiting", for ordering constraints
private double	recheckForCapacityProbability How often to check for new capacity if we have run out
private static String	requestsOverQuotaMetricName

Fields inherited from class com.yahoo.component.AbstractComponent

isDeconstructable

Constructor Summary

Constructors

Constructor and Description
RateLimitingSearcher(RateLimitingConfig rateLimitingConfig, com.yahoo.cloud.config.ClusterInfoConfig clusterInfoConfig, com.yahoo.metrics.simple.MetricReceiver metric)
RateLimitingSearcher(RateLimitingConfig rateLimitingConfig, com.yahoo.cloud.config.ClusterInfoConfig clusterInfoConfig, com.yahoo.metrics.simple.MetricReceiver metric, Clock clock) For testing - allows injection of a timer to avoid depending on the system clock

Method Summary

Modifier and Type	Method and Description
private void	addAllocatedCapacity(String id, double newCapacity)
private com.yahoo.metrics.simple.Point	createContext(String dimensionName, String dimensionValue)
private double	getAllocatedCapacity(String id)
private void	requestCapacity(String id, double rate)
Result	search(Query query, Execution execution) Override this to implement your searcher.

Methods inherited from class com.yahoo.search.Searcher

ensureFilled, fill, getLogger, process, toString

Methods inherited from class com.yahoo.component.chain.ChainedComponent

getAnnotatedDependencies, getDefaultAnnotatedDependencies, getDependencies, initDependencies

Methods inherited from class com.yahoo.component.AbstractComponent

clone, compareTo, deconstruct, getClassName, getId, getIdString, hasInitializedId, initId, isDeconstructable, setIsDeconstructable

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

RATE_LIMITING

public static final String RATE_LIMITING

Constant containing the name this Provides - "rateLimiting", for ordering constraints

See Also:

Constant Field Values

idKey

public static final com.yahoo.processing.request.CompoundName idKey

costKey

public static final com.yahoo.processing.request.CompoundName costKey

quotaKey

public static final com.yahoo.processing.request.CompoundName quotaKey

idDimensionKey

public static final com.yahoo.processing.request.CompoundName idDimensionKey

dryRunKey

public static final com.yahoo.processing.request.CompoundName dryRunKey

requestsOverQuotaMetricName

private static final String requestsOverQuotaMetricName

See Also:

Constant Field Values

nodeCount

private final int nodeCount

Used to divide quota by nodes. Assumption: All nodes get the same share of traffic.

availableCapacity

private final RateLimitingSearcher.AvailableCapacity availableCapacity

Shared capacity across all threads. Each thread will ask for more capacity from here when they run out.

allocatedCapacity

private final ThreadLocal<Map<String,Double>> allocatedCapacity

Capacity already allocated to this thread

overQuotaCounter

private final com.yahoo.metrics.simple.Counter overQuotaCounter

For emitting metrics

capacityIncrement

private final double capacityIncrement

How much capacity to allocate to a thread each time it runs out. A higher value means less contention and less accuracy.

recheckForCapacityProbability

private final double recheckForCapacityProbability

How often to check for new capacity if we have run out

Constructor Detail

RateLimitingSearcher

@Inject
public RateLimitingSearcher(RateLimitingConfig rateLimitingConfig,
                                    com.yahoo.cloud.config.ClusterInfoConfig clusterInfoConfig,
                                    com.yahoo.metrics.simple.MetricReceiver metric)

RateLimitingSearcher

public RateLimitingSearcher(RateLimitingConfig rateLimitingConfig,
                            com.yahoo.cloud.config.ClusterInfoConfig clusterInfoConfig,
                            com.yahoo.metrics.simple.MetricReceiver metric,
                            Clock clock)

For testing - allows injection of a timer to avoid depending on the system clock

Method Detail

search

public Result search(Query query,
                     Execution execution)

Description copied from class: Searcher

Override this to implement your searcher.

Searcher implementation subclasses will, depending on their type of logic, do one of the following:

• Query processors: Access the query, then call execution.search and return the result
• Result processors: Call execution.search to get the result, access it and return
• Sources (which produces results): Create a result, add the desired hits and return it.
• Federators (which forwards the search to multiple subchains): Call search on the desired subchains in parallel and get the results. Combine the results to one and return it.
• Workflows: Call execution.search as many times as desired, using different queries. Eventually return a result.

Hits come in two kinds - concrete hits are actual content of the kind requested by the user, meta hits are hits which provides information about the collection of hits, on the query, the service and so on.

The query specifies a window into a larger result list that must be returned from the searcher through hits and offset; Searchers which returns list of hits in the top level in the result must return at least hits number of hits (or if impossible; all that are available), starting at the given offset. In addition, searchers are allowed to return any number of meta hits (although this number is expected to be low). For hits contained in nested hit groups, the concept of a window defined by hits and offset is not well defined and does not apply.

Error handling in searchers:

• Unexpected events: Throw any RuntimeException. This query will fail with the exception message, and the error will be logged
• Expected events: Create (new Result(Query, ErrorMessage) or add result.setErrorIfNoOtherErrors(ErrorMessage) an error message to the Result.
• Recoverable user errors: Add a FeedbackHit explaining the condition and how to correct it.

Specified by:

search in class Searcher

Parameters:

query - the query

Returns:

the result of making this query

createContext

private com.yahoo.metrics.simple.Point createContext(String dimensionName,
                                                     String dimensionValue)

getAllocatedCapacity

private double getAllocatedCapacity(String id)

addAllocatedCapacity

private void addAllocatedCapacity(String id,
                                  double newCapacity)

requestCapacity

private void requestCapacity(String id,
                             double rate)