com.yahoo.prelude.statistics

Class StatisticsSearcher

java.lang.Object

com.yahoo.component.AbstractComponent

com.yahoo.component.chain.ChainedComponent

com.yahoo.processing.Processor

com.yahoo.search.Searcher

com.yahoo.prelude.statistics.StatisticsSearcher

All Implemented Interfaces:

com.yahoo.component.Component, Comparable<com.yahoo.component.Component>

public class StatisticsSearcher
extends Searcher

A searcher to gather statistics such as queries completed and query latency. There may be more than 1 StatisticsSearcher in the Searcher chain, each identified by a Searcher ID. The statistics accumulated by all StatisticsSearchers are stored in the singleton StatisticsManager object.

TODO: Fix events to handle more than one of these searchers properly.

Author:

Gene Meyers, Steinar Knutsen, bergum

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
private class	StatisticsSearcher.ActivitySampler
private class	StatisticsSearcher.PeakQpsReporter

Field Summary

Fields

Modifier and Type	Field and Description
private static String	ACTIVE_QUERIES_METRIC
private com.yahoo.statistics.Value	activeQueries
private Map<String,com.yahoo.jdisc.Metric.Context>	chainContexts
private static String	EMPTY_RESULTS_METRIC
private com.yahoo.statistics.Counter	emptyResults
private static String	FAILED_QUERIES_METRIC
private com.yahoo.statistics.Counter	failedQueries
private static String	HITS_PER_QUERY_METRIC
private com.yahoo.statistics.Value	hitsPerQuery
private com.yahoo.statistics.Counter	illegalQueries
private static String	MAX_QUERY_LATENCY_METRIC
private com.yahoo.statistics.Value	maxQueryLatency
private static String	MEAN_QUERY_LATENCY_METRIC
private com.yahoo.jdisc.Metric	metric
private com.yahoo.statistics.Counter	nullQueries
private static String	PEAK_QPS_METRIC
private com.yahoo.statistics.Value	peakQPS
private StatisticsSearcher.PeakQpsReporter	peakQpsReporter
private com.yahoo.statistics.Counter	queries
private static String	QUERIES_METRIC
private static String	QUERY_LATENCY_METRIC
private com.yahoo.statistics.Value	queryLatency
private com.yahoo.statistics.Value	queryLatencyBuckets
private Timer	scheduler
private Map<String,com.yahoo.jdisc.Metric.Context>	statePageOnlyContexts
private static String	TOTALHITS_PER_QUERY_METRIC

Fields inherited from class com.yahoo.component.AbstractComponent

isDeconstructable

Constructor Summary

Constructors

Constructor and Description
StatisticsSearcher(com.yahoo.jdisc.Metric metric)
StatisticsSearcher(com.yahoo.statistics.Statistics manager, com.yahoo.jdisc.Metric metric, com.yahoo.metrics.simple.MetricReceiver metricReceiver)

Method Summary

Modifier and Type	Method and Description
private void	addLatency(long latency, com.yahoo.jdisc.Metric.Context metricContext)
void	deconstruct()
private com.yahoo.jdisc.Metric.Context	getChainMetricContext(String chainName)
private com.yahoo.jdisc.Metric.Context	getDimensions(String source, Result r, Execution execution)
String	getMyID()
private void	incrementStatePageOnlyErrors(Result result, Execution execution) Creates error metric for StateHandler only.
private void	incrErrorCount(Result result, com.yahoo.jdisc.Metric.Context metricContext)
private void	incrQueryCount(com.yahoo.jdisc.Metric.Context metricContext)
private void	initEvents(com.yahoo.statistics.Statistics manager, com.yahoo.metrics.simple.MetricReceiver metricReceiver)
private void	logQuery(Query query)
private void	qps(com.yahoo.jdisc.Metric.Context metricContext)
Result	search(Query query, Execution execution) Generate statistics for the query passing through this Searcher 1) Add 1 to total query count 2) Add response time to total response time (time from entry to return) 3) .....

Methods inherited from class com.yahoo.search.Searcher

ensureFilled, fill, getLogger, process, toString

Methods inherited from class com.yahoo.component.chain.ChainedComponent

getAnnotatedDependencies, getDefaultAnnotatedDependencies, getDependencies, initDependencies

Methods inherited from class com.yahoo.component.AbstractComponent

clone, compareTo, getClassName, getId, getIdString, hasInitializedId, initId, isDeconstructable, setIsDeconstructable

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

MAX_QUERY_LATENCY_METRIC

private static final String MAX_QUERY_LATENCY_METRIC

See Also:

Constant Field Values

EMPTY_RESULTS_METRIC

private static final String EMPTY_RESULTS_METRIC

See Also:

Constant Field Values

HITS_PER_QUERY_METRIC

private static final String HITS_PER_QUERY_METRIC

See Also:

Constant Field Values

TOTALHITS_PER_QUERY_METRIC

private static final String TOTALHITS_PER_QUERY_METRIC

See Also:

Constant Field Values

FAILED_QUERIES_METRIC

private static final String FAILED_QUERIES_METRIC

See Also:

Constant Field Values

MEAN_QUERY_LATENCY_METRIC

private static final String MEAN_QUERY_LATENCY_METRIC

See Also:

Constant Field Values

QUERY_LATENCY_METRIC

private static final String QUERY_LATENCY_METRIC

See Also:

Constant Field Values

QUERIES_METRIC

private static final String QUERIES_METRIC

See Also:

Constant Field Values

ACTIVE_QUERIES_METRIC

private static final String ACTIVE_QUERIES_METRIC

See Also:

Constant Field Values

PEAK_QPS_METRIC

private static final String PEAK_QPS_METRIC

See Also:

Constant Field Values

queries

private com.yahoo.statistics.Counter queries

failedQueries

private com.yahoo.statistics.Counter failedQueries

nullQueries

private com.yahoo.statistics.Counter nullQueries

illegalQueries

private com.yahoo.statistics.Counter illegalQueries

queryLatency

private com.yahoo.statistics.Value queryLatency

queryLatencyBuckets

private com.yahoo.statistics.Value queryLatencyBuckets

maxQueryLatency

private com.yahoo.statistics.Value maxQueryLatency

activeQueries

private com.yahoo.statistics.Value activeQueries

peakQPS

private com.yahoo.statistics.Value peakQPS

emptyResults

private com.yahoo.statistics.Counter emptyResults

hitsPerQuery

private com.yahoo.statistics.Value hitsPerQuery

peakQpsReporter

private final StatisticsSearcher.PeakQpsReporter peakQpsReporter

metric

private com.yahoo.jdisc.Metric metric

chainContexts

private Map<String,com.yahoo.jdisc.Metric.Context> chainContexts

statePageOnlyContexts

private Map<String,com.yahoo.jdisc.Metric.Context> statePageOnlyContexts

scheduler

private Timer scheduler

Constructor Detail

StatisticsSearcher

StatisticsSearcher(com.yahoo.jdisc.Metric metric)

StatisticsSearcher

public StatisticsSearcher(com.yahoo.statistics.Statistics manager,
                          com.yahoo.jdisc.Metric metric,
                          com.yahoo.metrics.simple.MetricReceiver metricReceiver)

Method Detail

initEvents

private void initEvents(com.yahoo.statistics.Statistics manager,
                        com.yahoo.metrics.simple.MetricReceiver metricReceiver)

deconstruct

public void deconstruct()

Overrides:

deconstruct in class com.yahoo.component.AbstractComponent

getMyID

public String getMyID()

qps

private void qps(com.yahoo.jdisc.Metric.Context metricContext)

getChainMetricContext

private com.yahoo.jdisc.Metric.Context getChainMetricContext(String chainName)

search

public Result search(Query query,
                     Execution execution)

Generate statistics for the query passing through this Searcher 1) Add 1 to total query count 2) Add response time to total response time (time from entry to return) 3) .....

Specified by:

search in class Searcher

Parameters:

query - the query

Returns:

the result of making this query

logQuery

private void logQuery(Query query)

addLatency

private void addLatency(long latency,
                        com.yahoo.jdisc.Metric.Context metricContext)

incrQueryCount

private void incrQueryCount(com.yahoo.jdisc.Metric.Context metricContext)

incrErrorCount

private void incrErrorCount(Result result,
                            com.yahoo.jdisc.Metric.Context metricContext)

incrementStatePageOnlyErrors

private void incrementStatePageOnlyErrors(Result result,
                                          Execution execution)

Creates error metric for StateHandler only. These metrics are only exposed on /state/v1/metrics page and not forwarded to the log file.

Parameters:

result - The result to check for errors

getDimensions

private com.yahoo.jdisc.Metric.Context getDimensions(String source,
                                                     Result r,
                                                     Execution execution)