Package com.yahoo.search.cluster

Class ClusterSearcher<T>

java.lang.Object
com.yahoo.component.AbstractComponent
com.yahoo.component.chain.ChainedComponent
com.yahoo.processing.Processor
com.yahoo.search.Searcher
com.yahoo.search.cluster.PingableSearcher
com.yahoo.search.cluster.ClusterSearcher<T>
All Implemented Interfaces:
com.yahoo.component.Component, com.yahoo.component.Deconstructable, NodeManager<T>, Comparable<com.yahoo.component.Component>
public abstract class ClusterSearcher<T> extends PingableSearcher implements NodeManager<T>
Implements clustering (failover and load balancing) over a set of client connections to a homogenuos cluster of nodes. Searchers which wants to make clustered connections to some service should use this.

This replaces the usual searcher methods by ones which have the same contract and semantics but which takes an additional parameter which is the Connection selected by the cluster searcher which the method should use. Overrides of these connection methods must not call the super methods to pass on but must use the methods on execution.

The type argument is the class (of any type) representing the connections. The connection objects should implement a good toString to ease diagnostics.

Author:
bratseth, Arne Bergene Fossaa

  • Field Summary

    Fields inherited from class com.yahoo.component.AbstractComponent

    isDeconstructable

  • Constructor Summary

    Constructors
    Modifier
    Constructor
    Description
     
    ClusterSearcher(com.yahoo.component.ComponentId id, List<T> connections, boolean internal)
    Creates a new cluster searcher
     
    ClusterSearcher(com.yahoo.component.ComponentId id, List<T> connections, Hasher<T> hasher, boolean internal)
     
    protected
    ClusterSearcher(com.yahoo.component.ComponentId id, List<T> connections, Hasher<T> hasher, boolean internal, boolean startPingThread)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    deconstruct()
     
    void
    failed(T node)
    NodeManager method, called from ClusterMonitor
    final void
    fill(Result result, String summaryClass, Execution execution)
    Fill hit properties with data using the given summary class.
    protected abstract void
    fill(Result result, String summaryClass, Execution execution, T connection)
    Perform the fill against the given connection.
    protected T
    getFirstConnection(Hasher.NodeList<T> nodes, int code, int trynum, Query query)
     
    Hasher<T>
    getHasher()
    Returns the hasher used internally in this.
    ClusterMonitor<T>
    getMonitor()
    Returns the monitor of these nodes
    protected void
    log(Level level, Object... objects)
     
    String
    name()
    Name to identify Nodemanager
    protected abstract Pong
    ping(Ping ping, T connection)
    Pings this connection.
    final void
    ping(ClusterMonitor<T> clusterMonitor, T p, Executor executor)
    Pinging a node, called from ClusterMonitor
    protected Result
    robustSearch(Query query, Execution execution, T connection)
    Call search(Query, Execution, T) and handle any exceptions returned which we do not want to propagate upwards By default this catches all runtime exceptions and puts them into the result
    final Result
    search(Query query, Execution execution)
    Override this to implement your searcher.
    protected Result
    search(Query query, Execution execution, ErrorMessage message)
    This is called (instead of search(query, execution, connection) to handle searches where no (suitable) backend was available.
    protected abstract Result
    search(Query query, Execution execution, T connection)
    Perform the search against the given connection.
    protected boolean
    shouldRetry(Query query, Result result)
    Returns whether this query and result should be retried against another connection if possible.
    protected boolean
    timedOut(Query query)
    Returns true if this query has timed out now
    void
    working(T node)
    NodeManager method, called from ClusterMonitor

    Methods inherited from class com.yahoo.search.cluster.PingableSearcher

    ping

    Methods inherited from class com.yahoo.search.Searcher

    ensureFilled, getLogger, process, toString

    Methods inherited from class com.yahoo.component.chain.ChainedComponent

    getAnnotatedDependencies, getDefaultAnnotatedDependencies, getDependencies, initDependencies

    Methods inherited from class com.yahoo.component.AbstractComponent

    clone, compareTo, getClassName, getId, getIdString, hasInitializedId, initId, isDeconstructable, setIsDeconstructable

    Methods inherited from class java.lang.Object

    equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

    Methods inherited from interface com.yahoo.search.cluster.NodeManager

    pingIterationCompleted

  • Constructor Details

    • ClusterSearcher

      public ClusterSearcher(com.yahoo.component.ComponentId id, List<T> connections, boolean internal)
      Creates a new cluster searcher
      Parameters:
      id - the id of this searcher
      connections - the connections of the cluster
      internal - whether or not this cluster is internal (part of the same installation)

    • ClusterSearcher

      public ClusterSearcher(com.yahoo.component.ComponentId id, List<T> connections, Hasher<T> hasher, boolean internal)

    • ClusterSearcher

      protected ClusterSearcher(com.yahoo.component.ComponentId id, List<T> connections, Hasher<T> hasher, boolean internal, boolean startPingThread)

  • Method Details

    • name

      public String name()
      Description copied from interface: NodeManager
      Name to identify Nodemanager
      Specified by:
      name in interface NodeManager<T>

    • ping

      public final void ping(ClusterMonitor<T> clusterMonitor, T p, Executor executor)
      Pinging a node, called from ClusterMonitor
      Specified by:
      ping in interface NodeManager<T>

    • ping

      protected abstract Pong ping(Ping ping, T connection)
      Pings this connection. Pings may be sent "out of band" at any time by the monitoring subsystem to determine the status of this connection. If the ping fails, it is ok both to set an error in the pong or to throw an exception.

    • getFirstConnection

      protected T getFirstConnection(Hasher.NodeList<T> nodes, int code, int trynum, Query query)

    • search

      public final Result search(Query query, Execution execution)
      Description copied from class: Searcher
      Override this to implement your searcher.

      Searcher implementation subclasses will, depending on their type of logic, do one of the following:

      • Query processors: Access the query, then call execution.search and return the result
      • Result processors: Call execution.search to get the result, access it and return
      • Sources (which produces results): Create a result, add the desired hits and return it.
      • Federators (which forwards the search to multiple subchains): Call search on the desired subchains in parallel and get the results. Combine the results to one and return it.
      • Workflows: Call execution.search as many times as desired, using different queries. Eventually return a result.

      Hits come in two kinds - concrete hits are actual content of the kind requested by the user, meta hits are hits which provides information about the collection of hits, on the query, the service and so on.

      The query specifies a window into a larger result list that must be returned from the searcher through hits and offset; Searchers which returns list of hits in the top level in the result must return at least hits number of hits (or if impossible; all that are available), starting at the given offset. In addition, searchers are allowed to return any number of meta hits (although this number is expected to be low). For hits contained in nested hit groups, the concept of a window defined by hits and offset is not well defined and does not apply.

      Error handling in searchers:

      • Unexpected events: Throw any RuntimeException. This query will fail with the exception message, and the error will be logged
      • Expected events: Create (new Result(Query, ErrorMessage) or add result.setErrorIfNoOtherErrors(ErrorMessage) an error message to the Result.
      • Recoverable user errors: Add a FeedbackHit explaining the condition and how to correct it.
      Specified by:
      search in class Searcher
      Parameters:
      query - the query
      Returns:
      the result of making this query

    • shouldRetry

      protected boolean shouldRetry(Query query, Result result)
      Returns whether this query and result should be retried against another connection if possible. This default implementation returns true if the result contains some error.

    • search

      protected Result search(Query query, Execution execution, ErrorMessage message)
      This is called (instead of search(query, execution, connection) to handle searches where no (suitable) backend was available. The default implementation returns an error result.

    • robustSearch

      protected Result robustSearch(Query query, Execution execution, T connection)
      Call search(Query, Execution, T) and handle any exceptions returned which we do not want to propagate upwards By default this catches all runtime exceptions and puts them into the result

    • search

      protected abstract Result search(Query query, Execution execution, T connection)
      Perform the search against the given connection. Return a result containing an error or throw an exception on failures.

    • fill

      public final void fill(Result result, String summaryClass, Execution execution)
      Description copied from class: Searcher
      Fill hit properties with data using the given summary class. Calling this on already filled results has no cost.

      This needs to be overridden by federating searchers to contact search sources again by propagating the fill call down through the search chain, and by source searchers which talks to fill capable backends to request the data to be filled. Other searchers do not need to override this.

      Overrides:
      fill in class Searcher
      Parameters:
      result - the result to fill
      summaryClass - the name of the collection of fields to fetch the values of, or null to use the default

    • fill

      protected abstract void fill(Result result, String summaryClass, Execution execution, T connection)
      Perform the fill against the given connection. Add an error to the result or throw an exception on failures.

    • working

      public void working(T node)
      NodeManager method, called from ClusterMonitor
      Specified by:
      working in interface NodeManager<T>

    • failed

      public void failed(T node)
      NodeManager method, called from ClusterMonitor
      Specified by:
      failed in interface NodeManager<T>

    • getHasher

      public Hasher<T> getHasher()
      Returns the hasher used internally in this. Do not mutate this hasher while in use.

    • getMonitor

      public ClusterMonitor<T> getMonitor()
      Returns the monitor of these nodes

    • timedOut

      protected boolean timedOut(Query query)
      Returns true if this query has timed out now

    • log

      protected void log(Level level, Object... objects)

    • deconstruct

      public void deconstruct()
      Specified by:
      deconstruct in interface com.yahoo.component.Deconstructable
      Overrides:
      deconstruct in class com.yahoo.component.AbstractComponent