Package com.yahoo.search

Class Searcher

java.lang.Object
com.yahoo.component.AbstractComponent
com.yahoo.component.chain.ChainedComponent
com.yahoo.processing.Processor
com.yahoo.search.Searcher
All Implemented Interfaces:
com.yahoo.component.Component, com.yahoo.component.Deconstructable, Comparable<com.yahoo.component.Component>
Direct Known Subclasses:
BlendingSearcher, BooleanSearcher, CacheControlSearcher, CJKSearcher, ClusterSearcher, CollapsePhraseSearcher, ConnectionControlSearcher, ContainerLatencySearcher, DefaultPositionSearcher, DocumentDb, DocumentSourceSearcher, ExampleSearcher, FieldCollapsingSearcher, FieldFiller, FieldFilter, FillSearcher, FlatteningSearcher, ForkingSearcher, GroupingExecutor, GroupingQueryParser, GroupingValidator, InputCheckingSearcher, JSONDebugSearcher, JuniperSearcher, LiteralBoostSearcher, LowercasingSearcher, MetricsSearcher, MinimalQueryInserter, MultipleResultsSearcher, NGramSearcher, NonPhrasingSearcher, NoRankingSearcher, NormalizingSearcher, PageTemplateSearcher, PhrasingSearcher, PingableSearcher, PosSearcher, QueryRewriteSearcher, QueryValidator, QuotingSearcher, RangeQueryOptimizer, RateLimitingSearcher, RecallSearcher, SearchChainDispatcherSearcher, SemanticSearcher, SortingDegrader, StatisticsSearcher, StemmingSearcher, UniqueGroupingSearcher, ValidateFuzzySearcher, ValidateMatchPhaseSearcher, ValidateNearestNeighborSearcher, ValidatePredicateSearcher, ValidateSortingSearcher, WandSearcher, WeakAndReplacementSearcher
public abstract class Searcher extends com.yahoo.processing.Processor
Superclass of all Components which produces Results in response to Queries by calling the search method.

Searchers are participants in chain of responsibility search chains where they passes the Queries downwards by synchroneously calling the next Searcher in the chain, and returns the Results back up as the response.

Any Searcher may

  • Do modifications to the Query before passing it on (a query rerwiter)
  • Do modifications to the Result before passing it on up, e.g removing altering, reorganizing or adding Hits (a result processor)
  • Pass the Query on to multiple other search chains, either in series (by creating a new Execution for each chain), or in parallel (by creating a AsyncExecution) (a federator)
  • Create a Result and pass it back up, either by calling some other node(s) to get the data, or by creating the Result from internal data (a source)
  • Pass some query on downwards multiple times, or in different ways, typically each time depending of the Result returned the last time (a workflow)

...or some combination of the above of course. Note that as Searchers work synchronously, any information can be retained on the stack in the Searcher from the Query is received until the Result is returned simply by declaring variables for the data in the search method (or whatever it calls), and for the same reason workflows are implemented as Java code. However, searchers are executed by many threads, for different Queries, in parallell, so any mutable data shared between queries (and hence stored as instance members must be accessed multithread safely. In many cases, shared data can simply be instantiated in the constructor and used in read-only mode afterwards

Searcher lifecycle: A searcher has a simple life-cycle:

  • Construction: While a constructor is running. A searcher is handed its id and configuration (if any) in the constructor. During construction, the searcher should build any in-memory structures needed. A new instance of the searcher will be created when the configuration is changed. Constructors are called with this priority:
    • The constructor taking a ComponentId, followed by the highest number of config classes (subclasses of ConfigInstance) as arguments.
    • The constructor taking a string id followed by the highest number of config classes as arguments.
    • The constructor taking only the highest number of config classes as arguments.
    • The constructor taking a ComponentId as the only argument
    • The constructor taking a string id as the only argument
    • The default (no-argument) constructor.
    If none of these constructors are declared, searcher construction will fail.
  • In service: After the constructor has returned. In this phase, searcher service methods are called at any time by multiple threads in parallel. Implementations should avoid synchronization and access to volatiles as much as possible by keeping data structures build in construction read-only.
  • Deconstruction: While deconstruct is running. All Searcher service method calls have completed when this method is called. When it returns, the searcher will be eligible for garbage collection.
Author:
bratseth

  • Field Summary

    Fields inherited from class com.yahoo.component.AbstractComponent

    isDeconstructable

  • Constructor Summary

    Constructors
    Constructor
    Description
    Searcher()
     
    Searcher(com.yahoo.component.ComponentId id)
    Creates a searcher from an id

  • Method Summary

    Modifier and Type
    Method
    Description
    final void
    ensureFilled(Result result, String summaryClass, Execution execution)
    Fills the result if it is not already filled for the given summary class.
    void
    fill(Result result, String summaryClass, Execution execution)
    Fill hit properties with data using the given summary class.
    protected Logger
    getLogger()
    Returns a logger unique for the instance subclass
    final com.yahoo.processing.Response
    process(com.yahoo.processing.Request request, com.yahoo.processing.execution.Execution execution)
    Use the search method in Searcher processors.
    abstract Result
    search(Query query, Execution execution)
    Override this to implement your searcher.
    String
    toString()
    Returns "searcher 'getId()'"

    Methods inherited from class com.yahoo.component.chain.ChainedComponent

    getAnnotatedDependencies, getDefaultAnnotatedDependencies, getDependencies, initDependencies

    Methods inherited from class com.yahoo.component.AbstractComponent

    clone, compareTo, deconstruct, getClassName, getId, getIdString, hasInitializedId, initId, isDeconstructable, setIsDeconstructable

    Methods inherited from class java.lang.Object

    equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

  • Constructor Details

    • Searcher

      public Searcher()

    • Searcher

      public Searcher(com.yahoo.component.ComponentId id)
      Creates a searcher from an id

  • Method Details

    • search

      public abstract Result search(Query query, Execution execution)
      Override this to implement your searcher.

      Searcher implementation subclasses will, depending on their type of logic, do one of the following:

      • Query processors: Access the query, then call execution.search and return the result
      • Result processors: Call execution.search to get the result, access it and return
      • Sources (which produces results): Create a result, add the desired hits and return it.
      • Federators (which forwards the search to multiple subchains): Call search on the desired subchains in parallel and get the results. Combine the results to one and return it.
      • Workflows: Call execution.search as many times as desired, using different queries. Eventually return a result.

      Hits come in two kinds - concrete hits are actual content of the kind requested by the user, meta hits are hits which provides information about the collection of hits, on the query, the service and so on.

      The query specifies a window into a larger result list that must be returned from the searcher through hits and offset; Searchers which returns list of hits in the top level in the result must return at least hits number of hits (or if impossible; all that are available), starting at the given offset. In addition, searchers are allowed to return any number of meta hits (although this number is expected to be low). For hits contained in nested hit groups, the concept of a window defined by hits and offset is not well defined and does not apply.

      Error handling in searchers:

      • Unexpected events: Throw any RuntimeException. This query will fail with the exception message, and the error will be logged
      • Expected events: Create (new Result(Query, ErrorMessage) or add result.setErrorIfNoOtherErrors(ErrorMessage) an error message to the Result.
      • Recoverable user errors: Add a FeedbackHit explaining the condition and how to correct it.
      Parameters:
      query - the query
      Returns:
      the result of making this query

    • process

      public final com.yahoo.processing.Response process(com.yahoo.processing.Request request, com.yahoo.processing.execution.Execution execution)
      Use the search method in Searcher processors. This forwards to it.
      Specified by:
      process in class com.yahoo.processing.Processor

    • fill

      public void fill(Result result, String summaryClass, Execution execution)
      Fill hit properties with data using the given summary class. Calling this on already filled results has no cost.

      This needs to be overridden by federating searchers to contact search sources again by propagating the fill call down through the search chain, and by source searchers which talks to fill capable backends to request the data to be filled. Other searchers do not need to override this.

      Parameters:
      result - the result to fill
      summaryClass - the name of the collection of fields to fetch the values of, or null to use the default

    • ensureFilled

      public final void ensureFilled(Result result, String summaryClass, Execution execution)
      Fills the result if it is not already filled for the given summary class. See the fill method.

    • getLogger

      protected Logger getLogger()
      Returns a logger unique for the instance subclass

    • toString

      public String toString()
      Returns "searcher 'getId()'"
      Overrides:
      toString in class com.yahoo.component.AbstractComponent