Package com.yahoo.search.query.rewrite.rewriters

Class GenericExpansionRewriter

java.lang.Object
com.yahoo.component.AbstractComponent
com.yahoo.component.chain.ChainedComponent
com.yahoo.processing.Processor
com.yahoo.search.Searcher
com.yahoo.search.query.rewrite.QueryRewriteSearcher
com.yahoo.search.query.rewrite.rewriters.GenericExpansionRewriter
All Implemented Interfaces:
com.yahoo.component.Component, com.yahoo.component.Deconstructable, Comparable<com.yahoo.component.Component>
public class GenericExpansionRewriter extends QueryRewriteSearcher
This rewriter would add rewrites to entities (e.g abbreviation, synonym, etc)
to boost precision - FSA dict: [normalized original query]\t[rewrite 1]\t[rewrite 2]\t[etc]
- Features:
RewritesAsUnitEquiv flag: add proximity boosted rewrites
PartialPhraseMatch flag: whether to match whole phrase or partial phrase
MaxRewrites flag: the maximum number of rewrites to be added
Author:
Karen Sze Wing Lee

  • Field Details

  • Constructor Details

    • GenericExpansionRewriter

      @Inject public GenericExpansionRewriter(com.yahoo.component.ComponentId id, com.yahoo.filedistribution.fileacquirer.FileAcquirer fileAcquirer, RewritesConfig config)
      Constructor for GenericExpansionRewriter. Load configs using default format

    • GenericExpansionRewriter

      public GenericExpansionRewriter(RewritesConfig config, HashMap<String,File> fileList)
      Constructor for GenericExpansionRewriter unit test. Load configs using default format

  • Method Details

    • configure

      public boolean configure(com.yahoo.filedistribution.fileacquirer.FileAcquirer fileAcquirer, RewritesConfig config, HashMap<String,File> fileList)
      Instance creation time config loading besides FSA. Create PhraseMatcher from FSA dict
      Specified by:
      configure in class QueryRewriteSearcher
      Parameters:
      fileAcquirer - Required param for retrieving file type config (see vespa's search container doc for more detail)
      config - Config from vespa-services.xml (see vespa's search container doc for more detail)
      fileList - pairs of file name and file handler for unit tests
      Returns:
      boolean true if loaded successfully, false otherwise

    • rewrite

      public HashMap<String,Object> rewrite(Query query, String dictKey) throws RuntimeException
      Main logic of rewriter
      - Retrieve rewrites from FSA dict
      - rewrite query using features that are enabled by user
      Specified by:
      rewrite in class QueryRewriteSearcher
      Parameters:
      query - Query object from searcher
      dictKey - the key passed from previous rewriter to be treated as "original query from user" For example, if previous is misspell rewriter, it would pass the corrected query as the "original query from user". For other rewriters which add variants, abbr, etc to the query, the original query should be passed as a key. This rewriter could still choose to ignore this key. This key is not the rewritten query itself. For example, if original query is (willl smith) and the rewritten query is (willl smith) OR (will smith) the key to be passed could be (will smith)
      Returns:
      HashMap which contains the key value pairs:
      - whether this query has been rewritten by this rewriter
      key: rewritten
      value: true or false
      - the key to be treated as "original query from user" in next rewriter downstream, for example, misspell rewriter would pass the corrected query as the "original query from user" to the next rewriter. For other rewriters which add variants, abbr, etc to the query, the original query should be passed as a key. This key is not necessarily consumed by the next rewriter. The next rewriter can still choose to ignore this key.
      key: newDictKey
      value: new dict key
      Throws:
      RuntimeException

    • getSkipRewriterIfRewritten

      public boolean getSkipRewriterIfRewritten()
      Get the flag which specifies whether this rewriter should be skipped if the query has been rewritten
      Specified by:
      getSkipRewriterIfRewritten in class QueryRewriteSearcher
      Returns:
      true if rewriter should be skipped, false otherwise

    • getRewriterName

      public String getRewriterName()
      Get the name of the rewriter
      Specified by:
      getRewriterName in class QueryRewriteSearcher
      Returns:
      Name of the rewriter

    • getDefaultFSAs

      public HashMap<String,String> getDefaultFSAs()
      Get default FSA dictionary names
      Specified by:
      getDefaultFSAs in class QueryRewriteSearcher
      Returns:
      Pair of FSA dictionary name and filename