com.yahoo.search.query.rewrite

Class QueryRewriteSearcher

java.lang.Object

com.yahoo.component.AbstractComponent

com.yahoo.component.chain.ChainedComponent

com.yahoo.processing.Processor

com.yahoo.search.Searcher

com.yahoo.search.query.rewrite.QueryRewriteSearcher

All Implemented Interfaces:

com.yahoo.component.Component, Comparable<com.yahoo.component.Component>

Direct Known Subclasses:

GenericExpansionRewriter, MisspellRewriter, NameRewriter

public abstract class QueryRewriteSearcher
extends Searcher

A template class for all rewriters

All rewriters extending this class would need to implement the rewrite method which contains the rewriter's main logic, getSkipRewriterIfRewritten method which indicates whether this rewriter should be skipped if the query has been rewritten, getRewriterName method which returns the name of the rewriter used in query profile, configure method which contains any instance creation time configuration besides the default FSA loading, and getDefaultDicts method which return the pair of dictionary name and filename.

Common rewrite features are in RewriterFeatures.java. Common rewriter utils are in RewriterUtils.java.

Author:

Karen Sze Wing Lee

Field Summary

Fields

Modifier and Type	Field and Description
private boolean	isOk
protected Logger	logger
protected HashMap<String,Object>	rewriterDicts

Fields inherited from class com.yahoo.component.AbstractComponent

isDeconstructable

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	QueryRewriteSearcher() Empty constructor for unit test.
protected	QueryRewriteSearcher(com.yahoo.component.ComponentId id) Empty constructor.
protected	QueryRewriteSearcher(com.yahoo.component.ComponentId id, com.yahoo.filedistribution.fileacquirer.FileAcquirer fileAcquirer, RewritesConfig config) Constructor for this rewriter.
protected	QueryRewriteSearcher(RewritesConfig config, HashMap<String,File> fileList) Constructor for unit test.

Method Summary

Modifier and Type	Method and Description
abstract boolean	configure(com.yahoo.filedistribution.fileacquirer.FileAcquirer fileAcquirer, RewritesConfig config, HashMap<String,File> fileList) Perform instance creation time configuration besides the default FSA loading
abstract HashMap<String,String>	getDefaultFSAs() Get default FSA dictionary names
protected String	getQPConfig(Query query, String paramName) Get config parameter value set in query profile
protected String	getRewriteFromFSA(Query query, String dictName, String key) Retrieve rewrite from FSA given the original query
abstract String	getRewriterName() Retrieve rewriter name It should match the name used in query profile
protected abstract boolean	getSkipRewriterIfRewritten() Check whether rewriter should be skipped if the query has been rewritten by other rewriter
private boolean	loadFSADicts(com.yahoo.filedistribution.fileacquirer.FileAcquirer fileAcquirer, RewritesConfig config, HashMap<String,File> fileList) Load the dicts specified in vespa-services.xml
protected abstract HashMap<String,Object>	rewrite(Query query, String dictKey) Perform the main rewrite logic
Result	search(Query query, Execution execution) Perform main rewrite logics for this searcher - Skip to next rewriter if query is previously rewritten and getSkipRewriterIfRewritten() is true for this rewriter - Execute rewriter's main rewrite logic - Pass to the next rewriter the query to be used for dictionary retrieval

Methods inherited from class com.yahoo.search.Searcher

ensureFilled, fill, getLogger, process, toString

Methods inherited from class com.yahoo.component.chain.ChainedComponent

getAnnotatedDependencies, getDefaultAnnotatedDependencies, getDependencies, initDependencies

Methods inherited from class com.yahoo.component.AbstractComponent

clone, compareTo, deconstruct, getClassName, getId, getIdString, hasInitializedId, initId, isDeconstructable, setIsDeconstructable

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

isOk

private boolean isOk

logger

protected final Logger logger

rewriterDicts

protected HashMap<String,Object> rewriterDicts

Constructor Detail

QueryRewriteSearcher

@Inject
protected QueryRewriteSearcher(com.yahoo.component.ComponentId id,
                                       com.yahoo.filedistribution.fileacquirer.FileAcquirer fileAcquirer,
                                       RewritesConfig config)

Constructor for this rewriter. Prepare the data needed by the rewriter

Parameters:

id - Component ID (see vespa's search container doc for more detail)

fileAcquirer - Required param for retrieving file type config (see vespa's search container doc for more detail)

config - Config from vespa-services.xml (see vespa's search container doc for more detail)

QueryRewriteSearcher

protected QueryRewriteSearcher(RewritesConfig config,
                               HashMap<String,File> fileList)

Constructor for unit test. Prepare the data needed by the rewriter

Parameters:

config - Config from vespa-services.xml (see vespa's search container doc for more detail)

fileList - pairs of file name and file handler for unit tests

QueryRewriteSearcher

protected QueryRewriteSearcher(com.yahoo.component.ComponentId id)

Empty constructor. Do nothing at instance creation time

QueryRewriteSearcher

protected QueryRewriteSearcher()

Empty constructor for unit test. Do nothing at instance creation time

Method Detail

loadFSADicts

private boolean loadFSADicts(com.yahoo.filedistribution.fileacquirer.FileAcquirer fileAcquirer,
                             RewritesConfig config,
                             HashMap<String,File> fileList)
                      throws RuntimeException

Load the dicts specified in vespa-services.xml

Parameters:

fileAcquirer - Required param for retrieving file type config (see vespa's search container doc for more detail)

config - Config from vespa-services.xml (see vespa's search container doc for more detail)

fileList - pairs of file name and file handler for unit tests

Returns:

boolean true if loaded successfully, false otherwise

Throws:

RuntimeException

configure

public abstract boolean configure(com.yahoo.filedistribution.fileacquirer.FileAcquirer fileAcquirer,
                                  RewritesConfig config,
                                  HashMap<String,File> fileList)
                           throws RuntimeException

Perform instance creation time configuration besides the default FSA loading

Parameters:

fileAcquirer - Required param for retrieving file type config (see vespa's search container doc for more detail)

config - Config from vespa-services.xml (see vespa's search container doc for more detail)

fileList - pairs of file name and file handler for unit tests

Returns:

boolean true if loaded successfully, false otherwise

Throws:

RuntimeException

search

public Result search(Query query,
                     Execution execution)

Perform main rewrite logics for this searcher
- Skip to next rewriter if query is previously rewritten and getSkipRewriterIfRewritten() is true for this rewriter
- Execute rewriter's main rewrite logic
- Pass to the next rewriter the query to be used for dictionary retrieval

Specified by:

search in class Searcher

Parameters:

query - the query

Returns:

the result of making this query

rewrite

protected abstract HashMap<String,Object> rewrite(Query query,
                                                  String dictKey)
                                           throws RuntimeException

Perform the main rewrite logic

Parameters:

query - Query object from searcher

dictKey - the key passed from previous rewriter to be treated as "original query from user" For example, if previous is misspell rewriter, it would pass the corrected query as the "original query from user". For other rewriters which add variants, abbr, etc to the query, the original query should be passed as a key. This rewriter could still choose to ignore this key. This key is not the rewritten query itself. For example, if original query is (willl smith) and the rewritten query is (willl smith) OR (will smith) the key to be passed could be (will smith)

Returns:

HashMap which contains the key value pairs:
- whether this query has been rewritten by this rewriter
key: rewritten
value: true or false
- the key to be treated as "original query from user" in next rewriter downstream, for example, misspell rewriter would pass the corrected query as the "original query from user" to the next rewriter. For other rewriters which add variants, abbr, etc to the query, the original query should be passed as a key. This key is not necessarily consumed by the next rewriter. The next rewriter can still choose to ignore this key.
key: newDictKey
value: new dict key

Throws:

RuntimeException

getSkipRewriterIfRewritten

protected abstract boolean getSkipRewriterIfRewritten()

Check whether rewriter should be skipped if the query has been rewritten by other rewriter

Returns:

boolean Whether rewriter should be skipped

getRewriterName

public abstract String getRewriterName()

Retrieve rewriter name It should match the name used in query profile

Returns:

Name of the rewriter

getDefaultFSAs

public abstract HashMap<String,String> getDefaultFSAs()

Get default FSA dictionary names

Returns:

Pair of FSA dictionary name and filename

getQPConfig

protected String getQPConfig(Query query,
                             String paramName)

Get config parameter value set in query profile

Parameters:

query - Query object from the searcher

paramName - parameter to be retrieved

Returns:

parameter value or null if not found

getRewriteFromFSA

protected String getRewriteFromFSA(Query query,
                                   String dictName,
                                   String key)
                            throws RuntimeException

Retrieve rewrite from FSA given the original query

Parameters:

query - Query object from searcher

dictName - FSA dictionary name

key - The original query used to retrieve rewrite from the dictionary

Returns:

String The retrieved rewrites, null if query doesn't exist

Throws:

RuntimeException