Package com.yahoo.prelude.query.parser
Class TokenizeParser
- java.lang.Object
-
- com.yahoo.prelude.query.parser.AbstractParser
-
- com.yahoo.prelude.query.parser.TokenizeParser
-
- All Implemented Interfaces:
CustomParser
,Parser
public final class TokenizeParser extends AbstractParser
A parser which turns contiguous searchable character into tokens and filters out other characters. The resulting tokens are collected into a WeakAnd item.- Author:
- bratseth
-
-
Field Summary
-
Fields inherited from class com.yahoo.prelude.query.parser.AbstractParser
braceLevelURL, environment, indexFacts, language, submodes, tokens
-
-
Constructor Summary
Constructors Constructor Description TokenizeParser(ParserEnvironment environment)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description protected Item
parseItems()
-
Methods inherited from class com.yahoo.prelude.query.parser.AbstractParser
normalize, parse, parse, segment, setState, simplifyPhrases, tokenize
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface com.yahoo.prelude.query.parser.CustomParser
parse
-
-
-
-
Constructor Detail
-
TokenizeParser
public TokenizeParser(ParserEnvironment environment)
-
-
Method Detail
-
parseItems
protected Item parseItems()
- Specified by:
parseItems
in classAbstractParser
-
-