Package com.yahoo.prelude.query.parser

Class TokenizeParser

java.lang.Object

com.yahoo.prelude.query.parser.AbstractParser

com.yahoo.prelude.query.parser.TokenizeParser

All Implemented Interfaces:

CustomParser, Parser

public final class TokenizeParser
extends AbstractParser

A parser which turns contiguous searchable character into tokens and filters out other characters. The resulting tokens are collected into a WeakAnd item.

Author:

bratseth

Field Summary

Fields inherited from class com.yahoo.prelude.query.parser.AbstractParser

braceLevelURL, environment, indexFacts, language, submodes, tokens

Constructor Summary

Constructors

Constructor	Description
TokenizeParser(ParserEnvironment environment)

Method Summary

Modifier and Type	Method	Description
protected Item	parseItems()

Methods inherited from class com.yahoo.prelude.query.parser.AbstractParser

normalize, parse, parse, segment, setState, simplifyPhrases, tokenize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.yahoo.prelude.query.parser.CustomParser

parse

Constructor Detail

TokenizeParser

public TokenizeParser(ParserEnvironment environment)

Method Detail

parseItems

protected Item parseItems()

Specified by:

parseItems in class AbstractParser