org.dizitart.no2.fulltext

Class UniversalTextTokenizer

java.lang.Object

org.dizitart.no2.fulltext.BaseTextTokenizer

org.dizitart.no2.fulltext.UniversalTextTokenizer

All Implemented Interfaces:

TextTokenizer

Direct Known Subclasses:

EnglishTextTokenizer

public class UniversalTextTokenizer
extends BaseTextTokenizer

A TextTokenizer implementation for various languages.

Since:

2.1.0

See Also:

Languages

Constructor Summary

Constructors

Constructor and Description
UniversalTextTokenizer()

Method Summary

Modifier and Type	Method and Description
void	loadAllLanguages()
void	loadLanguage(Languages... languages)
java.util.Set<java.lang.String>	stopWords() Gets all stop-words for a language.

Methods inherited from class org.dizitart.no2.fulltext.BaseTextTokenizer

convertWord, tokenize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

UniversalTextTokenizer

public UniversalTextTokenizer()

Method Detail

stopWords

public java.util.Set<java.lang.String> stopWords()

Description copied from interface: TextTokenizer

Gets all stop-words for a language.

Returns:

the set of all stop-words.

loadLanguage

public void loadLanguage(Languages... languages)

loadAllLanguages

public void loadAllLanguages()