Package com.yahoo.language
Interface Linguistics
- All Known Implementing Classes:
SimpleLinguistics
public interface Linguistics
Factory of linguistic processors. For technical reasons this provides more flexibility to provide separate components for different operations than is needed in many cases; in particular the tokenizer should typically stem, transform and normalize using the same operations as provided directly by this. A set of adaptors are provided that makes this easy to achieve. Refer to the {com.yahoo.language.simple.SimpleLinguistics} implementation to set this up.
Thread safety: Instances of this factory type must be thread safe but the processors returned by the factory methods do not. Clients should request separate processor instances for each thread.
- Author:
- Mathias Mølster Lidal, Simon Thoresen Hult, bratseth
-
Nested Class Summary
-
Method Summary
Modifier and TypeMethodDescriptionboolean
equals
(Linguistics other) Check if another instance is equivalent to this oneReturns a thread-unsafe character classes instance.Returns a thread-unsafe detector.Returns a thread-unsafe gram splitter.Returns a thread-unsafe normalizer.Returns a thread-unsafe segmenter.Returns a thread-unsafe stemmer or lemmatizer.Returns a thread-unsafe tokenizer.Returns a thread-unsafe transformer.
-
Method Details
-
getStemmer
Stemmer getStemmer()Returns a thread-unsafe stemmer or lemmatizer. This is used at query time to do stemming of search terms to indexes which contains text tokenized with stemming turned on -
getTokenizer
Tokenizer getTokenizer()Returns a thread-unsafe tokenizer. This is used at indexing time to produce an optionally stemmed and transformed (accent normalized) stream of indexable tokens. -
getNormalizer
Normalizer getNormalizer()Returns a thread-unsafe normalizer. This is used at query time to cjk normalize query text. -
getTransformer
Transformer getTransformer()Returns a thread-unsafe transformer. This is used at query time to do stemming of search terms to indexes which contains text tokenized with accent normalization turned on -
getSegmenter
Segmenter getSegmenter()Returns a thread-unsafe segmenter. This is used at query time to find the individual semantic components of search terms to indexes tokenized with segmentation. -
getDetector
Detector getDetector()Returns a thread-unsafe detector. The language of the text is a parameter to other linguistic operations. This is used to determine the language of a query or document field when not specified explicitly. -
getGramSplitter
GramSplitter getGramSplitter()Returns a thread-unsafe gram splitter. This is used to split query or document text into fixed-length grams which allows matching without needing or using segmented tokens. -
getCharacterClasses
CharacterClasses getCharacterClasses()Returns a thread-unsafe character classes instance. -
equals
Check if another instance is equivalent to this one
-