Package com.yahoo.language.process
Class SegmenterImpl
- java.lang.Object
-
- com.yahoo.language.process.SegmenterImpl
-
-
Constructor Summary
Constructors Constructor Description SegmenterImpl(Tokenizer tokenizer)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description java.util.List<java.lang.String>
segment(java.lang.String input, Language language)
Split input-string into tokens, and returned a list of tokens in unprocessed form (i.e.
-
-
-
Constructor Detail
-
SegmenterImpl
public SegmenterImpl(Tokenizer tokenizer)
-
-
Method Detail
-
segment
public java.util.List<java.lang.String> segment(java.lang.String input, Language language)
Description copied from interface:Segmenter
Split input-string into tokens, and returned a list of tokens in unprocessed form (i.e. lowercased, normalized and stemmed if applicable, see @link{StemMode} for list of stemming options). It is assumed that the input only contains word-characters, any punctuation and spacing tokens will be removed.
-
-