public final class CommonGramsFilter extends TokenFilter
PositionIncrementAttribute.setPositionIncrement(int)
. Bigrams have a type
of GRAM_TYPE
Example:
AttributeSource.AttributeFactory, AttributeSource.State
Constructor and Description |
---|
CommonGramsFilter(Version matchVersion,
TokenStream input,
CharArraySet commonWords)
Construct a token stream filtering the given input using a Set of common
words to create bigrams.
|
Modifier and Type | Method and Description |
---|---|
boolean |
incrementToken()
Inserts bigrams for common words into a token stream.
|
void |
reset()
This method is called by a consumer before it begins consumption using
TokenStream.incrementToken() . |
close, end
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, restoreState, toString
public static final String GRAM_TYPE
public CommonGramsFilter(Version matchVersion, TokenStream input, CharArraySet commonWords)
input
- TokenStream input in filter chaincommonWords
- The set of common words.public boolean incrementToken() throws IOException
incrementToken
in class TokenStream
IOException
public void reset() throws IOException
TokenStream.incrementToken()
.
Resets this stream to a clean state. Stateful implementations must implement this method so that they can be reused, just as if they had been created fresh.
If you override this method, always call super.reset()
, otherwise
some internal state will not be correctly reset (e.g., Tokenizer
will
throw IllegalStateException
on further usage).
NOTE:
The default implementation chains the call to the input TokenStream, so
be sure to call super.reset()
when overriding this method.
reset
in class TokenFilter
IOException
Copyright © 2010 - 2020 Adobe. All Rights Reserved