public final class WhitespaceTokenizer extends CharTokenizer
You must specify the required Version
compatibility when creating
WhitespaceTokenizer
:
CharTokenizer
uses an int based API to normalize and
detect token characters. See CharTokenizer.isTokenChar(int)
and
CharTokenizer.normalize(int)
for details.AttributeSource.AttributeFactory, AttributeSource.State
Constructor and Description |
---|
WhitespaceTokenizer(Version matchVersion,
AttributeSource.AttributeFactory factory,
Reader in)
Construct a new WhitespaceTokenizer using a given
AttributeSource.AttributeFactory . |
WhitespaceTokenizer(Version matchVersion,
Reader in)
Construct a new WhitespaceTokenizer.
|
end, incrementToken, reset
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, restoreState, toString
public WhitespaceTokenizer(Version matchVersion, Reader in)
in
- the input to split up into tokenspublic WhitespaceTokenizer(Version matchVersion, AttributeSource.AttributeFactory factory, Reader in)
AttributeSource.AttributeFactory
.Copyright © 2010 - 2020 Adobe. All Rights Reserved