Package com.yahoo.language.simple.kstem
Class CharacterUtils
java.lang.Object
com.yahoo.language.simple.kstem.CharacterUtils
CharacterUtils
provides a unified interface to Character-related
operations to implement backwards compatible character operations.-
Nested Class Summary
Modifier and TypeClassDescriptionstatic final class
A simple IO buffer to use withfill(CharacterBuffer, Reader)
. -
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionabstract int
codePointAt
(char[] chars, int offset, int limit) Returns the code point at the given index of the char array where only elements with index less than the limit are used.abstract int
codePointAt
(CharSequence seq, int offset) Returns the code point at the given index of theCharSequence
.abstract int
Return the number of characters inseq
.final boolean
fill
(CharacterUtils.CharacterBuffer buffer, Reader reader) Convenience method which callsfill(buffer, reader, buffer.buffer.length)
.abstract boolean
fill
(CharacterUtils.CharacterBuffer buffer, Reader reader, int numChars) Fills theCharacterUtils.CharacterBuffer
with characters read from the given readerReader
.static CharacterUtils
Returns aCharacterUtils
implementation.newCharacterBuffer
(int bufferSize) Creates a newCharacterUtils.CharacterBuffer
and allocates achar[]
of the given bufferSize.abstract int
offsetByCodePoints
(char[] buf, int start, int count, int index, int offset) Return the index withinbuf[start:start+count]
which is byoffset
code points fromindex
.final int
toChars
(int[] src, int srcOff, int srcLen, char[] dest, int destOff) Converts a sequence of unicode code points to a sequence of Java characters.final int
toCodePoints
(char[] src, int srcOff, int srcLen, int[] dest, int destOff) Converts a sequence of Java characters to a sequence of unicode code points.final void
toLowerCase
(char[] buffer, int offset, int limit) Converts each unicode codepoint to lowerCase viaCharacter.toLowerCase(int)
starting at the given offset.final void
toUpperCase
(char[] buffer, int offset, int limit) Converts each unicode codepoint to UpperCase viaCharacter.toUpperCase(int)
starting at the given offset.
-
Constructor Details
-
CharacterUtils
public CharacterUtils()
-
-
Method Details
-
getInstance
Returns aCharacterUtils
implementation. -
codePointAt
Returns the code point at the given index of theCharSequence
.- Parameters:
seq
- a character sequenceoffset
- the offset to the char values in the chars array to be converted- Returns:
- the Unicode code point at the given index
- Throws:
NullPointerException
- - if the sequence is null.IndexOutOfBoundsException
- - if the value offset is negative or not less than the length of the character sequence.
-
codePointAt
public abstract int codePointAt(char[] chars, int offset, int limit) Returns the code point at the given index of the char array where only elements with index less than the limit are used.- Parameters:
chars
- a character arrayoffset
- the offset to the char values in the chars array to be convertedlimit
- the index afer the last element that should be used to calculate codepoint.- Returns:
- the Unicode code point at the given index
- Throws:
NullPointerException
- - if the array is null.IndexOutOfBoundsException
- - if the value offset is negative or not less than the length of the char array.
-
codePointCount
Return the number of characters inseq
. -
newCharacterBuffer
Creates a newCharacterUtils.CharacterBuffer
and allocates achar[]
of the given bufferSize.- Parameters:
bufferSize
- the internal char buffer size, must be>= 2
- Returns:
- a new
CharacterUtils.CharacterBuffer
instance.
-
toLowerCase
public final void toLowerCase(char[] buffer, int offset, int limit) Converts each unicode codepoint to lowerCase viaCharacter.toLowerCase(int)
starting at the given offset.- Parameters:
buffer
- the char buffer to lowercaseoffset
- the offset to start atlimit
- the max char in the buffer to lower case
-
toUpperCase
public final void toUpperCase(char[] buffer, int offset, int limit) Converts each unicode codepoint to UpperCase viaCharacter.toUpperCase(int)
starting at the given offset.- Parameters:
buffer
- the char buffer to UPPERCASEoffset
- the offset to start atlimit
- the max char in the buffer to lower case
-
toCodePoints
public final int toCodePoints(char[] src, int srcOff, int srcLen, int[] dest, int destOff) Converts a sequence of Java characters to a sequence of unicode code points.- Returns:
- the number of code points written to the destination buffer
-
toChars
public final int toChars(int[] src, int srcOff, int srcLen, char[] dest, int destOff) Converts a sequence of unicode code points to a sequence of Java characters.- Returns:
- the number of chars written to the destination buffer
-
fill
public abstract boolean fill(CharacterUtils.CharacterBuffer buffer, Reader reader, int numChars) throws IOException Fills theCharacterUtils.CharacterBuffer
with characters read from the given readerReader
. This method tries to readnumChars
characters into theCharacterUtils.CharacterBuffer
, each call to fill will start filling the buffer from offset0
up tonumChars
. In case code points can span across 2 java characters, this method may only fillnumChars - 1
characters in order not to split in the middle of a surrogate pair, even if there are remaining characters in theReader
.This method guarantees that the given
CharacterUtils.CharacterBuffer
will never contain a high surrogate character as the last element in the buffer unless it is the last available character in the reader. In other words, high and low surrogate pairs will always be preserved across buffer boarders.A return value of
false
means that this method call exhausted the reader, but there may be some bytes which have been read, which can be verified by checking whetherbuffer.getLength() > 0
.- Parameters:
buffer
- the buffer to fill.reader
- the reader to read characters from.numChars
- the number of chars to read- Returns:
false
if and only if reader.read returned -1 while trying to fill the buffer- Throws:
IOException
- if the reader throws anIOException
.
-
fill
Convenience method which callsfill(buffer, reader, buffer.buffer.length)
.- Throws:
IOException
-
offsetByCodePoints
public abstract int offsetByCodePoints(char[] buf, int start, int count, int index, int offset) Return the index withinbuf[start:start+count]
which is byoffset
code points fromindex
.
-