Class PublicScanner
java.lang.Object
org.aspectj.org.eclipse.jdt.internal.core.util.PublicScanner
- All Implemented Interfaces:
IScanner
,ITerminalSymbols
-
Field Summary
Modifier and TypeFieldDescriptionstatic String
static int
char[][][][]
boolean
static int
int
int[]
int[]
int[]
long
boolean
static int
char
int
boolean
static String
int
int
char[][]
int[][]
char[][]
char[][]
static int
static int
static String
int
boolean
static String
static String
static String
static String
static String
static String
static String
static String
static String
static String
static String
static String
static String
static String
boolean
protected int
protected int
int[]
int
static int
static int
protected int
static String
static int
boolean
boolean
boolean
static int
boolean
boolean
char[]
long
static int
int
static char
static int
static char[]
static int
char[][]
char[][]
boolean
boolean
static String
boolean
static String
static String
static String
boolean
boolean
boolean
char[]
int
Fields inherited from interface org.aspectj.org.eclipse.jdt.core.compiler.ITerminalSymbols
TokenNameabstract, TokenNameAND, TokenNameAND_AND, TokenNameAND_EQUAL, TokenNameARROW, TokenNameassert, TokenNameAT, TokenNameboolean, TokenNamebreak, TokenNamebyte, TokenNamecase, TokenNamecatch, TokenNamechar, TokenNameCharacterLiteral, TokenNameclass, TokenNameCOLON, TokenNameCOLON_COLON, TokenNameCOMMA, TokenNameCOMMENT_BLOCK, TokenNameCOMMENT_JAVADOC, TokenNameCOMMENT_LINE, TokenNameconst, TokenNamecontinue, TokenNamedefault, TokenNameDIVIDE, TokenNameDIVIDE_EQUAL, TokenNamedo, TokenNameDOT, TokenNamedouble, TokenNameDoubleLiteral, TokenNameELLIPSIS, TokenNameelse, TokenNameenum, TokenNameEOF, TokenNameEQUAL, TokenNameEQUAL_EQUAL, TokenNameERROR, TokenNameextends, TokenNamefalse, TokenNamefinal, TokenNamefinally, TokenNamefloat, TokenNameFloatingPointLiteral, TokenNamefor, TokenNamegoto, TokenNameGREATER, TokenNameGREATER_EQUAL, TokenNameIdentifier, TokenNameif, TokenNameimplements, TokenNameimport, TokenNameinstanceof, TokenNameint, TokenNameIntegerLiteral, TokenNameinterface, TokenNameLBRACE, TokenNameLBRACKET, TokenNameLEFT_SHIFT, TokenNameLEFT_SHIFT_EQUAL, TokenNameLESS, TokenNameLESS_EQUAL, TokenNamelong, TokenNameLongLiteral, TokenNameLPAREN, TokenNameMINUS, TokenNameMINUS_EQUAL, TokenNameMINUS_MINUS, TokenNameMULTIPLY, TokenNameMULTIPLY_EQUAL, TokenNamenative, TokenNamenew, TokenNameNOT, TokenNameNOT_EQUAL, TokenNamenull, TokenNameOR, TokenNameOR_EQUAL, TokenNameOR_OR, TokenNamepackage, TokenNamePLUS, TokenNamePLUS_EQUAL, TokenNamePLUS_PLUS, TokenNameprivate, TokenNameprotected, TokenNamepublic, TokenNameQUESTION, TokenNameRBRACE, TokenNameRBRACKET, TokenNameREMAINDER, TokenNameREMAINDER_EQUAL, TokenNamereturn, TokenNameRIGHT_SHIFT, TokenNameRIGHT_SHIFT_EQUAL, TokenNameRPAREN, TokenNameSEMICOLON, TokenNameshort, TokenNamestatic, TokenNamestrictfp, TokenNameStringLiteral, TokenNamesuper, TokenNameswitch, TokenNamesynchronized, TokenNameTextBlock, TokenNamethis, TokenNamethrow, TokenNamethrows, TokenNametransient, TokenNametrue, TokenNametry, TokenNameTWIDDLE, TokenNameUNSIGNED_RIGHT_SHIFT, TokenNameUNSIGNED_RIGHT_SHIFT_EQUAL, TokenNamevoid, TokenNamevolatile, TokenNamewhile, TokenNameWHITESPACE, TokenNameXOR, TokenNameXOR_EQUAL
-
Constructor Summary
ConstructorDescriptionPublicScanner(boolean tokenizeComments, boolean tokenizeWhiteSpace, boolean checkNonExternalizedStringLiterals, long sourceLevel, char[][] taskTags, char[][] taskPriorities, boolean isTaskCaseSensitive)
PublicScanner(boolean tokenizeComments, boolean tokenizeWhiteSpace, boolean checkNonExternalizedStringLiterals, long sourceLevel, long complianceLevel, char[][] taskTags, char[][] taskPriorities, boolean isTaskCaseSensitive)
PublicScanner(boolean tokenizeComments, boolean tokenizeWhiteSpace, boolean checkNonExternalizedStringLiterals, long sourceLevel, long complianceLevel, char[][] taskTags, char[][] taskPriorities, boolean isTaskCaseSensitive, boolean previewEnabled)
-
Method Summary
Modifier and TypeMethodDescriptionboolean
atEnd()
void
checkTaskTag(int commentStart, int commentEnd)
char[]
int
Answers the ending position of the current token inside the original source.char[]
Answers the current identifier source, after unicode escape sequences have been translated into unicode characters.char[]
int
Answers the starting position of the current token inside the original source.int
getLineEnd(int lineNumber)
Answers the ending position of a given line number.int[]
Answers an array of the ending positions of the lines encountered so far.int
getLineNumber(int position)
Search the line number corresponding to a specific positionint
getLineStart(int lineNumber)
Search the source position corresponding to the beginning of a given line number Line numbers are 1-based, and relative to the scanner initialPosition.int
boolean
getNextChar(char testedChar)
int
getNextChar(char testedChar1, char testedChar2)
boolean
boolean
getNextCharAsDigit(int radix)
boolean
boolean
int
int
Read the next token in the source, and answers its ID as specified byITerminalSymbols
.void
NLSTag[]
char[]
Answers the current identifier source, before unicode escape sequences have been translated into unicode characters.char[]
char[]
Answers the original source being processed (not a copy of it).protected boolean
static boolean
isIdentifier(int token)
static boolean
isKeyword(int token)
static boolean
isLiteral(int token)
void
boolean
void
void
void
recordComment(int token)
void
resetTo(int begin, int end)
Reposition the scanner on some portion of the original source.protected void
protected boolean
protected boolean
int
int
int
int
scanNumber(boolean dotPrefix)
void
setSource(char[] sourceString)
Set the scanner source to process.void
setSource(char[] contents, CompilationResult compilationResult)
void
setSource(CompilationResult compilationResult)
toString()
toStringAction(int act)
void
unicodeInitializeBuffer(int length)
void
void
unicodeStore(char character)
-
Field Details
-
sourceLevel
public long sourceLevel -
complianceLevel
public long complianceLevel -
previewEnabled
public boolean previewEnabled -
useAssertAsAnIndentifier
public boolean useAssertAsAnIndentifier -
containsAssertKeyword
public boolean containsAssertKeyword -
useEnumAsAnIndentifier
public boolean useEnumAsAnIndentifier -
recordLineSeparator
public boolean recordLineSeparator -
currentCharacter
public char currentCharacter -
startPosition
public int startPosition -
currentPosition
public int currentPosition -
initialPosition
public int initialPosition -
eofPosition
public int eofPosition -
skipComments
public boolean skipComments -
tokenizeComments
public boolean tokenizeComments -
tokenizeWhiteSpace
public boolean tokenizeWhiteSpace -
source
public char[] source -
withoutUnicodeBuffer
public char[] withoutUnicodeBuffer -
withoutUnicodePtr
public int withoutUnicodePtr -
unicodeAsBackSlash
public boolean unicodeAsBackSlash -
scanningFloatLiteral
public boolean scanningFloatLiteral -
COMMENT_ARRAYS_SIZE
public static final int COMMENT_ARRAYS_SIZE- See Also:
- Constant Field Values
-
commentStops
public int[] commentStops -
commentStarts
public int[] commentStarts -
commentTagStarts
public int[] commentTagStarts -
commentPtr
public int commentPtr -
lastCommentLinePosition
protected int lastCommentLinePosition -
foundTaskTags
public char[][] foundTaskTags -
foundTaskMessages
public char[][] foundTaskMessages -
foundTaskPriorities
public char[][] foundTaskPriorities -
foundTaskPositions
public int[][] foundTaskPositions -
foundTaskCount
public int foundTaskCount -
taskTags
public char[][] taskTags -
taskPriorities
public char[][] taskPriorities -
isTaskCaseSensitive
public boolean isTaskCaseSensitive -
diet
public boolean diet -
lineEnds
public int[] lineEnds -
linePtr
public int linePtr -
wasAcr
public boolean wasAcr -
END_OF_SOURCE
- See Also:
- Constant Field Values
-
INVALID_HEXA
- See Also:
- Constant Field Values
-
INVALID_OCTAL
- See Also:
- Constant Field Values
-
INVALID_CHARACTER_CONSTANT
- See Also:
- Constant Field Values
-
INVALID_ESCAPE
- See Also:
- Constant Field Values
-
INVALID_INPUT
- See Also:
- Constant Field Values
-
INVALID_TEXTBLOCK
- See Also:
- Constant Field Values
-
INVALID_UNICODE_ESCAPE
- See Also:
- Constant Field Values
-
INVALID_FLOAT
- See Also:
- Constant Field Values
-
INVALID_LOW_SURROGATE
- See Also:
- Constant Field Values
-
INVALID_HIGH_SURROGATE
- See Also:
- Constant Field Values
-
NULL_SOURCE_STRING
- See Also:
- Constant Field Values
-
UNTERMINATED_STRING
- See Also:
- Constant Field Values
-
UNTERMINATED_TEXT_BLOCK
- See Also:
- Constant Field Values
-
UNTERMINATED_COMMENT
- See Also:
- Constant Field Values
-
INVALID_CHAR_IN_STRING
- See Also:
- Constant Field Values
-
INVALID_DIGIT
- See Also:
- Constant Field Values
-
INVALID_BINARY
- See Also:
- Constant Field Values
-
BINARY_LITERAL_NOT_BELOW_17
- See Also:
- Constant Field Values
-
ILLEGAL_HEXA_LITERAL
- See Also:
- Constant Field Values
-
INVALID_UNDERSCORE
- See Also:
- Constant Field Values
-
UNDERSCORES_IN_LITERALS_NOT_BELOW_17
- See Also:
- Constant Field Values
-
OptimizedLength
public static final int OptimizedLength- See Also:
- Constant Field Values
-
charArray_length
public final char[][][][] charArray_length -
TAG_PREFIX
public static final char[] TAG_PREFIX -
TAG_PREFIX_LENGTH
public static final int TAG_PREFIX_LENGTH -
TAG_POSTFIX
public static final char TAG_POSTFIX- See Also:
- Constant Field Values
-
TAG_POSTFIX_LENGTH
public static final int TAG_POSTFIX_LENGTH- See Also:
- Constant Field Values
-
nlsTagsPtr
protected int nlsTagsPtr -
checkNonExternalizedStringLiterals
public boolean checkNonExternalizedStringLiterals -
lastPosition
protected int lastPosition -
returnOnlyGreater
public boolean returnOnlyGreater -
insideRecovery
public boolean insideRecovery -
RoundBracket
public static final int RoundBracket- See Also:
- Constant Field Values
-
SquareBracket
public static final int SquareBracket- See Also:
- Constant Field Values
-
CurlyBracket
public static final int CurlyBracket- See Also:
- Constant Field Values
-
BracketKinds
public static final int BracketKinds- See Also:
- Constant Field Values
-
LOW_SURROGATE_MIN_VALUE
public static final int LOW_SURROGATE_MIN_VALUE- See Also:
- Constant Field Values
-
HIGH_SURROGATE_MIN_VALUE
public static final int HIGH_SURROGATE_MIN_VALUE- See Also:
- Constant Field Values
-
HIGH_SURROGATE_MAX_VALUE
public static final int HIGH_SURROGATE_MAX_VALUE- See Also:
- Constant Field Values
-
LOW_SURROGATE_MAX_VALUE
public static final int LOW_SURROGATE_MAX_VALUE- See Also:
- Constant Field Values
-
-
Constructor Details
-
PublicScanner
public PublicScanner() -
PublicScanner
public PublicScanner(boolean tokenizeComments, boolean tokenizeWhiteSpace, boolean checkNonExternalizedStringLiterals, long sourceLevel, long complianceLevel, char[][] taskTags, char[][] taskPriorities, boolean isTaskCaseSensitive) -
PublicScanner
public PublicScanner(boolean tokenizeComments, boolean tokenizeWhiteSpace, boolean checkNonExternalizedStringLiterals, long sourceLevel, long complianceLevel, char[][] taskTags, char[][] taskPriorities, boolean isTaskCaseSensitive, boolean previewEnabled) -
PublicScanner
public PublicScanner(boolean tokenizeComments, boolean tokenizeWhiteSpace, boolean checkNonExternalizedStringLiterals, long sourceLevel, char[][] taskTags, char[][] taskPriorities, boolean isTaskCaseSensitive)
-
-
Method Details
-
atEnd
public final boolean atEnd() -
checkTaskTag
- Throws:
InvalidInputException
-
getCurrentIdentifierSource
public char[] getCurrentIdentifierSource() -
getCurrentTokenEndPosition
public int getCurrentTokenEndPosition()Description copied from interface:IScanner
Answers the ending position of the current token inside the original source. This position is zero-based and inclusive. It corresponds to the position of the last character which is part of this token. If this character was a unicode escape sequence, it points at the last character of this sequence.- Specified by:
getCurrentTokenEndPosition
in interfaceIScanner
- Returns:
- the ending position of the current token inside the original source
-
getCurrentTokenSource
public char[] getCurrentTokenSource()Description copied from interface:IScanner
Answers the current identifier source, after unicode escape sequences have been translated into unicode characters. For example, if original source was\\u0061bc
then it will answerabc
.- Specified by:
getCurrentTokenSource
in interfaceIScanner
- Returns:
- the current identifier source, after unicode escape sequences have been translated into unicode characters
-
getCurrentTokenString
-
getCurrentTokenSourceString
public char[] getCurrentTokenSourceString() -
scanForTextBlockBeginning
protected final boolean scanForTextBlockBeginning() -
scanForTextBlockClose
- Throws:
InvalidInputException
-
getCurrentStringLiteral
-
getRawTokenSource
public final char[] getRawTokenSource()Description copied from interface:IScanner
Answers the current identifier source, before unicode escape sequences have been translated into unicode characters. For example, if original source was\\u0061bc
then it will answer\\u0061bc
.- Specified by:
getRawTokenSource
in interfaceIScanner
- Returns:
- the current identifier source, before unicode escape sequences have been translated into unicode characters
-
getRawTokenSourceEnd
public final char[] getRawTokenSourceEnd() -
getCurrentTokenStartPosition
public int getCurrentTokenStartPosition()Description copied from interface:IScanner
Answers the starting position of the current token inside the original source. This position is zero-based and inclusive. It corresponds to the position of the first character which is part of this token. If this character was a unicode escape sequence, it points at the first character of this sequence.- Specified by:
getCurrentTokenStartPosition
in interfaceIScanner
- Returns:
- the starting position of the current token inside the original source
-
getLineEnd
public final int getLineEnd(int lineNumber)Description copied from interface:IScanner
Answers the ending position of a given line number. This line has to have been encountered already in the tokenization process (in other words, it cannot be used to compute positions of lines beyond current token). Once the entire source has been processed, it can be used without any limit. Line ending positions are zero-based, and correspond to the last character of the line separator (in case multi-character line separators).- Specified by:
getLineEnd
in interfaceIScanner
- Parameters:
lineNumber
- the given line number- Returns:
- the ending position of a given line number
-
getLineEnds
public final int[] getLineEnds()Description copied from interface:IScanner
Answers an array of the ending positions of the lines encountered so far. Line ending positions are zero-based, and correspond to the last character of the line separator (in case multi-character line separators).- Specified by:
getLineEnds
in interfaceIScanner
- Returns:
- an array of the ending positions of the lines encountered so far
-
getLineStart
public final int getLineStart(int lineNumber)Search the source position corresponding to the beginning of a given line number Line numbers are 1-based, and relative to the scanner initialPosition. Character positions are 0-based. e.g. getLineStart(1) --> 0 indicates that the first line starts at character 0. In case the given line number is inconsistent, answers -1.- Specified by:
getLineStart
in interfaceIScanner
- Parameters:
lineNumber
- int- Returns:
- int
-
getNextChar
public final int getNextChar() -
getNextCharWithBoundChecks
public final int getNextCharWithBoundChecks() -
getNextChar
public final boolean getNextChar(char testedChar) -
getNextChar
public final int getNextChar(char testedChar1, char testedChar2) -
getNextCharAsDigit
- Throws:
InvalidInputException
-
getNextCharAsDigit
public final boolean getNextCharAsDigit(int radix) -
getNextCharAsJavaIdentifierPartWithBoundCheck
public boolean getNextCharAsJavaIdentifierPartWithBoundCheck() -
getNextCharAsJavaIdentifierPart
public boolean getNextCharAsJavaIdentifierPart() -
scanIdentifier
- Throws:
InvalidInputException
-
getNextToken
Description copied from interface:IScanner
Read the next token in the source, and answers its ID as specified byITerminalSymbols
. Note that the actual token ID values are subject to change if new keywords were added to the language (for instance, 'assert' is a keyword in 1.4).- Specified by:
getNextToken
in interfaceIScanner
- Returns:
- the next token
- Throws:
InvalidInputException
- in case a lexical error was detected while reading the current token
-
getNextUnicodeChar
- Throws:
InvalidInputException
-
getNLSTags
-
getSource
public char[] getSource()Description copied from interface:IScanner
Answers the original source being processed (not a copy of it). -
isFirstTag
protected boolean isFirstTag() -
jumpOverMethodBody
public final void jumpOverMethodBody() -
jumpOverUnicodeWhiteSpace
- Throws:
InvalidInputException
-
pushLineSeparator
public final void pushLineSeparator() -
pushUnicodeLineSeparator
public final void pushUnicodeLineSeparator() -
recordComment
public void recordComment(int token) -
resetTo
public void resetTo(int begin, int end)Reposition the scanner on some portion of the original source. The given endPosition is the last valid position. Beyond this position, the scanner will answer EOF tokens (ITerminalSymbols.TokenNameEOF
). -
scanEscapeCharacter
- Throws:
InvalidInputException
-
scanIdentifierOrKeywordWithBoundCheck
public int scanIdentifierOrKeywordWithBoundCheck() -
scanIdentifierOrKeyword
public int scanIdentifierOrKeyword() -
scanNumber
- Throws:
InvalidInputException
-
getLineNumber
public final int getLineNumber(int position)Search the line number corresponding to a specific position- Specified by:
getLineNumber
in interfaceIScanner
- Parameters:
position
- int- Returns:
- int
-
setSource
public final void setSource(char[] sourceString)Description copied from interface:IScanner
Set the scanner source to process. By default, the scanner will consider starting at the beginning of the source until it reaches its end. If the given source isnull
, this clears the source. -
setSource
-
setSource
-
toString
-
toStringAction
-
unicodeInitializeBuffer
public void unicodeInitializeBuffer(int length) -
unicodeStore
public void unicodeStore() -
unicodeStore
public void unicodeStore(char character) -
isIdentifier
public static boolean isIdentifier(int token) -
isLiteral
public static boolean isLiteral(int token) -
isKeyword
public static boolean isKeyword(int token)
-