Package org.antlr.v4.runtime
Class CommonToken
- java.lang.Object
-
- org.antlr.v4.runtime.CommonToken
-
- All Implemented Interfaces:
Serializable
,Token
,WritableToken
- Direct Known Subclasses:
TokenTagToken
public class CommonToken extends Object implements WritableToken, Serializable
- See Also:
- Serialized Form
-
-
Field Summary
Fields Modifier and Type Field Description protected int
channel
This is the backing field forgetChannel()
andsetChannel(int)
.protected int
charPositionInLine
This is the backing field forgetCharPositionInLine()
andsetCharPositionInLine(int)
.protected static Pair<TokenSource,CharStream>
EMPTY_SOURCE
protected int
index
This is the backing field forgetTokenIndex()
andsetTokenIndex(int)
.protected int
line
This is the backing field forgetLine()
andsetLine(int)
.protected Pair<TokenSource,CharStream>
source
This is the backing field forgetTokenSource()
andgetInputStream()
.protected int
start
This is the backing field forgetStartIndex()
andsetStartIndex(int)
.protected int
stop
This is the backing field forgetStopIndex()
andsetStopIndex(int)
.protected String
text
This is the backing field forgetText()
when the token text is explicitly set in the constructor or viasetText(java.lang.String)
.protected int
type
This is the backing field forgetType()
andsetType(int)
.-
Fields inherited from interface org.antlr.v4.runtime.Token
DEFAULT_CHANNEL, EOF, EPSILON, HIDDEN_CHANNEL, INVALID_TYPE, MIN_USER_CHANNEL_VALUE, MIN_USER_TOKEN_TYPE
-
-
Constructor Summary
Constructors Constructor Description CommonToken(int type)
Constructs a newCommonToken
with the specified token type.CommonToken(int type, String text)
Constructs a newCommonToken
with the specified token type and text.CommonToken(Pair<TokenSource,CharStream> source, int type, int channel, int start, int stop)
CommonToken(Token oldToken)
Constructs a newCommonToken
as a copy of anotherToken
.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description int
getChannel()
Return the channel this token.int
getCharPositionInLine()
The index of the first character of this token relative to the beginning of the line at which it occurs, 0..n-1CharStream
getInputStream()
Gets theCharStream
from which this token was derived.int
getLine()
The line number on which the 1st character of this token was matched, line=1..nint
getStartIndex()
The starting character index of the token This method is optional; return -1 if not implemented.int
getStopIndex()
The last character index of the token.String
getText()
Get the text of the token.int
getTokenIndex()
An index from 0..n-1 of the token object in the input stream.TokenSource
getTokenSource()
Gets theTokenSource
which created this token.int
getType()
Get the token type of the tokenvoid
setChannel(int channel)
void
setCharPositionInLine(int charPositionInLine)
void
setLine(int line)
void
setStartIndex(int start)
void
setStopIndex(int stop)
void
setText(String text)
Explicitly set the text for this token.void
setTokenIndex(int index)
void
setType(int type)
String
toString()
String
toString(Recognizer<?,?> r)
-
-
-
Field Detail
-
EMPTY_SOURCE
protected static final Pair<TokenSource,CharStream> EMPTY_SOURCE
-
type
protected int type
This is the backing field forgetType()
andsetType(int)
.
-
line
protected int line
This is the backing field forgetLine()
andsetLine(int)
.
-
charPositionInLine
protected int charPositionInLine
This is the backing field forgetCharPositionInLine()
andsetCharPositionInLine(int)
.
-
channel
protected int channel
This is the backing field forgetChannel()
andsetChannel(int)
.
-
source
protected Pair<TokenSource,CharStream> source
This is the backing field forgetTokenSource()
andgetInputStream()
.These properties share a field to reduce the memory footprint of
CommonToken
. Tokens created by aCommonTokenFactory
from the same source and input stream share a reference to the samePair
containing these values.
-
text
protected String text
This is the backing field forgetText()
when the token text is explicitly set in the constructor or viasetText(java.lang.String)
.- See Also:
getText()
-
index
protected int index
This is the backing field forgetTokenIndex()
andsetTokenIndex(int)
.
-
start
protected int start
This is the backing field forgetStartIndex()
andsetStartIndex(int)
.
-
stop
protected int stop
This is the backing field forgetStopIndex()
andsetStopIndex(int)
.
-
-
Constructor Detail
-
CommonToken
public CommonToken(int type)
Constructs a newCommonToken
with the specified token type.- Parameters:
type
- The token type.
-
CommonToken
public CommonToken(Pair<TokenSource,CharStream> source, int type, int channel, int start, int stop)
-
CommonToken
public CommonToken(int type, String text)
Constructs a newCommonToken
with the specified token type and text.- Parameters:
type
- The token type.text
- The text of the token.
-
CommonToken
public CommonToken(Token oldToken)
Constructs a newCommonToken
as a copy of anotherToken
.If
oldToken
is also aCommonToken
instance, the newly constructed token will share a reference to thetext
field and thePair
stored insource
. Otherwise,text
will be assigned the result of callinggetText()
, andsource
will be constructed from the result ofToken.getTokenSource()
andToken.getInputStream()
.- Parameters:
oldToken
- The token to copy.
-
-
Method Detail
-
getType
public int getType()
Description copied from interface:Token
Get the token type of the token
-
setLine
public void setLine(int line)
- Specified by:
setLine
in interfaceWritableToken
-
setText
public void setText(String text)
Explicitly set the text for this token. If {code text} is notnull
, thengetText()
will return this value rather than extracting the text from the input.- Specified by:
setText
in interfaceWritableToken
- Parameters:
text
- The explicit text of the token, ornull
if the text should be obtained from the input along with the start and stop indexes of the token.
-
getLine
public int getLine()
Description copied from interface:Token
The line number on which the 1st character of this token was matched, line=1..n
-
getCharPositionInLine
public int getCharPositionInLine()
Description copied from interface:Token
The index of the first character of this token relative to the beginning of the line at which it occurs, 0..n-1- Specified by:
getCharPositionInLine
in interfaceToken
-
setCharPositionInLine
public void setCharPositionInLine(int charPositionInLine)
- Specified by:
setCharPositionInLine
in interfaceWritableToken
-
getChannel
public int getChannel()
Description copied from interface:Token
Return the channel this token. Each token can arrive at the parser on a different channel, but the parser only "tunes" to a single channel. The parser ignores everything not on DEFAULT_CHANNEL.- Specified by:
getChannel
in interfaceToken
-
setChannel
public void setChannel(int channel)
- Specified by:
setChannel
in interfaceWritableToken
-
setType
public void setType(int type)
- Specified by:
setType
in interfaceWritableToken
-
getStartIndex
public int getStartIndex()
Description copied from interface:Token
The starting character index of the token This method is optional; return -1 if not implemented.- Specified by:
getStartIndex
in interfaceToken
-
setStartIndex
public void setStartIndex(int start)
-
getStopIndex
public int getStopIndex()
Description copied from interface:Token
The last character index of the token. This method is optional; return -1 if not implemented.- Specified by:
getStopIndex
in interfaceToken
-
setStopIndex
public void setStopIndex(int stop)
-
getTokenIndex
public int getTokenIndex()
Description copied from interface:Token
An index from 0..n-1 of the token object in the input stream. This must be valid in order to print token streams and use TokenRewriteStream. Return -1 to indicate that this token was conjured up since it doesn't have a valid index.- Specified by:
getTokenIndex
in interfaceToken
-
setTokenIndex
public void setTokenIndex(int index)
- Specified by:
setTokenIndex
in interfaceWritableToken
-
getTokenSource
public TokenSource getTokenSource()
Description copied from interface:Token
Gets theTokenSource
which created this token.- Specified by:
getTokenSource
in interfaceToken
-
getInputStream
public CharStream getInputStream()
Description copied from interface:Token
Gets theCharStream
from which this token was derived.- Specified by:
getInputStream
in interfaceToken
-
toString
public String toString(Recognizer<?,?> r)
-
-