Package org.eclipse.xtext.parser.antlr
Class XtextTokenStream
- java.lang.Object
-
- org.antlr.runtime.CommonTokenStream
-
- org.eclipse.xtext.parser.antlr.XtextTokenStream
-
- All Implemented Interfaces:
org.antlr.runtime.IntStream
,org.antlr.runtime.TokenStream
public class XtextTokenStream extends org.antlr.runtime.CommonTokenStream
A token stream that is aware of the current lookahead.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static interface
XtextTokenStream.HiddenTokens
-
Constructor Summary
Constructors Constructor Description XtextTokenStream()
XtextTokenStream(org.antlr.runtime.TokenSource tokenSource, int channel)
XtextTokenStream(org.antlr.runtime.TokenSource tokenSource, ITokenDefProvider tokenDefProvider)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description protected void
basicConsume()
Make super-impl ofconsume()
accessible to subtypes.protected int
basicMark()
Make super-impl ofmark()
accessible to subtypes.protected void
basicRewind(int marker)
Make super-impl ofCommonTokenStream.rewind()
accessible to subtypes.void
consume()
protected void
doSetHiddenTokens(java.lang.String... lexerRules)
int
getCurrentLookAhead()
protected int
getFirstMarker()
java.lang.String
getLexerErrorMessage(org.antlr.runtime.Token invalidToken)
protected int
getTokenIndex(org.antlr.runtime.Token tok)
void
initCurrentLookAhead(int currentLookAhead)
int
LA(int i)
org.antlr.runtime.Token
LT(int k)
Same asCommonTokenStream.LT(int)
except that we skip hidden tokens even fork == 1
.
int
mark()
void
rewind(int marker)
XtextTokenStream.HiddenTokens
setHiddenTokens(java.lang.String... lexerRules)
void
setInitialHiddenTokens(java.lang.String... lexerRules)
protected void
skipHiddenTokens()
java.lang.String
toString(int start, int stop)
-
Methods inherited from class org.antlr.runtime.CommonTokenStream
discardOffChannelTokens, discardTokenType, fillBuffer, get, getSourceName, getTokens, getTokens, getTokens, getTokens, getTokens, getTokenSource, index, LB, release, reset, rewind, seek, setTokenSource, setTokenTypeChannel, size, skipOffTokenChannels, skipOffTokenChannelsReverse, toString, toString
-
-
-
-
Constructor Detail
-
XtextTokenStream
public XtextTokenStream()
-
XtextTokenStream
public XtextTokenStream(org.antlr.runtime.TokenSource tokenSource, int channel)
-
XtextTokenStream
public XtextTokenStream(org.antlr.runtime.TokenSource tokenSource, ITokenDefProvider tokenDefProvider)
-
-
Method Detail
-
toString
public java.lang.String toString(int start, int stop)
- Specified by:
toString
in interfaceorg.antlr.runtime.TokenStream
- Overrides:
toString
in classorg.antlr.runtime.CommonTokenStream
-
setHiddenTokens
public XtextTokenStream.HiddenTokens setHiddenTokens(java.lang.String... lexerRules)
-
setInitialHiddenTokens
public void setInitialHiddenTokens(java.lang.String... lexerRules)
-
doSetHiddenTokens
protected void doSetHiddenTokens(java.lang.String... lexerRules)
-
skipHiddenTokens
protected void skipHiddenTokens()
-
LA
public int LA(int i)
- Specified by:
LA
in interfaceorg.antlr.runtime.IntStream
- Overrides:
LA
in classorg.antlr.runtime.CommonTokenStream
-
getTokenIndex
protected int getTokenIndex(org.antlr.runtime.Token tok)
- Since:
- 2.22
-
consume
public void consume()
- Specified by:
consume
in interfaceorg.antlr.runtime.IntStream
- Overrides:
consume
in classorg.antlr.runtime.CommonTokenStream
-
basicConsume
protected void basicConsume()
Make super-impl ofconsume()
accessible to subtypes.- Since:
- 2.22
-
getFirstMarker
protected int getFirstMarker()
- Since:
- 2.1
-
mark
public int mark()
- Specified by:
mark
in interfaceorg.antlr.runtime.IntStream
- Overrides:
mark
in classorg.antlr.runtime.CommonTokenStream
-
basicMark
protected int basicMark()
Make super-impl ofmark()
accessible to subtypes.- Since:
- 2.22
-
rewind
public void rewind(int marker)
- Specified by:
rewind
in interfaceorg.antlr.runtime.IntStream
- Overrides:
rewind
in classorg.antlr.runtime.CommonTokenStream
-
basicRewind
protected void basicRewind(int marker)
Make super-impl ofCommonTokenStream.rewind()
accessible to subtypes.- Since:
- 2.22
-
LT
public org.antlr.runtime.Token LT(int k)
Same asCommonTokenStream.LT(int)
except that we skip hidden tokens even fork == 1
.
- Specified by:
LT
in interfaceorg.antlr.runtime.TokenStream
- Overrides:
LT
in classorg.antlr.runtime.CommonTokenStream
-
getCurrentLookAhead
public int getCurrentLookAhead()
-
initCurrentLookAhead
public void initCurrentLookAhead(int currentLookAhead)
-
getLexerErrorMessage
public java.lang.String getLexerErrorMessage(org.antlr.runtime.Token invalidToken)
-
-