Package org.eclipse.xtext.parser.antlr
Class AbstractIndentationTokenSource
- java.lang.Object
-
- org.eclipse.xtext.parser.antlr.AbstractSplittingTokenSource
-
- org.eclipse.xtext.parser.antlr.AbstractIndentationTokenSource
-
- All Implemented Interfaces:
org.antlr.runtime.TokenSource
public abstract class AbstractIndentationTokenSource extends AbstractSplittingTokenSource
Abstract implementation of a token source, that splits according to indentation levels after newlines. Blank lines are ignored. At the end of file token, all pending indentations will be handled and dedentation tokens will be issued accordingly ifshouldEmitPendingEndTokens()
returns true.- Since:
- 2.8
-
-
Field Summary
Fields Modifier and Type Field Description protected int
currentEnd
protected int
currentIndentation
protected int
currentOffset
protected java.util.Stack<java.lang.Integer>
indentationStack
protected int
nextOffset
-
Fields inherited from class org.eclipse.xtext.parser.antlr.AbstractSplittingTokenSource
LEXER_RULE_PREFIX
-
-
Constructor Summary
Constructors Modifier Constructor Description protected
AbstractIndentationTokenSource(org.antlr.runtime.TokenSource delegate)
-
Method Summary
All Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description protected int
computeIndentation(java.lang.String text)
protected int
computeIndentationRelevantCharCount(java.lang.String text)
protected org.antlr.runtime.Token
createBeginToken(int offset)
protected org.antlr.runtime.Token
createEndToken(int offset)
protected void
decreaseIndentation(int indentation, ITokenAcceptor result)
protected void
doSplitEofToken(org.antlr.runtime.Token token, ITokenAcceptor result)
protected void
doSplitToken(org.antlr.runtime.Token token, ITokenAcceptor result)
protected void
doSplitTokenImpl(org.antlr.runtime.Token token, ITokenAcceptor result)
The token was previously determined as potentially to-be-splitted thus we emit additional indentation or dedenting tokens.protected abstract int
getBeginTokenType()
protected abstract int
getEndTokenType()
protected char
getTabWidth()
protected void
handleRemainingText(org.antlr.runtime.Token token, java.lang.String text, int indentation, ITokenAcceptor result)
org.antlr.runtime.Token
nextToken()
protected boolean
shouldEmitPendingEndTokens()
protected boolean
shouldSplitToken(org.antlr.runtime.Token token)
protected abstract boolean
shouldSplitTokenImpl(org.antlr.runtime.Token token)
Returntrue
if the given token is a whitespace token that should be splitted.protected void
splitWithText(org.antlr.runtime.Token token, java.lang.String text, ITokenAcceptor result)
-
Methods inherited from class org.eclipse.xtext.parser.antlr.AbstractSplittingTokenSource
createToken, getDelegate, getLexerRuleName, getRuleName, getSourceName, getTokenType, initializeTokenDefsFrom, setDelegate, splitToken
-
-
-
-
Method Detail
-
shouldSplitToken
protected boolean shouldSplitToken(org.antlr.runtime.Token token)
- Specified by:
shouldSplitToken
in classAbstractSplittingTokenSource
-
shouldSplitTokenImpl
protected abstract boolean shouldSplitTokenImpl(org.antlr.runtime.Token token)
Returntrue
if the given token is a whitespace token that should be splitted. Usually the token type will be compared to one of the int constants from the parser.
-
doSplitEofToken
protected void doSplitEofToken(org.antlr.runtime.Token token, ITokenAcceptor result)
- Since:
- 2.13
-
doSplitToken
protected void doSplitToken(org.antlr.runtime.Token token, ITokenAcceptor result)
- Specified by:
doSplitToken
in classAbstractSplittingTokenSource
-
shouldEmitPendingEndTokens
protected boolean shouldEmitPendingEndTokens()
-
nextToken
public org.antlr.runtime.Token nextToken()
- Specified by:
nextToken
in interfaceorg.antlr.runtime.TokenSource
- Overrides:
nextToken
in classAbstractSplittingTokenSource
-
doSplitTokenImpl
protected void doSplitTokenImpl(org.antlr.runtime.Token token, ITokenAcceptor result)
The token was previously determined as potentially to-be-splitted thus we emit additional indentation or dedenting tokens.
-
handleRemainingText
protected void handleRemainingText(org.antlr.runtime.Token token, java.lang.String text, int indentation, ITokenAcceptor result)
- Since:
- 2.13
-
decreaseIndentation
protected void decreaseIndentation(int indentation, ITokenAcceptor result)
-
splitWithText
protected void splitWithText(org.antlr.runtime.Token token, java.lang.String text, ITokenAcceptor result)
-
createEndToken
protected org.antlr.runtime.Token createEndToken(int offset)
-
getEndTokenType
protected abstract int getEndTokenType()
-
getBeginTokenType
protected abstract int getBeginTokenType()
-
createBeginToken
protected org.antlr.runtime.Token createBeginToken(int offset)
-
computeIndentationRelevantCharCount
protected int computeIndentationRelevantCharCount(java.lang.String text)
-
computeIndentation
protected int computeIndentation(java.lang.String text)
-
getTabWidth
protected char getTabWidth()
-
-