Package org.eclipse.xtext.parser.antlr
Class AbstractSplittingTokenSource
- java.lang.Object
-
- org.eclipse.xtext.parser.antlr.AbstractSplittingTokenSource
-
- All Implemented Interfaces:
org.antlr.runtime.TokenSource
- Direct Known Subclasses:
AbstractIndentationTokenSource
public abstract class AbstractSplittingTokenSource extends java.lang.Object implements org.antlr.runtime.TokenSource
Delegating token source that is capable of splitting a given token into other tokens. This allows clients to implement sophisticated semantics when dealing with certain token types in different use cases, e.g. a token source that will provide tokens for a python-like whitespace aware language.
-
-
Field Summary
Fields Modifier and Type Field Description static java.lang.String
LEXER_RULE_PREFIX
Deprecated.useTokenTool.LEXER_RULE_PREFIX
if strictly necessary.
-
Constructor Summary
Constructors Modifier Constructor Description protected
AbstractSplittingTokenSource()
-
Method Summary
All Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description protected org.antlr.runtime.CommonToken
createToken(org.antlr.runtime.CommonToken prototype, java.lang.String text, java.lang.Integer charPosInLine, java.lang.Integer channel, java.lang.Integer start, java.lang.Integer stop, java.lang.Integer type)
Create a new token from the given prototype.protected abstract void
doSplitToken(org.antlr.runtime.Token splitMe, ITokenAcceptor acceptor)
org.antlr.runtime.TokenSource
getDelegate()
java.lang.String
getLexerRuleName(java.lang.String antlrTokenDef)
protected java.lang.String
getRuleName(org.antlr.runtime.Token token)
java.lang.String
getSourceName()
protected int
getTokenType(java.lang.String ruleNameWithRulePrefix)
void
initializeTokenDefsFrom(ITokenDefProvider provider)
org.antlr.runtime.Token
nextToken()
void
setDelegate(org.antlr.runtime.TokenSource delegate)
protected abstract boolean
shouldSplitToken(org.antlr.runtime.Token candidate)
protected void
splitToken(org.antlr.runtime.Token splitMe, ITokenAcceptor acceptor)
-
-
-
Field Detail
-
LEXER_RULE_PREFIX
@Deprecated public static final java.lang.String LEXER_RULE_PREFIX
Deprecated.useTokenTool.LEXER_RULE_PREFIX
if strictly necessary.- See Also:
- Constant Field Values
-
-
Method Detail
-
setDelegate
public void setDelegate(org.antlr.runtime.TokenSource delegate)
-
getDelegate
public org.antlr.runtime.TokenSource getDelegate()
-
nextToken
public org.antlr.runtime.Token nextToken()
- Specified by:
nextToken
in interfaceorg.antlr.runtime.TokenSource
-
getSourceName
public java.lang.String getSourceName()
- Specified by:
getSourceName
in interfaceorg.antlr.runtime.TokenSource
-
getRuleName
protected java.lang.String getRuleName(org.antlr.runtime.Token token)
-
getTokenType
protected int getTokenType(java.lang.String ruleNameWithRulePrefix)
-
getLexerRuleName
public java.lang.String getLexerRuleName(java.lang.String antlrTokenDef)
-
initializeTokenDefsFrom
@Inject public void initializeTokenDefsFrom(ITokenDefProvider provider)
-
splitToken
protected void splitToken(org.antlr.runtime.Token splitMe, ITokenAcceptor acceptor)
-
shouldSplitToken
protected abstract boolean shouldSplitToken(org.antlr.runtime.Token candidate)
-
doSplitToken
protected abstract void doSplitToken(org.antlr.runtime.Token splitMe, ITokenAcceptor acceptor)
-
createToken
protected org.antlr.runtime.CommonToken createToken(org.antlr.runtime.CommonToken prototype, java.lang.String text, java.lang.Integer charPosInLine, java.lang.Integer channel, java.lang.Integer start, java.lang.Integer stop, java.lang.Integer type)
Create a new token from the given prototype. Any argument besides the prototype is optional and will be ignored if its value isnull
.
-
-