Class SafeTokenSource

  • All Implemented Interfaces:
    org.antlr.v4.runtime.TokenSource

    public class SafeTokenSource
    extends java.lang.Object
    implements org.antlr.v4.runtime.TokenSource
    This token source can wrap a lexer and if it asks for more than a maximum number of tokens the user can take some action, typically throw an exception to stop lexing. It tracks the maximum number per token channel, so we have 3 at the moment, and they will all be tracked. This is used to protect us from evil input. The lexer will eagerly try to find all tokens at times and certain inputs (directives butted together for example) will cause the lexer to keep doing work even though before the tokens are presented back to the parser and hence before it has a chance to stop work once too much as been done.
    • Constructor Summary

      Constructors 
      Constructor Description
      SafeTokenSource​(org.antlr.v4.runtime.TokenSource lexer, int maxTokens, int maxWhitespaceTokens, java.util.function.BiConsumer<java.lang.Integer,​org.antlr.v4.runtime.Token> whenMaxTokensExceeded)  
    • Constructor Detail

      • SafeTokenSource

        public SafeTokenSource​(org.antlr.v4.runtime.TokenSource lexer,
                               int maxTokens,
                               int maxWhitespaceTokens,
                               java.util.function.BiConsumer<java.lang.Integer,​org.antlr.v4.runtime.Token> whenMaxTokensExceeded)
    • Method Detail

      • nextToken

        public org.antlr.v4.runtime.Token nextToken()
        Specified by:
        nextToken in interface org.antlr.v4.runtime.TokenSource
      • getLine

        public int getLine()
        Specified by:
        getLine in interface org.antlr.v4.runtime.TokenSource
      • getCharPositionInLine

        public int getCharPositionInLine()
        Specified by:
        getCharPositionInLine in interface org.antlr.v4.runtime.TokenSource
      • getInputStream

        public org.antlr.v4.runtime.CharStream getInputStream()
        Specified by:
        getInputStream in interface org.antlr.v4.runtime.TokenSource
      • getSourceName

        public java.lang.String getSourceName()
        Specified by:
        getSourceName in interface org.antlr.v4.runtime.TokenSource
      • setTokenFactory

        public void setTokenFactory​(org.antlr.v4.runtime.TokenFactory<?> factory)
        Specified by:
        setTokenFactory in interface org.antlr.v4.runtime.TokenSource
      • getTokenFactory

        public org.antlr.v4.runtime.TokenFactory<?> getTokenFactory()
        Specified by:
        getTokenFactory in interface org.antlr.v4.runtime.TokenSource