Class ListTokenSource

  • All Implemented Interfaces:
    TokenSource

    public class ListTokenSource
    extends Object
    implements TokenSource
    Provides an implementation of TokenSource as a wrapper around a list of Token objects.

    If the final token in the list is an Token.EOF token, it will be used as the EOF token for every call to nextToken() after the end of the list is reached. Otherwise, an EOF token will be created.

    • Field Detail

      • tokens

        protected final List<? extends Token> tokens
        The wrapped collection of Token objects to return.
      • i

        protected int i
        The index into tokens of token to return by the next call to nextToken(). The end of the input is indicated by this value being greater than or equal to the number of items in tokens.
      • eofToken

        protected Token eofToken
        This field caches the EOF token for the token source.
    • Method Detail

      • getCharPositionInLine

        public int getCharPositionInLine()
        Get the index into the current line for the current position in the input stream. The first character on a line has position 0.
        Specified by:
        getCharPositionInLine in interface TokenSource
        Returns:
        The line number for the current position in the input stream, or -1 if the current token source does not track character positions.
      • nextToken

        public Token nextToken()
        Return a Token object from your input stream (usually a CharStream). Do not fail/return upon lexing error; keep chewing on the characters until you get a good one; errors are not passed through to the parser.
        Specified by:
        nextToken in interface TokenSource
      • getLine

        public int getLine()
        Get the line number for the current position in the input stream. The first line in the input is line 1.
        Specified by:
        getLine in interface TokenSource
        Returns:
        The line number for the current position in the input stream, or 0 if the current token source does not track line numbers.
      • getInputStream

        public CharStream getInputStream()
        Get the CharStream from which this token source is currently providing tokens.
        Specified by:
        getInputStream in interface TokenSource
        Returns:
        The CharStream associated with the current position in the input, or null if no input stream is available for the token source.