Package com.globalmentor.io

Class ReaderTokenizer

  • All Implemented Interfaces:
    java.lang.Iterable<java.lang.String>, java.util.Iterator<java.lang.String>
    public class ReaderTokenizer
extends java.lang.Object
implements java.util.Iterator<java.lang.String>, java.lang.Iterable<java.lang.String>
    Tokenizes input from a reader, recognizing groups. All text within a group will be ignored when delimiting tokens, except that group delimiters are checked for matching. If a group has the same beginning and ending character, direct nesting of that group is not allowed, although other groups may be interspersed. This class does not return independent iterators.
    Author:
    Garret Wilson

    • Field Summary

      Fields 
      Modifier and Type Field Description
      protected static Characters DEFAULT_DELIMITERS
      The default delimiter characters: whitespace.
      protected static java.lang.String DEFAULT_GROUP_BEGINS
      The default beginning group characters: "([{".
      protected static java.lang.String DEFAULT_GROUP_ENDS
      The default ending group characters: ")]}".

    • Constructor Summary

      Constructors 
      Constructor Description
      ReaderTokenizer​(java.io.Reader reader)
      Reader constructor with default token delimiters and no group delimiters.
      ReaderTokenizer​(java.io.Reader reader, Characters delimiters)
      Token delimiter constructor with no group delimiters.
      ReaderTokenizer​(java.io.Reader reader, Characters delimiters, java.lang.String groupBegins, java.lang.String groupEnds)
      Delimiter and group constructor.

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      char getDelimiter()
      Returns the current delimiter encountered, which delimits the current token.
      Characters getDelimiters()  
      java.lang.String getGroupBegins()  
      protected int getGroupDepth()  
      java.lang.String getGroupEnds()  
      char getLastDelimiter()
      Returns the last delimiter encountered, which introduced the current token.
      protected java.io.Reader getReader()  
      boolean hasNext()  
      java.util.Iterator<java.lang.String> iterator()  
      java.lang.String next()  
      protected char peekGroup()
      Returns the character of the group currently on the stack.
      protected char popGroup()
      Pops a group from the stack and returns the group beginning character.
      java.lang.String primeToken()
      If there is no token primed, attempts to retrieve the next token.
      protected void pushGroup​(char groupBegin)
      Pushes a group onto the stack.
      void remove()
      Removes the last token from the underlying collection.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

      • Methods inherited from interface java.lang.Iterable

        forEach, spliterator

      • Methods inherited from interface java.util.Iterator

        forEachRemaining

    • Field Detail

      • DEFAULT_GROUP_BEGINS

        protected static final java.lang.String DEFAULT_GROUP_BEGINS
        The default beginning group characters: "([{".
        See Also:
        Constant Field Values

      • DEFAULT_GROUP_ENDS

        protected static final java.lang.String DEFAULT_GROUP_ENDS
        The default ending group characters: ")]}".
        See Also:
        Constant Field Values

    • Constructor Detail

      • ReaderTokenizer

        public ReaderTokenizer​(java.io.Reader reader)
        Reader constructor with default token delimiters and no group delimiters.
        Parameters:
        reader - The input characters to tokenize.
        See Also:
        DEFAULT_DELIMITERS

      • ReaderTokenizer

        public ReaderTokenizer​(java.io.Reader reader,
                       Characters delimiters)
        Token delimiter constructor with no group delimiters.
        Parameters:
        reader - The input characters to tokenize.
        delimiters - The delimiter characters.

      • ReaderTokenizer

        public ReaderTokenizer​(java.io.Reader reader,
                       Characters delimiters,
                       java.lang.String groupBegins,
                       java.lang.String groupEnds)
        Delimiter and group constructor.
        Parameters:
        reader - The input characters to tokenize.
        delimiters - The delimiter characters.
        groupBegins - The valid group beginning characters.
        groupEnds - The valid group ending characters, matching to beginning characters.

    • Method Detail

      • getReader

        protected java.io.Reader getReader()
        Returns:
        The source of the input characters.

      • getDelimiters

        public Characters getDelimiters()
        Returns:
        The characters delimiting tokens.

      • getGroupBegins

        public java.lang.String getGroupBegins()
        Returns:
        The valid group beginning characters.

      • getGroupEnds

        public java.lang.String getGroupEnds()
        Returns:
        The valid group ending characters, matching to beginning characters.

      • pushGroup

        protected void pushGroup​(char groupBegin)
        Pushes a group onto the stack.
        Parameters:
        groupBegin - The character representing the start of the group.

      • popGroup

        protected char popGroup()
                 throws java.util.EmptyStackException
        Pops a group from the stack and returns the group beginning character.
        Returns:
        The character used to begin the group.
        Throws:
        java.lang.StringIndexOutOfBoundsException
        java.util.EmptyStackException - if there are no groups left.

      • peekGroup

        protected char peekGroup()
                  throws java.util.EmptyStackException
        Returns the character of the group currently on the stack.
        Returns:
        The character used to begin the group.
        Throws:
        java.lang.StringIndexOutOfBoundsException
        java.util.EmptyStackException - if there are no groups left.

      • getGroupDepth

        protected int getGroupDepth()
        Returns:
        The number of nested groups currently being processed.

      • hasNext

        public boolean hasNext()
        Specified by:
        hasNext in interface java.util.Iterator<java.lang.String>
        Returns:
        true if there are more tokens.

      • next

        public java.lang.String next()
        Specified by:
        next in interface java.util.Iterator<java.lang.String>
        Returns:
        The next token available.
        Throws:
        java.util.NoSuchElementException - if there are no more tokens.

      • remove

        public void remove()
        Removes the last token from the underlying collection. This method is not supported, and an UnsupportedOperationException is always thrown.
        Specified by:
        remove in interface java.util.Iterator<java.lang.String>
        Throws:
        java.lang.UnsupportedOperationException - if the the remove operation is not supported .
        java.lang.IllegalStateException - if the next() method has not yet been called, or the remove method has already been called after the last call to the next method.

      • getLastDelimiter

        public char getLastDelimiter()
        Returns the last delimiter encountered, which introduced the current token. If no last delimiter has been encountered, Characters.UNDEFINED_CHAR is returned.
        Returns:
        The last delimiter encountered.
        See Also:
        getDelimiters()

      • getDelimiter

        public char getDelimiter()
        Returns the current delimiter encountered, which delimits the current token. If no delimiters have been encountered, or the end of the reader was reached, Characters.UNDEFINED_CHAR is returned.
        Returns:
        The current delimiter encountered.
        See Also:
        getDelimiters()

      • primeToken

        public java.lang.String primeToken()
        If there is no token primed, attempts to retrieve the next token.
        Returns:
        Either the token that was already primed or, if there is no token already, the token newly primed by this method. If no token is available, this method returns null.

      • iterator

        public java.util.Iterator<java.lang.String> iterator()
        Specified by:
        iterator in interface java.lang.Iterable<java.lang.String>
        Returns:
        An iterator over the tokens.