Class TokenParser


  • public class TokenParser
    extends java.lang.Object
    The Class TokenParser.
    • Constructor Summary

      Constructors 
      Constructor Description
      TokenParser()  
    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      static Token[] makeTokens​(java.lang.String text, boolean tokenize)
      Convert the given string into tokens.
      static Token[] parse​(java.lang.String text)
      Returns an array of tokens that contains tokenized string.
      static Token[] parse​(java.lang.String text, boolean optimize)
      Returns an array of tokens that contains tokenized string.
      static java.util.List<Token[]> parseAsList​(java.lang.String text)  
      static java.util.Map<java.lang.String,​Token[]> parseAsMap​(java.lang.String text)  
      static java.lang.String toString​(Token[] tokens)
      Convert to string from the token array.
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Constructor Detail

      • TokenParser

        public TokenParser()
    • Method Detail

      • parse

        public static Token[] parse​(java.lang.String text)
        Returns an array of tokens that contains tokenized string.
        Parameters:
        text - the string to parse
        Returns:
        an array of tokens
      • parse

        public static Token[] parse​(java.lang.String text,
                                    boolean optimize)
        Returns an array of tokens that contains tokenized string.
        Parameters:
        text - the string to parse
        optimize - whether to optimize tokens
        Returns:
        an array of tokens
      • parseAsList

        public static java.util.List<Token[]> parseAsList​(java.lang.String text)
      • parseAsMap

        public static java.util.Map<java.lang.String,​Token[]> parseAsMap​(java.lang.String text)
      • makeTokens

        public static Token[] makeTokens​(java.lang.String text,
                                         boolean tokenize)
        Convert the given string into tokens.
        Parameters:
        text - the text
        tokenize - whether to tokenize
        Returns:
        the token[]
      • toString

        public static java.lang.String toString​(Token[] tokens)
        Convert to string from the token array.
        Parameters:
        tokens - the tokens
        Returns:
        the string