Package com.yahoo.prelude.query.parser

Class AbstractParser

    • Field Summary

      Fields 
      Modifier and Type Field Description
      protected int braceLevelURL
      The counter for braces in URLs, braces in URLs are accepted so long as they are balanced.
      protected ParserEnvironment environment  
      protected IndexFacts.Session indexFacts
      The IndexFacts.Session of this query
      protected com.yahoo.language.Language language
      The current language of this parser.
      protected com.yahoo.prelude.query.parser.AbstractParser.Submodes submodes
      The current submodes of this parser
      protected com.yahoo.prelude.query.parser.TokenPosition tokens  

    • Constructor Summary

      Constructors 
      Modifier Constructor Description
      protected AbstractParser​(ParserEnvironment environment)
      Creates a new instance of this class, storing the given ParserEnvironment for parse-time access to the environment.

    • Method Summary

      All Methods Static Methods Instance Methods Abstract Methods Concrete Methods 
      Modifier and Type Method Description
      protected java.lang.String normalize​(java.lang.String input)
      Unicode normalizes some piece of natural language text.
      QueryTree parse​(Parsable query)
      Parser the given Parsable, and returns a corresponding QueryTree.
      Item parse​(java.lang.String queryToParse, java.lang.String filterToParse, com.yahoo.language.Language parsingLanguage, IndexFacts.Session indexFacts, java.lang.String defaultIndexName)  
      protected abstract Item parseItems()  
      protected Item segment​(java.lang.String indexName, Token token, boolean quoted)
      Segments a token
      protected void setState​(com.yahoo.language.Language queryLanguage, IndexFacts.Session indexFacts)  
      static Item simplifyPhrases​(Item unwashed)
      Collapses single item phrases in the tree to the contained item.
      protected void tokenize​(java.lang.String query, java.lang.String defaultIndexName, IndexFacts.Session indexFacts, com.yahoo.language.Language language)
      Tokenizes the given string and initializes tokens with the found tokens.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • submodes

        protected com.yahoo.prelude.query.parser.AbstractParser.Submodes submodes
        The current submodes of this parser

      • language

        protected com.yahoo.language.Language language
        The current language of this parser. Used to decide whether and how to use the CJKSegmenter

      • indexFacts

        protected IndexFacts.Session indexFacts
        The IndexFacts.Session of this query

      • braceLevelURL

        protected int braceLevelURL
        The counter for braces in URLs, braces in URLs are accepted so long as they are balanced.

      • tokens

        protected final com.yahoo.prelude.query.parser.TokenPosition tokens

    • Constructor Detail

      • AbstractParser

        protected AbstractParser​(ParserEnvironment environment)
        Creates a new instance of this class, storing the given ParserEnvironment for parse-time access to the environment.
        Parameters:
        environment - the environment settings to attach to the Parser

    • Method Detail

      • parse

        public final QueryTree parse​(Parsable query)
        Description copied from interface: Parser
        Parser the given Parsable, and returns a corresponding QueryTree. If parsing fails without an exception, the contained root will be an instance of NullItem.
        Specified by:
        parse in interface Parser
        Parameters:
        query - the Parsable to parse
        Returns:
        the parsed QueryTree, never null

      • parse

        public final Item parse​(java.lang.String queryToParse,
                        java.lang.String filterToParse,
                        com.yahoo.language.Language parsingLanguage,
                        IndexFacts.Session indexFacts,
                        java.lang.String defaultIndexName)
        Specified by:
        parse in interface CustomParser

      • parseItems

        protected abstract Item parseItems()

      • normalize

        protected java.lang.String normalize​(java.lang.String input)
        Unicode normalizes some piece of natural language text. The chosen form is compatibility decomposition, canonical composition (NFKC).

      • setState

        protected void setState​(com.yahoo.language.Language queryLanguage,
                        IndexFacts.Session indexFacts)

      • tokenize

        protected void tokenize​(java.lang.String query,
                        java.lang.String defaultIndexName,
                        IndexFacts.Session indexFacts,
                        com.yahoo.language.Language language)
        Tokenizes the given string and initializes tokens with the found tokens.
        Parameters:
        query - the string to tokenize
        defaultIndexName - the name of the index to use as default
        indexFacts - resolved information about the index we are searching
        language - the language set for this query, or null if none

      • simplifyPhrases

        public static Item simplifyPhrases​(Item unwashed)
        Collapses single item phrases in the tree to the contained item.
        Parameters:
        unwashed - The item whose phrases to simplify.
        Returns:
        The simplified item.

      • segment

        protected Item segment​(java.lang.String indexName,
                       Token token,
                       boolean quoted)
        Segments a token
        Parameters:
        indexName - the index name which preceeded this token, or null if none
        token - the token to segment
        quoted - whether this segment is within quoted text
        Returns:
        the resulting item