org.apache.spark.sql.catalyst.analysis
All the attributes that are used for this plan.
All the attributes that are used for this plan.
Returns true if this node and its children have already been gone through analysis and verification.
Returns true if this node and its children have already been gone through analysis and verification. Note that this is only an optimization used to avoid analyzing trees that have already been analyzed, and can be reset by transformations.
Returns the tree node at the specified number, used primarily for interactive debugging.
Returns the tree node at the specified number, used primarily for interactive debugging. Numbers for each node can be found in the numberedTreeString.
Note that this cannot return BaseType because logical plan's plan node might return physical plan for innerChildren, e.g. in-memory relation logical plan node has a reference to the physical plan node it is referencing.
Returns a string representing the arguments to this node, minus any children
Returns a string representing the arguments to this node, minus any children
Returns a 'scala code' representation of this TreeNode and its children.
Returns a 'scala code' representation of this TreeNode and its children. Intended for use
when debugging where the prettier toString function is obfuscating the actual structure. In the
case of 'pure' TreeNodes that only contain primitives and other TreeNodes, the result can be
pasted in the REPL to build an equivalent Tree.
Returns a plan where a best effort attempt has been made to transform this in a way
that preserves the result but removes cosmetic variations (case sensitivity, ordering for
commutative operations, expression id, etc.
Returns a plan where a best effort attempt has been made to transform this in a way
that preserves the result but removes cosmetic variations (case sensitivity, ordering for
commutative operations, expression id, etc.)
Plans where this.canonicalized == other.canonicalized will always evaluate to the same
result.
Some nodes should overwrite this to provide proper canonicalize logic.
Returns a Seq of the children of this node.
Returns true if all its children of this query plan have been resolved.
Returns true if all its children of this query plan have been resolved.
Returns a Seq containing the result of applying a partial function to all elements in this tree on which the function is defined.
Returns a Seq containing the result of applying a partial function to all elements in this tree on which the function is defined.
Finds and returns the first TreeNode of the tree for which the given partial function is defined (pre-order), and applies the partial function to it.
Finds and returns the first TreeNode of the tree for which the given partial function is defined (pre-order), and applies the partial function to it.
Returns a Seq containing the leaves in this tree.
Returns a Seq containing the leaves in this tree.
Computes Statistics for this plan.
Computes Statistics for this plan. The default implementation assumes the output cardinality is the product of all child plan's cardinality, i.e. applies in the case of cartesian joins.
LeafNodes must override this.
An ExpressionSet that contains invariants about the rows output by this operator.
An ExpressionSet that contains invariants about the rows output by this operator. For
example, if this set contains the expression a = 2 then that expression is guaranteed to
evaluate to true for all rows produced.
Returns all of the expressions present in this query plan operator.
Returns all of the expressions present in this query plan operator.
Faster version of equality which short-circuits when two treeNodes are the same instance.
Faster version of equality which short-circuits when two treeNodes are the same instance.
We don't just override Object.equals, as doing so prevents the scala compiler from
generating case class equals methods
Find the first TreeNode that satisfies the condition specified by f.
Find the first TreeNode that satisfies the condition specified by f.
The condition is recursively applied to this node and all of its children (pre-order).
Returns a Seq by applying a function to all nodes in this tree and using the elements of the resulting collections.
Returns a Seq by applying a function to all nodes in this tree and using the elements of the resulting collections.
Runs the given function on this node and then recursively on children.
Runs the given function recursively on children then on this node.
Appends the string represent of this node and its children to the given StringBuilder.
Appends the string represent of this node and its children to the given StringBuilder.
The i-th element in lastChildren indicates whether the ancestor of the current node at
depth i + 1 is the last child of its own parent node. The depth of the root node is 0, and
lastChildren for the root node should be empty.
Note that this traversal (numbering) order must be the same as getNodeNumbered.
Extracts the relevant constraints from a given set of constraints based on the attributes that appear in the outputSet.
All the nodes that should be shown as a inner nested tree of this node.
The set of all attributes that are input to this operator by its children.
The set of all attributes that are input to this operator by its children.
Invalidates the stats cache.
Invalidates the stats cache. See stats for more information.
Returns true if this subtree contains any streaming data sources.
Returns true if this subtree contains any streaming data sources.
Creates a copy of this type of tree node after a transformation.
Creates a copy of this type of tree node after a transformation. Must be overridden by child classes that have constructor arguments that are not present in the productIterator.
the new product arguments.
Returns a Seq containing the result of applying the given function to each node in this tree in a preorder traversal.
Returns a Seq containing the result of applying the given function to each node in this tree in a preorder traversal.
the function to be applied.
Returns a copy of this node where f has been applied to all the nodes children.
Returns a copy of this node where f has been applied to all the nodes children.
Apply a map function to each expression present in this query operator, and return a new query operator based on the mapped expressions.
Apply a map function to each expression present in this query operator, and return a new query operator based on the mapped expressions.
Efficient alternative to productIterator.map(f).toArray.
Efficient alternative to productIterator.map(f).toArray.
Returns the maximum number of rows that this plan may compute.
Returns the maximum number of rows that this plan may compute.
Any operator that a Limit can be pushed passed should override this function (e.g., Union). Any operator that can push through a Limit should override this function (e.g., Project).
Attributes that are referenced by expressions but not provided by this node's children.
Attributes that are referenced by expressions but not provided by this node's children. Subclasses should override this method if they produce attributes internally as it is used by assertions designed to prevent the construction of invalid plans.
Returns the name of this type of TreeNode.
Returns the name of this type of TreeNode. Defaults to the class name. Note that we remove the "Exec" suffix for physical operators here.
Returns a string representation of the nodes in this tree, where each operator is numbered.
Returns a string representation of the nodes in this tree, where each operator is numbered. The numbers can be used with TreeNode.apply to easily access specific subtrees.
The numbers are based on depth-first traversal of the tree (with innerChildren traversed first before children).
Args to the constructor that should be copied, but not transformed.
Args to the constructor that should be copied, but not transformed. These are appended to the transformed args automatically by makeCopy
Returns the set of attributes that are output by this node.
Returns the set of attributes that are output by this node.
Returns the tree node at the specified number, used primarily for interactive debugging.
Returns the tree node at the specified number, used primarily for interactive debugging. Numbers for each node can be found in the numberedTreeString.
This is a variant of apply that returns the node as BaseType (if the type matches).
Do some simple transformation on this plan before canonicalizing.
Do some simple transformation on this plan before canonicalizing. Implementations can override this method to provide customized canonicalize logic without rewriting the whole logic.
Prints out the schema in the tree format
Prints out the schema in the tree format
The set of all attributes that are produced by this node.
All Attributes that appear in expressions from this operator.
All Attributes that appear in expressions from this operator. Note that this set does not include attributes that are implicitly referenced by being passed through to the output tuple.
Refreshes (or invalidates) any metadata/data cached in the plan recursively.
Refreshes (or invalidates) any metadata/data cached in the plan recursively.
Performs attribute resolution given a name and a sequence of possible attributes.
Performs attribute resolution given a name and a sequence of possible attributes.
Optionally resolves the given strings to a NamedExpression based on the output of this LogicalPlan.
Optionally resolves the given strings to a NamedExpression based on the output of this
LogicalPlan. The attribute is expressed as string in the following form:
[scope].AttributeName.[nested].[fields]....
Resolves a given schema to concrete Attribute references in this query plan.
Resolves a given schema to concrete Attribute references in this query plan. This function should only be called on analyzed plans since it will throw AnalysisException for unresolved Attributes.
Optionally resolves the given strings to a NamedExpression using the input from all child nodes of this LogicalPlan.
Optionally resolves the given strings to a NamedExpression using the input from all child
nodes of this LogicalPlan. The attribute is expressed as
as string in the following form: [scope].AttributeName.[nested].[fields]....
Recursively transforms the expressions of a tree, skipping nodes that have already been analyzed.
Recursively transforms the expressions of a tree, skipping nodes that have already been analyzed.
Returns a copy of this node where rule has been recursively applied first to all of its
children and then itself (post-order).
Returns a copy of this node where rule has been recursively applied first to all of its
children and then itself (post-order). When rule does not apply to a given node, it is left
unchanged. This function is similar to transformUp, but skips sub-trees that have already
been marked as analyzed.
the function use to transform this nodes children
Given an attribute name, split it to name parts by dot, but
don't split the name parts quoted by backticks, for example,
ab.cd.
Given an attribute name, split it to name parts by dot, but
don't split the name parts quoted by backticks, for example,
ab.cd.efg should be split into two parts "ab.cd" and "efg".
Returns true when the given query plan will return the same results as this query plan.
Returns true when the given query plan will return the same results as this query plan.
Since its likely undecidable to generally determine if two given plans will produce the same results, it is okay for this function to return false, even if the results are actually the same. Such behavior will not affect correctness, only the application of performance enhancements like caching. However, it is not acceptable to return true if the results could possibly be different.
This function performs a modified version of equality that is tolerant of cosmetic differences like attribute naming and or expression id differences.
Returns the output schema in the tree format.
Returns the output schema in the tree format.
Returns a hashCode for the calculation performed by this plan.
Returns a hashCode for the calculation performed by this plan. Unlike the standard
hashCode, an attempt has been made to eliminate cosmetic differences.
ONE line description of this node.
A prefix string used when printing the plan.
A prefix string used when printing the plan.
We use "!" to indicate an invalid plan, and "'" to indicate an unresolved plan.
Returns the estimated statistics for the current logical plan node.
Returns the estimated statistics for the current logical plan node. Under the hood, this method caches the return value, which is computed based on the configuration passed in the first time. If the configuration changes, the cache can be invalidated by calling invalidateStatsCache().
The arguments that should be included in the arg string.
The arguments that should be included in the arg string. Defaults to the productIterator.
All the subqueries of current plan.
All the subqueries of current plan.
Returns a copy of this node where rule has been recursively applied to the tree.
Returns a copy of this node where rule has been recursively applied to the tree.
When rule does not apply to a given node it is left unchanged.
Users should not expect a specific directionality. If a specific directionality is needed,
transformDown or transformUp should be used.
the function use to transform this nodes children
Returns the result of running transformExpressions on this node and all its children.
Returns the result of running transformExpressions on this node and all its children.
Returns a copy of this node where rule has been recursively applied to it and all of its
children (pre-order).
Returns a copy of this node where rule has been recursively applied to it and all of its
children (pre-order). When rule does not apply to a given node it is left unchanged.
the function used to transform this nodes children
Runs transformExpressionsDown with rule on all expressions present
in this query operator.
Runs transformExpressionsDown with rule on all expressions present
in this query operator.
Users should not expect a specific directionality. If a specific directionality is needed,
transformExpressionsDown or transformExpressionsUp should be used.
the rule to be applied to every expression in this operator.
Runs transformDown with rule on all expressions present in this query operator.
Runs transformDown with rule on all expressions present in this query operator.
the rule to be applied to every expression in this operator.
Runs transformUp with rule on all expressions present in this query operator.
Runs transformUp with rule on all expressions present in this query operator.
the rule to be applied to every expression in this operator.
Returns a copy of this node where rule has been recursively applied first to all of its
children and then itself (post-order).
Returns a copy of this node where rule has been recursively applied first to all of its
children and then itself (post-order). When rule does not apply to a given node, it is left
unchanged.
the function use to transform this nodes children
Returns a string representation of the nodes in this tree
Returns a string representation of the nodes in this tree
This method can be overridden by any child class of QueryPlan to specify a set of constraints based on the given operator's constraint propagation logic.
This method can be overridden by any child class of QueryPlan to specify a set of constraints based on the given operator's constraint propagation logic. These constraints are then canonicalized and filtered automatically to contain only those attributes that appear in the outputSet.
See Canonicalize for more details.
ONE line description of this node with more information
ONE line description of this node with some suffix information
ONE line description of this node with some suffix information
Returns a copy of this node with the children replaced.
Returns a copy of this node with the children replaced. TODO: Validate somewhere (in debug mode?) that children are ordered correctly.
A table-valued function, e.g.
select * from range(10);