public abstract class AbstractMaterializedViewRule extends RelOptRule
Modifier and Type | Class and Description |
---|---|
private static class |
AbstractMaterializedViewRule.Edge
Edge for graph
|
private static class |
AbstractMaterializedViewRule.EquivalenceClasses
Class representing an equivalence class, i.e., a set of equivalent columns
|
private static class |
AbstractMaterializedViewRule.MatchModality
Complete, view partial, or query partial.
|
private static class |
AbstractMaterializedViewRule.MaterializedViewAggregateRule
Materialized view rewriting for aggregate
|
private static class |
AbstractMaterializedViewRule.MaterializedViewJoinRule
Materialized view rewriting for join
|
static class |
AbstractMaterializedViewRule.MaterializedViewOnlyAggregateRule
Rule that matches Aggregate.
|
static class |
AbstractMaterializedViewRule.MaterializedViewOnlyFilterRule
Rule that matches Filter.
|
static class |
AbstractMaterializedViewRule.MaterializedViewOnlyJoinRule
Rule that matches Join.
|
static class |
AbstractMaterializedViewRule.MaterializedViewProjectAggregateRule
Rule that matches Project on Aggregate.
|
static class |
AbstractMaterializedViewRule.MaterializedViewProjectFilterRule
Rule that matches Project on Filter.
|
static class |
AbstractMaterializedViewRule.MaterializedViewProjectJoinRule
Rule that matches Project on Join.
|
private static class |
AbstractMaterializedViewRule.NodeLineage
Expression lineage details.
|
private static class |
AbstractMaterializedViewRule.ViewPartialRewriting
View partitioning result
|
description, operands, relBuilderFactory
Modifier | Constructor and Description |
---|---|
protected |
AbstractMaterializedViewRule(RelOptRuleOperand operand,
RelBuilderFactory relBuilderFactory,
java.lang.String description,
boolean generateUnionRewriting,
HepProgram unionRewritingPullProgram,
boolean fastBailOut)
Creates a AbstractMaterializedViewRule.
|
Modifier and Type | Method and Description |
---|---|
private static boolean |
compensatePartial(java.util.Set<RexTableInputRef.RelTableRef> sourceTableRefs,
AbstractMaterializedViewRule.EquivalenceClasses sourceEC,
java.util.Set<RexTableInputRef.RelTableRef> targetTableRefs,
com.google.common.collect.Multimap<RexTableInputRef,RexTableInputRef> compensationEquiColumns)
It checks whether the target can be rewritten using the source even though the
source uses additional tables.
|
protected abstract AbstractMaterializedViewRule.ViewPartialRewriting |
compensateViewPartial(RelBuilder relBuilder,
RexBuilder rexBuilder,
RelMetadataQuery mq,
RelNode input,
Project topProject,
RelNode node,
java.util.Set<RexTableInputRef.RelTableRef> queryTableRefs,
AbstractMaterializedViewRule.EquivalenceClasses queryEC,
Project topViewProject,
RelNode viewNode,
java.util.Set<RexTableInputRef.RelTableRef> viewTableRefs)
It checks whether the query can be rewritten using the view even though the
query uses additional tables.
|
private static Pair<RexNode,RexNode> |
computeCompensationPredicates(RexBuilder rexBuilder,
RexSimplify simplify,
AbstractMaterializedViewRule.EquivalenceClasses sourceEC,
Pair<RexNode,RexNode> sourcePreds,
AbstractMaterializedViewRule.EquivalenceClasses targetEC,
Pair<RexNode,RexNode> targetPreds,
com.google.common.collect.BiMap<RexTableInputRef.RelTableRef,RexTableInputRef.RelTableRef> sourceToTargetTableMapping)
We check whether the predicates in the source are contained in the predicates
in the target.
|
protected abstract RelNode |
createUnion(RelBuilder relBuilder,
RexBuilder rexBuilder,
RelNode topProject,
RelNode unionInputQuery,
RelNode unionInputView)
If the view will be used in a union rewriting, this method is responsible for
generating the union and any other operator needed on top of it, e.g., a Project
operator.
|
private static com.google.common.collect.Multimap<java.lang.Integer,java.lang.Integer> |
extractPossibleMapping(java.util.List<java.util.Set<RexTableInputRef>> sourceEquivalenceClasses,
java.util.List<java.util.Set<RexTableInputRef>> targetEquivalenceClasses)
Given the source and target equivalence classes, it extracts the possible mappings
from each source equivalence class to each target equivalence class.
|
private static java.util.List<RexNode> |
extractReferences(RexBuilder rexBuilder,
RelNode node)
If the node is an Aggregate, it returns a list of references to the grouping columns.
|
private static RexNode |
generateEquivalenceClasses(RexBuilder rexBuilder,
AbstractMaterializedViewRule.EquivalenceClasses sourceEC,
AbstractMaterializedViewRule.EquivalenceClasses targetEC)
Given the equi-column predicates of the source and the target and the
computed equivalence classes, it extracts possible mappings between
the equivalence classes.
|
private static AbstractMaterializedViewRule.NodeLineage |
generateSwapColumnTableReferencesLineage(RexBuilder rexBuilder,
RelMetadataQuery mq,
RelNode node,
com.google.common.collect.BiMap<RexTableInputRef.RelTableRef,RexTableInputRef.RelTableRef> tableMapping,
AbstractMaterializedViewRule.EquivalenceClasses ec,
java.util.List<RexNode> nodeExprs)
It swaps the column references and then the table references of the input
expressions using the equivalence classes and the table mapping.
|
private static AbstractMaterializedViewRule.NodeLineage |
generateSwapTableColumnReferencesLineage(RexBuilder rexBuilder,
RelMetadataQuery mq,
RelNode node,
com.google.common.collect.BiMap<RexTableInputRef.RelTableRef,RexTableInputRef.RelTableRef> tableMapping,
AbstractMaterializedViewRule.EquivalenceClasses ec,
java.util.List<RexNode> nodeExprs)
It swaps the table references and then the column references of the input
expressions using the table mapping and the equivalence classes.
|
private static java.util.List<com.google.common.collect.BiMap<RexTableInputRef.RelTableRef,RexTableInputRef.RelTableRef>> |
generateTableMappings(com.google.common.collect.Multimap<RexTableInputRef.RelTableRef,RexTableInputRef.RelTableRef> multiMapTables)
It will flatten a multimap containing table references to table references,
producing all possible combinations of mappings.
|
protected abstract boolean |
isValidPlan(Project topProject,
RelNode node,
RelMetadataQuery mq) |
private static boolean |
isValidRelNodePlan(RelNode node,
RelMetadataQuery mq)
Currently we only support TableScan - Project - Filter - Inner Join
|
protected void |
perform(RelOptRuleCall call,
Project topProject,
RelNode node)
Rewriting logic is based on "Optimizing Queries Using Materialized Views:
A Practical, Scalable Solution" by Goldstein and Larson.
|
protected abstract Pair<RelNode,RelNode> |
pushFilterToOriginalViewPlan(RelBuilder builder,
RelNode topViewProject,
RelNode viewNode,
RexNode cond)
Once we create a compensation predicate, this method is responsible for pushing
the resulting filter through the view nodes.
|
private static RexNode |
replaceWithOriginalReferences(RexBuilder rexBuilder,
RelNode node,
AbstractMaterializedViewRule.NodeLineage nodeLineage,
RexNode exprToRewrite)
Given the input expression, it will replace (sub)expressions when possible
using the content of the mapping.
|
private static RexNode |
rewriteExpression(RexBuilder rexBuilder,
RelMetadataQuery mq,
RelNode targetNode,
RelNode node,
java.util.List<RexNode> nodeExprs,
com.google.common.collect.BiMap<RexTableInputRef.RelTableRef,RexTableInputRef.RelTableRef> tableMapping,
AbstractMaterializedViewRule.EquivalenceClasses ec,
boolean swapTableColumn,
RexNode exprToRewrite)
First, the method takes the node expressions
nodeExprs and swaps the table
and column references using the table mapping and the equivalence classes. |
private static java.util.List<RexNode> |
rewriteExpressions(RexBuilder rexBuilder,
RelMetadataQuery mq,
RelNode targetNode,
RelNode node,
java.util.List<RexNode> nodeExprs,
com.google.common.collect.BiMap<RexTableInputRef.RelTableRef,RexTableInputRef.RelTableRef> tableMapping,
AbstractMaterializedViewRule.EquivalenceClasses ec,
boolean swapTableColumn,
java.util.List<RexNode> exprsToRewrite)
First, the method takes the node expressions
nodeExprs and swaps the table
and column references using the table mapping and the equivalence classes. |
protected abstract RelNode |
rewriteQuery(RelBuilder relBuilder,
RexBuilder rexBuilder,
RexSimplify simplify,
RelMetadataQuery mq,
RexNode compensationColumnsEquiPred,
RexNode otherCompensationPred,
Project topProject,
RelNode node,
com.google.common.collect.BiMap<RexTableInputRef.RelTableRef,RexTableInputRef.RelTableRef> viewToQueryTableMapping,
AbstractMaterializedViewRule.EquivalenceClasses viewEC,
AbstractMaterializedViewRule.EquivalenceClasses queryEC)
If the view will be used in a union rewriting, this method is responsible for
rewriting the query branch of the union using the given compensation predicate.
|
protected abstract RelNode |
rewriteView(RelBuilder relBuilder,
RexBuilder rexBuilder,
RexSimplify simplify,
RelMetadataQuery mq,
AbstractMaterializedViewRule.MatchModality matchModality,
boolean unionRewriting,
RelNode input,
Project topProject,
RelNode node,
Project topViewProject,
RelNode viewNode,
com.google.common.collect.BiMap<RexTableInputRef.RelTableRef,RexTableInputRef.RelTableRef> queryToViewTableMapping,
AbstractMaterializedViewRule.EquivalenceClasses queryEC)
Rewrites the query using the given view query.
|
private static RexNode |
shuttleReferences(RexBuilder rexBuilder,
RexNode node,
Mapping mapping)
Replaces all the input references by the position in the
input column set.
|
private static RexNode |
shuttleReferences(RexBuilder rexBuilder,
RexNode expr,
com.google.common.collect.Multimap<RexNode,java.lang.Integer> exprsLineage)
Replaces all the possible sub-expressions by input references
to the input node.
|
private static RexNode |
shuttleReferences(RexBuilder rexBuilder,
RexNode expr,
com.google.common.collect.Multimap<RexNode,java.lang.Integer> exprsLineage,
RelNode node,
Mapping rewritingMapping)
Replaces all the possible sub-expressions by input references
to the input node.
|
private static Pair<RexNode,RexNode> |
splitPredicates(RexBuilder rexBuilder,
RexNode pred)
Classifies each of the predicates in the list into one of these two
categories:
1-l) column equality predicates, or
2-r) residual predicates, all the rest
|
any, convert, convert, convertList, convertOperand, convertOperand, equals, equals, getOperand, getOperands, getOutConvention, getOutTrait, hashCode, matches, none, onMatch, operand, operand, operand, operand, operand, operandJ, operandJ, some, toString, unordered
private static final CalciteLogger LOGGER
public static final AbstractMaterializedViewRule.MaterializedViewProjectFilterRule INSTANCE_PROJECT_FILTER
public static final AbstractMaterializedViewRule.MaterializedViewOnlyFilterRule INSTANCE_FILTER
public static final AbstractMaterializedViewRule.MaterializedViewProjectJoinRule INSTANCE_PROJECT_JOIN
public static final AbstractMaterializedViewRule.MaterializedViewOnlyJoinRule INSTANCE_JOIN
public static final AbstractMaterializedViewRule.MaterializedViewProjectAggregateRule INSTANCE_PROJECT_AGGREGATE
public static final AbstractMaterializedViewRule.MaterializedViewOnlyAggregateRule INSTANCE_AGGREGATE
protected final boolean generateUnionRewriting
protected final HepProgram unionRewritingPullProgram
protected final boolean fastBailOut
protected AbstractMaterializedViewRule(RelOptRuleOperand operand, RelBuilderFactory relBuilderFactory, java.lang.String description, boolean generateUnionRewriting, HepProgram unionRewritingPullProgram, boolean fastBailOut)
protected void perform(RelOptRuleCall call, Project topProject, RelNode node)
On the query side, rules matches a Project-node chain or node, where node is either an Aggregate or a Join. Subplan rooted at the node operator must be composed of one or more of the following operators: TableScan, Project, Filter, and Join.
For each join MV, we need to check the following:
In turn, for each aggregate MV, we need to check the following:
The rule contains multiple extensions compared to the original paper. One of them is the possibility of creating rewritings using Union operators, e.g., if the result of a query is partially contained in the materialized view.
protected abstract boolean isValidPlan(Project topProject, RelNode node, RelMetadataQuery mq)
protected abstract AbstractMaterializedViewRule.ViewPartialRewriting compensateViewPartial(RelBuilder relBuilder, RexBuilder rexBuilder, RelMetadataQuery mq, RelNode input, Project topProject, RelNode node, java.util.Set<RexTableInputRef.RelTableRef> queryTableRefs, AbstractMaterializedViewRule.EquivalenceClasses queryEC, Project topViewProject, RelNode viewNode, java.util.Set<RexTableInputRef.RelTableRef> viewTableRefs)
Rules implementing the method should follow different approaches depending on the operators they rewrite.
protected abstract RelNode rewriteQuery(RelBuilder relBuilder, RexBuilder rexBuilder, RexSimplify simplify, RelMetadataQuery mq, RexNode compensationColumnsEquiPred, RexNode otherCompensationPred, Project topProject, RelNode node, com.google.common.collect.BiMap<RexTableInputRef.RelTableRef,RexTableInputRef.RelTableRef> viewToQueryTableMapping, AbstractMaterializedViewRule.EquivalenceClasses viewEC, AbstractMaterializedViewRule.EquivalenceClasses queryEC)
If a rewriting can be produced, we return that rewriting. If it cannot be produced, we will return null.
protected abstract RelNode createUnion(RelBuilder relBuilder, RexBuilder rexBuilder, RelNode topProject, RelNode unionInputQuery, RelNode unionInputView)
protected abstract RelNode rewriteView(RelBuilder relBuilder, RexBuilder rexBuilder, RexSimplify simplify, RelMetadataQuery mq, AbstractMaterializedViewRule.MatchModality matchModality, boolean unionRewriting, RelNode input, Project topProject, RelNode node, Project topViewProject, RelNode viewNode, com.google.common.collect.BiMap<RexTableInputRef.RelTableRef,RexTableInputRef.RelTableRef> queryToViewTableMapping, AbstractMaterializedViewRule.EquivalenceClasses queryEC)
The input node is a Scan on the view table and possibly a compensation Filter on top. If a rewriting can be produced, we return that rewriting. If it cannot be produced, we will return null.
protected abstract Pair<RelNode,RelNode> pushFilterToOriginalViewPlan(RelBuilder builder, RelNode topViewProject, RelNode viewNode, RexNode cond)
The method will return a pair of nodes: the new top project on the left and the new node on the right.
private static java.util.List<RexNode> extractReferences(RexBuilder rexBuilder, RelNode node)
private static java.util.List<com.google.common.collect.BiMap<RexTableInputRef.RelTableRef,RexTableInputRef.RelTableRef>> generateTableMappings(com.google.common.collect.Multimap<RexTableInputRef.RelTableRef,RexTableInputRef.RelTableRef> multiMapTables)
private static boolean isValidRelNodePlan(RelNode node, RelMetadataQuery mq)
private static Pair<RexNode,RexNode> splitPredicates(RexBuilder rexBuilder, RexNode pred)
For each category, it creates the conjunction of the predicates. The result is an pair of RexNode objects corresponding to each category.
private static boolean compensatePartial(java.util.Set<RexTableInputRef.RelTableRef> sourceTableRefs, AbstractMaterializedViewRule.EquivalenceClasses sourceEC, java.util.Set<RexTableInputRef.RelTableRef> targetTableRefs, com.google.common.collect.Multimap<RexTableInputRef,RexTableInputRef> compensationEquiColumns)
If it can be rewritten, it returns true. Further, it inserts the missing equi-join
predicates in the input compensationEquiColumns
multimap if it is provided.
If it cannot be rewritten, it returns false.
private static Pair<RexNode,RexNode> computeCompensationPredicates(RexBuilder rexBuilder, RexSimplify simplify, AbstractMaterializedViewRule.EquivalenceClasses sourceEC, Pair<RexNode,RexNode> sourcePreds, AbstractMaterializedViewRule.EquivalenceClasses targetEC, Pair<RexNode,RexNode> targetPreds, com.google.common.collect.BiMap<RexTableInputRef.RelTableRef,RexTableInputRef.RelTableRef> sourceToTargetTableMapping)
If the containment is confirmed, we produce compensation predicates that need to be added to the target to produce the results in the source. Thus, if source and target expressions are equivalent, those predicates will be the true constant.
In turn, if containment cannot be confirmed, the method returns null.
private static RexNode generateEquivalenceClasses(RexBuilder rexBuilder, AbstractMaterializedViewRule.EquivalenceClasses sourceEC, AbstractMaterializedViewRule.EquivalenceClasses targetEC)
If there is no mapping, it returns null. If there is a exact match, it will return a compensation predicate that evaluates to true. Finally, if a compensation predicate needs to be enforced on top of the target to make the equivalences classes match, it returns that compensation predicate.
private static com.google.common.collect.Multimap<java.lang.Integer,java.lang.Integer> extractPossibleMapping(java.util.List<java.util.Set<RexTableInputRef>> sourceEquivalenceClasses, java.util.List<java.util.Set<RexTableInputRef>> targetEquivalenceClasses)
If any of the source equivalence classes cannot be mapped to a target equivalence class, it returns null.
private static RexNode rewriteExpression(RexBuilder rexBuilder, RelMetadataQuery mq, RelNode targetNode, RelNode node, java.util.List<RexNode> nodeExprs, com.google.common.collect.BiMap<RexTableInputRef.RelTableRef,RexTableInputRef.RelTableRef> tableMapping, AbstractMaterializedViewRule.EquivalenceClasses ec, boolean swapTableColumn, RexNode exprToRewrite)
nodeExprs
and swaps the table
and column references using the table mapping and the equivalence classes.
If swapTableColumn
is true, it swaps the table reference and then the column reference,
otherwise it swaps the column reference and then the table reference.
Then, the method will rewrite the input expression exprToRewrite
, replacing the
RexTableInputRef
by references to the positions in nodeExprs
.
The method will return the rewritten expression. If any of the expressions in the input expression cannot be mapped, it will return null.
private static java.util.List<RexNode> rewriteExpressions(RexBuilder rexBuilder, RelMetadataQuery mq, RelNode targetNode, RelNode node, java.util.List<RexNode> nodeExprs, com.google.common.collect.BiMap<RexTableInputRef.RelTableRef,RexTableInputRef.RelTableRef> tableMapping, AbstractMaterializedViewRule.EquivalenceClasses ec, boolean swapTableColumn, java.util.List<RexNode> exprsToRewrite)
nodeExprs
and swaps the table
and column references using the table mapping and the equivalence classes.
If swapTableColumn
is true, it swaps the table reference and then the column reference,
otherwise it swaps the column reference and then the table reference.
Then, the method will rewrite the input expressions exprsToRewrite
, replacing the
RexTableInputRef
by references to the positions in nodeExprs
.
The method will return the rewritten expressions. If any of the subexpressions in the input expressions cannot be mapped, it will return null.
private static AbstractMaterializedViewRule.NodeLineage generateSwapTableColumnReferencesLineage(RexBuilder rexBuilder, RelMetadataQuery mq, RelNode node, com.google.common.collect.BiMap<RexTableInputRef.RelTableRef,RexTableInputRef.RelTableRef> tableMapping, AbstractMaterializedViewRule.EquivalenceClasses ec, java.util.List<RexNode> nodeExprs)
private static AbstractMaterializedViewRule.NodeLineage generateSwapColumnTableReferencesLineage(RexBuilder rexBuilder, RelMetadataQuery mq, RelNode node, com.google.common.collect.BiMap<RexTableInputRef.RelTableRef,RexTableInputRef.RelTableRef> tableMapping, AbstractMaterializedViewRule.EquivalenceClasses ec, java.util.List<RexNode> nodeExprs)
private static RexNode replaceWithOriginalReferences(RexBuilder rexBuilder, RelNode node, AbstractMaterializedViewRule.NodeLineage nodeLineage, RexNode exprToRewrite)
private static RexNode shuttleReferences(RexBuilder rexBuilder, RexNode node, Mapping mapping)
private static RexNode shuttleReferences(RexBuilder rexBuilder, RexNode expr, com.google.common.collect.Multimap<RexNode,java.lang.Integer> exprsLineage)
private static RexNode shuttleReferences(RexBuilder rexBuilder, RexNode expr, com.google.common.collect.Multimap<RexNode,java.lang.Integer> exprsLineage, RelNode node, Mapping rewritingMapping)
Copyright © 2012–2019 The Apache Software Foundation. All rights reserved.