org.apache.drill.exec.planner.fragment

Class SimpleParallelizer

java.lang.Object

org.apache.drill.exec.planner.fragment.SimpleParallelizer

public class SimpleParallelizer
extends Object

The simple parallelizer determines the level of parallelization of a plan based on the cost of the underlying operations. It doesn't take into account system load or other factors. Based on the cost of the query, the parallelization for each major fragment will be determined. Once the amount of parallelization is done, assignment is done based on round robin assignment ordered by operator affinity (locality) to available execution Drillbits.

Constructor Summary

Constructors

Constructor and Description
SimpleParallelizer(long parallelizationThreshold, int maxWidthPerNode, int maxGlobalWidth, double affinityFactor)
SimpleParallelizer(QueryContext context)

Method Summary

Methods

Modifier and Type	Method and Description
QueryWorkUnit	getFragments(OptionList options, CoordinationProtos.DrillbitEndpoint foremanNode, UserBitShared.QueryId queryId, Collection<CoordinationProtos.DrillbitEndpoint> activeEndpoints, PhysicalPlanReader reader, Fragment rootFragment, UserSession session, BitControl.QueryContextInformation queryContextInfo) Generate a set of assigned fragments based on the provided fragment tree.
void	initFragmentWrappers(Fragment rootFragment, PlanningSet planningSet)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SimpleParallelizer

public SimpleParallelizer(QueryContext context)

SimpleParallelizer

public SimpleParallelizer(long parallelizationThreshold,
                  int maxWidthPerNode,
                  int maxGlobalWidth,
                  double affinityFactor)

Method Detail

getFragments

public QueryWorkUnit getFragments(OptionList options,
                         CoordinationProtos.DrillbitEndpoint foremanNode,
                         UserBitShared.QueryId queryId,
                         Collection<CoordinationProtos.DrillbitEndpoint> activeEndpoints,
                         PhysicalPlanReader reader,
                         Fragment rootFragment,
                         UserSession session,
                         BitControl.QueryContextInformation queryContextInfo)
                           throws ExecutionSetupException

Generate a set of assigned fragments based on the provided fragment tree. Do not allow parallelization stages to go beyond the global max width.

Parameters:

options - Option list

foremanNode - The driving/foreman node for this query. (this node)

queryId - The queryId for this query.

activeEndpoints - The list of endpoints to consider for inclusion in planning this query.

reader - Tool used to read JSON plans

rootFragment - The root node of the PhysicalPlan that we will be parallelizing.

session - UserSession of user who launched this query.

queryContextInfo - Info related to the context when query has started.

Returns:

The list of generated PlanFragment protobuf objects to be assigned out to the individual nodes.

Throws:

ExecutionSetupException

initFragmentWrappers

public void initFragmentWrappers(Fragment rootFragment,
                        PlanningSet planningSet)