org.apache.flink.runtime.operators

Class AbstractOuterJoinDriver<IT1,IT2,OT>

java.lang.Object

org.apache.flink.runtime.operators.AbstractOuterJoinDriver<IT1,IT2,OT>

All Implemented Interfaces:

Driver<org.apache.flink.api.common.functions.FlatJoinFunction<IT1,IT2,OT>,OT>

Direct Known Subclasses:

FullOuterJoinDriver, LeftOuterJoinDriver, RightOuterJoinDriver

public abstract class AbstractOuterJoinDriver<IT1,IT2,OT>
extends Object
implements Driver<org.apache.flink.api.common.functions.FlatJoinFunction<IT1,IT2,OT>,OT>

The abstract outer join driver implements the logic of an outer join operator at runtime. It instantiates a sort-merge based strategy to find joining pairs of records or joining records with null depending on the outer join type.

See Also:

FlatJoinFunction

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.slf4j.Logger	LOG
protected JoinTaskIterator<IT1,IT2,OT>	outerJoinIterator
protected boolean	running
protected TaskContext<org.apache.flink.api.common.functions.FlatJoinFunction<IT1,IT2,OT>,OT>	taskContext

Constructor Summary

Constructors

Constructor and Description
AbstractOuterJoinDriver()

Method Summary

Modifier and Type	Method and Description
void	cancel() This method is invoked when the driver must aborted in mid processing.
void	cleanup() This method is invoked in any case (clean termination and exception) at the end of the tasks operation.
protected abstract JoinTaskIterator<IT1,IT2,OT>	getNonReusingOuterJoinIterator(DriverStrategy driverStrategy, org.apache.flink.util.MutableObjectIterator<IT1> in1, org.apache.flink.util.MutableObjectIterator<IT2> in2, org.apache.flink.api.common.typeutils.TypeSerializer<IT1> serializer1, org.apache.flink.api.common.typeutils.TypeComparator<IT1> comparator1, org.apache.flink.api.common.typeutils.TypeSerializer<IT2> serializer2, org.apache.flink.api.common.typeutils.TypeComparator<IT2> comparator2, org.apache.flink.api.common.typeutils.TypePairComparatorFactory<IT1,IT2> pairComparatorFactory, MemoryManager memoryManager, IOManager ioManager, double driverMemFraction)
int	getNumberOfDriverComparators() Gets the number of comparators required for this driver.
int	getNumberOfInputs() Gets the number of inputs that the task has.
protected abstract JoinTaskIterator<IT1,IT2,OT>	getReusingOuterJoinIterator(DriverStrategy driverStrategy, org.apache.flink.util.MutableObjectIterator<IT1> in1, org.apache.flink.util.MutableObjectIterator<IT2> in2, org.apache.flink.api.common.typeutils.TypeSerializer<IT1> serializer1, org.apache.flink.api.common.typeutils.TypeComparator<IT1> comparator1, org.apache.flink.api.common.typeutils.TypeSerializer<IT2> serializer2, org.apache.flink.api.common.typeutils.TypeComparator<IT2> comparator2, org.apache.flink.api.common.typeutils.TypePairComparatorFactory<IT1,IT2> pairComparatorFactory, MemoryManager memoryManager, IOManager ioManager, double driverMemFraction)
Class<org.apache.flink.api.common.functions.FlatJoinFunction<IT1,IT2,OT>>	getStubType() Gets the class of the stub type that is run by this task.
void	prepare() This method is called before the user code is opened.
void	run() The main operation method of the task.
void	setup(TaskContext<org.apache.flink.api.common.functions.FlatJoinFunction<IT1,IT2,OT>,OT> context)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

protected static final org.slf4j.Logger LOG

taskContext

protected TaskContext<org.apache.flink.api.common.functions.FlatJoinFunction<IT1,IT2,OT>,OT> taskContext

outerJoinIterator

protected volatile JoinTaskIterator<IT1,IT2,OT> outerJoinIterator

running

protected volatile boolean running

Constructor Detail

AbstractOuterJoinDriver

public AbstractOuterJoinDriver()

Method Detail

setup

public void setup(TaskContext<org.apache.flink.api.common.functions.FlatJoinFunction<IT1,IT2,OT>,OT> context)

Specified by:

setup in interface Driver<org.apache.flink.api.common.functions.FlatJoinFunction<IT1,IT2,OT>,OT>

getNumberOfInputs

public int getNumberOfInputs()

Description copied from interface: Driver

Gets the number of inputs that the task has.

Specified by:

getNumberOfInputs in interface Driver<org.apache.flink.api.common.functions.FlatJoinFunction<IT1,IT2,OT>,OT>

Returns:

The number of inputs.

getStubType

public Class<org.apache.flink.api.common.functions.FlatJoinFunction<IT1,IT2,OT>> getStubType()

Description copied from interface: Driver

Gets the class of the stub type that is run by this task. For example, a MapTask should return MapFunction.class.

Specified by:

getStubType in interface Driver<org.apache.flink.api.common.functions.FlatJoinFunction<IT1,IT2,OT>,OT>

Returns:

The class of the stub type run by the task.

getNumberOfDriverComparators

public int getNumberOfDriverComparators()

Description copied from interface: Driver

Gets the number of comparators required for this driver.

Specified by:

getNumberOfDriverComparators in interface Driver<org.apache.flink.api.common.functions.FlatJoinFunction<IT1,IT2,OT>,OT>

Returns:

The number of comparators required for this driver.

prepare

public void prepare()
             throws Exception

Description copied from interface: Driver

This method is called before the user code is opened. An exception thrown by this method signals failure of the task.

Specified by:

prepare in interface Driver<org.apache.flink.api.common.functions.FlatJoinFunction<IT1,IT2,OT>,OT>

Throws:

Exception - Exceptions may be forwarded and signal task failure.

run

public void run()
         throws Exception

Description copied from interface: Driver

The main operation method of the task. It should call the user code with the data subsets until the input is depleted.

Specified by:

run in interface Driver<org.apache.flink.api.common.functions.FlatJoinFunction<IT1,IT2,OT>,OT>

Throws:

Exception - Any exception thrown by this method signals task failure. Because exceptions in the user code typically signal situations where this instance in unable to proceed, exceptions from the user code should be forwarded.

cleanup

public void cleanup()
             throws Exception

Description copied from interface: Driver

This method is invoked in any case (clean termination and exception) at the end of the tasks operation.

Specified by:

cleanup in interface Driver<org.apache.flink.api.common.functions.FlatJoinFunction<IT1,IT2,OT>,OT>

Throws:

Exception - Exceptions may be forwarded.

cancel

public void cancel()

Description copied from interface: Driver

This method is invoked when the driver must aborted in mid processing. It is invoked asynchronously by a different thread.

Specified by:

cancel in interface Driver<org.apache.flink.api.common.functions.FlatJoinFunction<IT1,IT2,OT>,OT>

getReusingOuterJoinIterator

protected abstract JoinTaskIterator<IT1,IT2,OT> getReusingOuterJoinIterator(DriverStrategy driverStrategy,
                                                                            org.apache.flink.util.MutableObjectIterator<IT1> in1,
                                                                            org.apache.flink.util.MutableObjectIterator<IT2> in2,
                                                                            org.apache.flink.api.common.typeutils.TypeSerializer<IT1> serializer1,
                                                                            org.apache.flink.api.common.typeutils.TypeComparator<IT1> comparator1,
                                                                            org.apache.flink.api.common.typeutils.TypeSerializer<IT2> serializer2,
                                                                            org.apache.flink.api.common.typeutils.TypeComparator<IT2> comparator2,
                                                                            org.apache.flink.api.common.typeutils.TypePairComparatorFactory<IT1,IT2> pairComparatorFactory,
                                                                            MemoryManager memoryManager,
                                                                            IOManager ioManager,
                                                                            double driverMemFraction)
                                                                     throws Exception

Throws:

Exception

getNonReusingOuterJoinIterator

protected abstract JoinTaskIterator<IT1,IT2,OT> getNonReusingOuterJoinIterator(DriverStrategy driverStrategy,
                                                                               org.apache.flink.util.MutableObjectIterator<IT1> in1,
                                                                               org.apache.flink.util.MutableObjectIterator<IT2> in2,
                                                                               org.apache.flink.api.common.typeutils.TypeSerializer<IT1> serializer1,
                                                                               org.apache.flink.api.common.typeutils.TypeComparator<IT1> comparator1,
                                                                               org.apache.flink.api.common.typeutils.TypeSerializer<IT2> serializer2,
                                                                               org.apache.flink.api.common.typeutils.TypeComparator<IT2> comparator2,
                                                                               org.apache.flink.api.common.typeutils.TypePairComparatorFactory<IT1,IT2> pairComparatorFactory,
                                                                               MemoryManager memoryManager,
                                                                               IOManager ioManager,
                                                                               double driverMemFraction)
                                                                        throws Exception

Throws:

Exception