Package org.apache.flink.runtime.taskmanager

Class InputGateWithMetrics

java.lang.Object

org.apache.flink.runtime.io.network.partition.consumer.InputGate

org.apache.flink.runtime.io.network.partition.consumer.IndexedInputGate

org.apache.flink.runtime.taskmanager.InputGateWithMetrics

All Implemented Interfaces:

AutoCloseable, AvailabilityProvider, ChannelStateHolder, CheckpointableInput, PullingAsyncDataInput<BufferOrEvent>

public class InputGateWithMetrics
extends IndexedInputGate

This class wraps InputGate provided by shuffle service and it is mainly used for increasing general input metrics from TaskIOMetricGroup.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.runtime.io.network.partition.consumer.InputGate

InputGate.InputWithData<INPUT,DATA>

Nested classes/interfaces inherited from interface org.apache.flink.runtime.io.AvailabilityProvider

AvailabilityProvider.AvailabilityHelper

Nested classes/interfaces inherited from interface org.apache.flink.runtime.io.PullingAsyncDataInput

PullingAsyncDataInput.EndOfDataStatus

Field Summary

Fields inherited from class org.apache.flink.runtime.io.network.partition.consumer.InputGate

availabilityHelper, priorityAvailabilityHelper

Fields inherited from interface org.apache.flink.runtime.io.AvailabilityProvider

AVAILABLE

Constructor Summary

Constructors

Constructor	Description
InputGateWithMetrics(IndexedInputGate inputGate, org.apache.flink.metrics.Counter numBytesIn)

Method Summary

Modifier and Type	Method	Description
void	acknowledgeAllRecordsProcessed(InputChannelInfo channelInfo)
void	close()
void	finishReadRecoveredState()
CompletableFuture<?>	getAvailableFuture()
InputChannel	getChannel(int channelIndex)	Returns the channel of this gate.
int	getGateIndex()	Returns the index of this input gate.
Optional<BufferOrEvent>	getNext()	Blocking call waiting for next BufferOrEvent.
int	getNumberOfInputChannels()
CompletableFuture<?>	getPriorityEventAvailableFuture()	Notifies when a priority event has been enqueued.
CompletableFuture<Void>	getStateConsumedFuture()
List<InputChannelInfo>	getUnfinishedChannels()	Returns the list of channels that have not received EndOfPartitionEvent.
PullingAsyncDataInput.EndOfDataStatus	hasReceivedEndOfData()	Tells if we consumed all available data.
boolean	isFinished()
Optional<BufferOrEvent>	pollNext()	Poll the BufferOrEvent.
void	requestPartitions()
void	resumeConsumption(InputChannelInfo channelInfo)
void	resumeGateConsumption()
void	sendTaskEvent(TaskEvent event)
void	setChannelStateWriter(ChannelStateWriter channelStateWriter)	Injects the ChannelStateWriter.
void	setup()	Setup gate, potentially heavy-weight, blocking operation comparing to just creation.
void	triggerDebloating()

Methods inherited from class org.apache.flink.runtime.io.network.partition.consumer.IndexedInputGate

blockConsumption, checkpointStarted, checkpointStopped, convertToPriorityEvent, getInputGateIndex

Methods inherited from class org.apache.flink.runtime.io.network.partition.consumer.InputGate

getChannelInfos

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.io.AvailabilityProvider

isApproximatelyAvailable, isAvailable

Methods inherited from interface org.apache.flink.runtime.io.network.partition.consumer.CheckpointableInput

getChannelInfos

Constructor Detail

InputGateWithMetrics

public InputGateWithMetrics(IndexedInputGate inputGate,
                            org.apache.flink.metrics.Counter numBytesIn)

Method Detail

getAvailableFuture

public CompletableFuture<?> getAvailableFuture()

Specified by:

getAvailableFuture in interface AvailabilityProvider

Overrides:

getAvailableFuture in class InputGate

Returns:

a future that is completed if there are more records available. If there are more records available immediately, AvailabilityProvider.AVAILABLE should be returned. Previously returned not completed futures should become completed once there are more records available.

resumeGateConsumption

public void resumeGateConsumption()
                           throws IOException

Specified by:

resumeGateConsumption in class InputGate

Throws:

IOException

resumeConsumption

public void resumeConsumption(InputChannelInfo channelInfo)
                       throws IOException

Specified by:

resumeConsumption in interface CheckpointableInput

Specified by:

resumeConsumption in class InputGate

Throws:

IOException

acknowledgeAllRecordsProcessed

public void acknowledgeAllRecordsProcessed(InputChannelInfo channelInfo)
                                    throws IOException

Specified by:

acknowledgeAllRecordsProcessed in class InputGate

Throws:

IOException

getNumberOfInputChannels

public int getNumberOfInputChannels()

Specified by:

getNumberOfInputChannels in interface CheckpointableInput

Specified by:

getNumberOfInputChannels in class InputGate

getChannel

public InputChannel getChannel(int channelIndex)

Description copied from class: InputGate

Returns the channel of this gate.

Specified by:

getChannel in class InputGate

getGateIndex

public int getGateIndex()

Description copied from class: IndexedInputGate

Returns the index of this input gate. Only supported on

Specified by:

getGateIndex in class IndexedInputGate

getUnfinishedChannels

public List<InputChannelInfo> getUnfinishedChannels()

Description copied from class: IndexedInputGate

Returns the list of channels that have not received EndOfPartitionEvent.

Specified by:

getUnfinishedChannels in class IndexedInputGate

triggerDebloating

public void triggerDebloating()

Specified by:

triggerDebloating in class IndexedInputGate

isFinished

public boolean isFinished()

Specified by:

isFinished in interface PullingAsyncDataInput<BufferOrEvent>

Specified by:

isFinished in class InputGate

Returns:

true if is finished and for example end of input was reached, false otherwise.

hasReceivedEndOfData

public PullingAsyncDataInput.EndOfDataStatus hasReceivedEndOfData()

Description copied from interface: PullingAsyncDataInput

Tells if we consumed all available data.

Moreover it tells us the reason why there is no more data incoming. If any of the upstream subtasks finished because of the stop-with-savepoint --no-drain, we should not drain the input. See also StopMode.

setup

public void setup()
           throws IOException

Description copied from class: InputGate

Setup gate, potentially heavy-weight, blocking operation comparing to just creation.

Specified by:

setup in class InputGate

Throws:

IOException

getStateConsumedFuture

public CompletableFuture<Void> getStateConsumedFuture()

Specified by:

getStateConsumedFuture in class InputGate

requestPartitions

public void requestPartitions()
                       throws IOException

Specified by:

requestPartitions in class InputGate

Throws:

IOException

setChannelStateWriter

public void setChannelStateWriter(ChannelStateWriter channelStateWriter)

Description copied from interface: ChannelStateHolder

Injects the ChannelStateWriter. Must only be called once.

Specified by:

setChannelStateWriter in interface ChannelStateHolder

Overrides:

setChannelStateWriter in class InputGate

getNext

public Optional<BufferOrEvent> getNext()
                                throws IOException,
                                       InterruptedException

Description copied from class: InputGate

Blocking call waiting for next BufferOrEvent.

Note: It should be guaranteed that the previous returned buffer has been recycled before getting next one.

Specified by:

getNext in class InputGate

Returns:

Optional.empty() if InputGate.isFinished() returns true.

Throws:

IOException

InterruptedException

pollNext

public Optional<BufferOrEvent> pollNext()
                                 throws IOException,
                                        InterruptedException

Description copied from class: InputGate

Poll the BufferOrEvent.

Note: It should be guaranteed that the previous returned buffer has been recycled before polling next one.

Specified by:

pollNext in interface PullingAsyncDataInput<BufferOrEvent>

Specified by:

pollNext in class InputGate

Returns:

Optional.empty() if there is no data to return or if InputGate.isFinished() returns true.

Throws:

IOException

InterruptedException

sendTaskEvent

public void sendTaskEvent(TaskEvent event)
                   throws IOException

Specified by:

sendTaskEvent in class InputGate

Throws:

IOException

close

public void close()
           throws Exception

Throws:

Exception

getPriorityEventAvailableFuture

public CompletableFuture<?> getPriorityEventAvailableFuture()

Description copied from class: InputGate

Notifies when a priority event has been enqueued. If this future is queried from task thread, it is guaranteed that a priority event is available and retrieved through InputGate.getNext().

Overrides:

getPriorityEventAvailableFuture in class InputGate

finishReadRecoveredState

public void finishReadRecoveredState()
                              throws IOException

Specified by:

finishReadRecoveredState in class InputGate

Throws:

IOException