Package org.apache.nifi.controller.queue

Interface LoadBalancedFlowFileQueue

All Superinterfaces:

FlowFileQueue

public interface LoadBalancedFlowFileQueue
extends FlowFileQueue

Method Summary

Modifier and Type	Method	Description
void	distributeToPartitions(Collection<FlowFileRecord> flowFiles)	Distributes the given FlowFiles to the appropriate partitions.
void	handleExpiredRecords(Collection<FlowFileRecord> flowFiles)	Handles updating the repositories for the given FlowFiles, which have been expired
boolean	isLocalPartitionFull()	Determines whether or not the local partition's size >= backpressure threshold
boolean	isPropagateBackpressureAcrossNodes()	There are times when we want to ensure that if a node in the cluster reaches the point where backpressure is engaged, that we honor that backpressure and do not attempt to load balance from a different node in the cluster to that node.
void	onAbort(Collection<FlowFileRecord> flowFiles)	Notifies the queue the a transaction containing the given FlowFiles was aborted
void	onTransfer(Collection<FlowFileRecord> flowFiles)	Notifies the queue that the given FlowFiles have been successfully transferred to another node
void	receiveFromPeer(Collection<FlowFileRecord> flowFiles)	Adds the given FlowFiles to this queue, as they have been received from another node in the cluster

Methods inherited from interface org.apache.nifi.controller.queue.FlowFileQueue

acknowledge, acknowledge, cancelDropFlowFileRequest, cancelListFlowFileRequest, dropFlowFiles, getBackPressureDataSizeThreshold, getBackPressureObjectThreshold, getDropFlowFileStatus, getFlowFile, getFlowFileAvailability, getFlowFileExpiration, getFlowFileExpiration, getIdentifier, getListFlowFileStatus, getLoadBalanceCompression, getLoadBalanceStrategy, getMinLastQueueDate, getPartitioningAttribute, getPriorities, getQueueDiagnostics, getTotalQueuedDuration, isActivelyLoadBalancing, isActiveQueueEmpty, isEmpty, isFull, isUnacknowledgedFlowFile, listFlowFiles, lock, offloadQueue, poll, poll, poll, poll, poll, poll, purgeSwapFiles, put, putAll, recoverSwappedFlowFiles, resetOffloadedQueue, setBackPressureDataSizeThreshold, setBackPressureObjectThreshold, setFlowFileExpiration, setLoadBalanceCompression, setLoadBalanceStrategy, setPriorities, size, startLoadBalancing, stopLoadBalancing, unlock, verifyCanList

Method Details

receiveFromPeer

void receiveFromPeer(Collection<FlowFileRecord> flowFiles)
              throws IllegalClusterStateException

Adds the given FlowFiles to this queue, as they have been received from another node in the cluster

Parameters:

flowFiles - the FlowFiles received from the peer

Throws:

IllegalClusterStateException - if the node is not currently in a state in which it can receive data from other nodes

distributeToPartitions

void distributeToPartitions(Collection<FlowFileRecord> flowFiles)

Distributes the given FlowFiles to the appropriate partitions. Unlike the FlowFileQueue.putAll(Collection) method, this does not alter the size of the FlowFile Queue itself, as it is intended only to place the FlowFiles into their appropriate partitions

Parameters:

flowFiles - the FlowFiles to distribute

onTransfer

void onTransfer(Collection<FlowFileRecord> flowFiles)

Notifies the queue that the given FlowFiles have been successfully transferred to another node

Parameters:

flowFiles - the FlowFiles that were transferred

onAbort

void onAbort(Collection<FlowFileRecord> flowFiles)

Notifies the queue the a transaction containing the given FlowFiles was aborted

Parameters:

flowFiles - the FlowFiles in the transaction

handleExpiredRecords

void handleExpiredRecords(Collection<FlowFileRecord> flowFiles)

Handles updating the repositories for the given FlowFiles, which have been expired

Parameters:

flowFiles - the expired FlowFiles

isPropagateBackpressureAcrossNodes

boolean isPropagateBackpressureAcrossNodes()

There are times when we want to ensure that if a node in the cluster reaches the point where backpressure is engaged, that we honor that backpressure and do not attempt to load balance from a different node in the cluster to that node. There are other times when we may want to push data to the remote node even though it has already reached its backpressure threshold. This method indicates whether or not we want to propagate that backpressure indicator across the cluster.

Returns:

true if backpressure on Node A should prevent Node B from sending to it, false if Node B should send to Node A even when backpressure is engaged on Node A.

isLocalPartitionFull

boolean isLocalPartitionFull()

Determines whether or not the local partition's size >= backpressure threshold

Returns:

true if the number of FlowFiles or total size of FlowFiles in the local partition alone meets or exceeds the backpressure threshold, false otherwise.