Package org.apache.nifi.controller.queue

Interface FlowFileQueue

All Known Subinterfaces:
LoadBalancedFlowFileQueue
public interface FlowFileQueue

  • Method Details

    • getIdentifier

      String getIdentifier()
      Returns:
      the unique identifier for this FlowFileQueue

    • getPriorities

      List<FlowFilePrioritizer> getPriorities()
      Returns:
      list of processing priorities for this queue

    • recoverSwappedFlowFiles

      SwapSummary recoverSwappedFlowFiles()
      Reads any Swap Files that belong to this queue and returns a summary of what is swapped out. This will be called only during NiFi startup as an initialization step. This method is then responsible for returning a FlowFileSummary of the FlowFiles that are swapped out, or null if no FlowFiles are swapped out for this queue.
      Returns:
      a SwapSummary that describes the FlowFiles that exist in the queue but are swapped out.

    • purgeSwapFiles

      void purgeSwapFiles()
      Destroys any Swap Files that exist for this queue without updating the FlowFile Repository or Provenance Repository. This is done only on startup in the case of non-persistent repositories. In the case of non-persistent repositories, we may still have Swap Files because we may still need to overflow the FlowFiles from heap onto disk, even though we don't want to keep the FlowFiles on restart.

    • setPriorities

      void setPriorities(List<FlowFilePrioritizer> newPriorities)
      Resets the comparator used by this queue to maintain order.
      Parameters:
      newPriorities - the ordered list of prioritizers to use to determine order within this queue.
      Throws:
      NullPointerException - if arg is null

    • setBackPressureObjectThreshold

      void setBackPressureObjectThreshold(long maxQueueSize)
      Establishes this queue's preferred maximum work load.
      Parameters:
      maxQueueSize - the maximum number of flow files this processor recommends having in its work queue at any one time

    • getBackPressureObjectThreshold

      long getBackPressureObjectThreshold()
      Returns:
      maximum number of flow files that should be queued up at any one time

    • setBackPressureDataSizeThreshold

      void setBackPressureDataSizeThreshold(String maxDataSize)
      Parameters:
      maxDataSize - Establishes this queue's preferred maximum data size.

    • getBackPressureDataSizeThreshold

      String getBackPressureDataSizeThreshold()
      Returns:
      maximum data size that should be queued up at any one time

    • size

      QueueSize size()

    • getTotalQueuedDuration

      long getTotalQueuedDuration(long fromTimestamp)
      Parameters:
      fromTimestamp - The timestamp in milliseconds from which to calculate durations. This will typically be the current timestamp.
      Returns:
      the sum in milliseconds of how long all FlowFiles within this queue have currently been in this queue.

    • getMinLastQueueDate

      long getMinLastQueueDate()
      Returns:
      The minimum lastQueueDate in milliseconds of all FlowFiles currently enqueued. If no FlowFile is enqueued, this returns 0.

    • isEmpty

      boolean isEmpty()
      Returns:
      true if no items queue; false otherwise

    • getFlowFileAvailability

      FlowFileAvailability getFlowFileAvailability()
      Returns:
      the FlowFile Availability for this queue

    • isActiveQueueEmpty

      boolean isActiveQueueEmpty()
      Returns:
      true if the queue is empty or contains only FlowFiles that already are being processed by others, false if the queue contains at least one FlowFile that is available for processing, regardless of whether that FlowFile(s) is in-memory or swapped out.

    • acknowledge

      void acknowledge(FlowFileRecord flowFile)

    • acknowledge

      void acknowledge(Collection<FlowFileRecord> flowFiles)

    • isUnacknowledgedFlowFile

      boolean isUnacknowledgedFlowFile()
      Returns:
      true if at least one FlowFile is unacknowledged, false if all FlowFiles that have been dequeued have been acknowledged

    • isFull

      boolean isFull()
      Returns:
      true if maximum queue size has been reached or exceeded; false otherwise

    • put

      void put(FlowFileRecord file)
      places the given file into the queue
      Parameters:
      file - to place into queue

    • putAll

      void putAll(Collection<FlowFileRecord> files)
      places the given files into the queue
      Parameters:
      files - to place into queue

    • poll

      FlowFileRecord poll(Set<FlowFileRecord> expiredRecords, PollStrategy pollStrategy)
      Parameters:
      expiredRecords - expired records
      pollStrategy - strategy of polling
      Returns:
      the next flow file on the queue; null if empty

    • poll

      FlowFileRecord poll(Set<FlowFileRecord> expiredRecords)

    • poll

      List<FlowFileRecord> poll(int maxResults, Set<FlowFileRecord> expiredRecords, PollStrategy pollStrategy)
      Parameters:
      maxResults - limits how many results can be polled
      expiredRecords - for expired records
      pollStrategy - strategy of polling
      Returns:
      the next flow files on the queue up to the max results; null if empty

    • poll

      List<FlowFileRecord> poll(int maxResults, Set<FlowFileRecord> expiredRecords)

    • poll

      List<FlowFileRecord> poll(FlowFileFilter filter, Set<FlowFileRecord> expiredRecords, PollStrategy pollStrategy)

    • poll

      List<FlowFileRecord> poll(FlowFileFilter filter, Set<FlowFileRecord> expiredRecords)

    • getFlowFileExpiration

      String getFlowFileExpiration()

    • getFlowFileExpiration

      long getFlowFileExpiration(TimeUnit timeUnit)

    • setFlowFileExpiration

      void setFlowFileExpiration(String flowExpirationPeriod)

    • dropFlowFiles

      DropFlowFileStatus dropFlowFiles(String requestIdentifier, String requestor)
      Initiates a request to drop all FlowFiles in this queue. This method returns a DropFlowFileStatus that can be used to determine the current state of the request. Additionally, the DropFlowFileStatus provides a request identifier that can then be passed to the getDropFlowFileStatus(String) and cancelDropFlowFileRequest(String) methods in order to obtain the status later or cancel a request
      Parameters:
      requestIdentifier - the identifier of the Drop FlowFile Request
      requestor - the entity that is requesting that the FlowFiles be dropped; this will be included in the Provenance Events that are generated.
      Returns:
      the status of the drop request.

    • getDropFlowFileStatus

      DropFlowFileStatus getDropFlowFileStatus(String requestIdentifier)
      Returns the current status of a Drop FlowFile Request that was initiated via the dropFlowFiles(String, String) method that has the given identifier
      Parameters:
      requestIdentifier - the identifier of the Drop FlowFile Request
      Returns:
      the status for the request with the given identifier, or null if no request status exists with that identifier

    • cancelDropFlowFileRequest

      DropFlowFileStatus cancelDropFlowFileRequest(String requestIdentifier)
      Cancels the request to drop FlowFiles that has the given identifier. After this method is called, the request will no longer be known by this queue, so subsequent calls to getDropFlowFileStatus(String) or cancelDropFlowFileRequest(String) will return null
      Parameters:
      requestIdentifier - the identifier of the Drop FlowFile Request
      Returns:
      the status for the request with the given identifier after it has been canceled, or null if no request status exists with that identifier

    • listFlowFiles

      ListFlowFileStatus listFlowFiles(String requestIdentifier, int maxResults)

      Initiates a request to obtain a listing of FlowFiles in this queue. This method returns a ListFlowFileStatus that can be used to obtain information about the FlowFiles that exist within the queue. Additionally, the ListFlowFileStatus provides a request identifier that can then be passed to the getListFlowFileStatus(String). The listing of FlowFiles will be returned ordered by the position of the FlowFile in the queue.

      Note that if maxResults is larger than the size of the "active queue" (i.e., the un-swapped queued, FlowFiles that are swapped out will not be returned.)

      Parameters:
      requestIdentifier - the identifier of the List FlowFile Request
      maxResults - the maximum number of FlowFileSummary objects to add to the ListFlowFileStatus
      Returns:
      the status for the request
      Throws:
      IllegalStateException - if either the source or the destination of the connection to which this queue belongs is currently running.

    • getListFlowFileStatus

      ListFlowFileStatus getListFlowFileStatus(String requestIdentifier)
      Returns the current status of a List FlowFile Request that was initiated via the listFlowFiles(String, int) method that has the given identifier
      Parameters:
      requestIdentifier - the identifier of the Drop FlowFile Request
      Returns:
      the current status of the List FlowFile Request with the given identifier or null if no request status exists with that identifier

    • cancelListFlowFileRequest

      ListFlowFileStatus cancelListFlowFileRequest(String requestIdentifier)
      Cancels the request to list FlowFiles that has the given identifier. After this method is called, the request will no longer be known by this queue, so subsequent calls to getListFlowFileStatus(String) or cancelListFlowFileRequest(String) will return null
      Parameters:
      requestIdentifier - the identifier of the Drop FlowFile Request
      Returns:
      the current status of the List FlowFile Request with the given identifier or null if no request status exists with that identifier

    • getFlowFile

      FlowFileRecord getFlowFile(String flowFileUuid) throws IOException
      Returns the FlowFile with the given UUID or null if no FlowFile can be found in this queue with the given UUID
      Parameters:
      flowFileUuid - the UUID of the FlowFile to retrieve
      Returns:
      the FlowFile with the given UUID or null if no FlowFile can be found in this queue with the given UUID
      Throws:
      IOException - if unable to read FlowFiles that are stored on some external device

    • verifyCanList

      void verifyCanList() throws IllegalStateException
      Ensures that a listing can be performed on the queue
      Throws:
      IllegalStateException - if the queue is not in a state in which a listing can be performed

    • getQueueDiagnostics

      QueueDiagnostics getQueueDiagnostics()
      Returns diagnostic information about the queue

    • lock

      void lock()

    • unlock

      void unlock()

    • setLoadBalanceStrategy

      void setLoadBalanceStrategy(LoadBalanceStrategy strategy, String partitioningAttribute)

    • offloadQueue

      void offloadQueue()
      Offloads the flowfiles in the queue to other nodes. This disables the queue from partition flowfiles locally.

      This operation is a no-op if the node that contains this queue is not in a cluster.

    • resetOffloadedQueue

      void resetOffloadedQueue()
      Resets a queue that has previously been offloaded. This allows the queue to partition flowfiles locally, and has no other effect on processors or remote process groups.

      This operation is a no-op if the queue is not currently offloaded or the node that contains this queue is not clustered.

    • getLoadBalanceStrategy

      LoadBalanceStrategy getLoadBalanceStrategy()

    • setLoadBalanceCompression

      void setLoadBalanceCompression(LoadBalanceCompression compression)

    • getLoadBalanceCompression

      LoadBalanceCompression getLoadBalanceCompression()

    • getPartitioningAttribute

      String getPartitioningAttribute()

    • startLoadBalancing

      void startLoadBalancing()

    • stopLoadBalancing

      void stopLoadBalancing()

    • isActivelyLoadBalancing

      boolean isActivelyLoadBalancing()
      Returns:
      true if the queue is actively transferring data to another node, false otherwise