Package org.apache.nifi.controller.repository

Interface FlowFileSwapManager

public interface FlowFileSwapManager
Defines a mechanism by which FlowFiles can be move into external storage or memory so that they can be removed from the Java heap and vice-versa

  • Method Details

    • initialize

      void initialize(SwapManagerInitializationContext initializationContext)
      Initializes the Swap Manager, providing a SwapManagerInitializationContext so that the Swap Manager has access to all of the components necessary to perform its functions
      Parameters:
      initializationContext - the context the provides the swap manager with access to the resources that it needs to perform its functions

    • swapOut

      String swapOut(List<FlowFileRecord> flowFiles, FlowFileQueue flowFileQueue, String partitionName) throws IOException
      Swaps out the given FlowFiles that belong to the queue with the given identifier.
      Parameters:
      flowFiles - the FlowFiles to swap out to external storage
      flowFileQueue - the queue that the FlowFiles belong to
      partitionName - the name of the partition within the queue, or null if the queue is not partitioned
      Returns:
      the location of the externally stored swap file
      Throws:
      IOException - if unable to swap the FlowFiles out

    • peek

      SwapContents peek(String swapLocation, FlowFileQueue flowFileQueue) throws IncompleteSwapFileException, IOException
      Recovers the FlowFiles from the swap file that lives at the given location. This action provides a view of the FlowFiles but does not actively swap them in, meaning that the swap file at the given location remains in that location and the FlowFile Repository is not updated.
      Parameters:
      swapLocation - the location of the swap file
      flowFileQueue - the queue that the FlowFiles belong to
      Returns:
      a SwapContents that includes the FlowFiles that live at the given swap location
      Throws:
      IOException - if unable to recover the FlowFiles from the given location
      IncompleteSwapFileException

    • swapIn

      SwapContents swapIn(String swapLocation, FlowFileQueue flowFileQueue) throws IncompleteSwapFileException, IOException
      Recovers the FlowFiles from the swap file that lives at the given location and belongs to the FlowFile Queue with the given identifier. The FlowFile Repository is then updated and the swap file is permanently removed from the external storage
      Parameters:
      swapLocation - the location of the swap file
      flowFileQueue - the queue to which the FlowFiles belong
      Returns:
      a SwapContents that includes FlowFiles that are stored in the given location
      Throws:
      IOException - if unable to recover the FlowFiles from the given location or update the FlowFileRepository
      IncompleteSwapFileException

    • recoverSwapLocations

      List<String> recoverSwapLocations(FlowFileQueue flowFileQueue, String partitionName) throws IOException
      Determines swap files that exist for the given FlowFileQueue
      Parameters:
      flowFileQueue - the queue for which the FlowFiles should be recovered
      partitionName - the partition within the FlowFileQueue to recover, or null if the queue is not partitioned
      Returns:
      all swap locations that have been identified for the given queue, in the order that they should be swapped back in
      Throws:
      IOException

    • getSwappedPartitionNames

      Set<String> getSwappedPartitionNames(FlowFileQueue queue) throws IOException
      Determines the names of each of the Partitions for which there are swap files for the given queue
      Parameters:
      queue - the queue to which the FlowFiles belong
      Returns:
      the Set of names of all Partitions for which there are swap files
      Throws:
      IOException - if unable to read the information from the underlying storage

    • changePartitionName

      String changePartitionName(String swapLocation, String newPartitionName) throws IOException
      Updates the name of the partition that owns a given swap file
      Parameters:
      swapLocation - the location of the swap file
      newPartitionName - the new name of the new partition that owns the swap file
      Returns:
      the new swap location
      Throws:
      IOException - if unable to rename the swap file

    • getSwapSummary

      SwapSummary getSwapSummary(String swapLocation) throws IOException
      Parses the contents of the swap file at the given location and provides a SwapSummary that provides pertinent information about the information stored within the swap file
      Parameters:
      swapLocation - the location of the swap file
      Returns:
      a SwapSummary that provides information about what is contained within the swap file
      Throws:
      IOException - if unable to read or parse the swap file

    • purge

      void purge()
      Purge all known Swap Files without updating FlowFileRepository or Provenance Repository

    • getQueueIdentifier

      String getQueueIdentifier(String swapLocation)
      Returns the ID of the queue that the given swap file belongs to
      Parameters:
      swapLocation - the swap location
      Returns:
      the ID of the queue, or null if unknown