Package org.springframework.integration.file

Class FileReadingMessageSource

java.lang.Object
org.springframework.integration.util.AbstractExpressionEvaluator
org.springframework.integration.endpoint.AbstractMessageSource<java.io.File>
org.springframework.integration.file.FileReadingMessageSource
All Implemented Interfaces:
org.springframework.beans.factory.Aware, org.springframework.beans.factory.BeanFactoryAware, org.springframework.beans.factory.BeanNameAware, org.springframework.beans.factory.DisposableBean, org.springframework.beans.factory.InitializingBean, org.springframework.context.Lifecycle, org.springframework.integration.core.MessageSource<java.io.File>, org.springframework.integration.IntegrationPattern, org.springframework.integration.support.context.NamedComponent, org.springframework.integration.support.management.IntegrationInboundManagement, org.springframework.integration.support.management.IntegrationManagement, org.springframework.integration.support.management.ManageableLifecycle
public class FileReadingMessageSource
extends org.springframework.integration.endpoint.AbstractMessageSource<java.io.File>
implements org.springframework.integration.support.management.ManageableLifecycle
MessageSource that creates messages from a file system directory. To prevent messages for certain files, you may supply a FileListFilter. By default, when configuring with XML or the DSL, an AcceptOnceFileListFilter is used. It ensures files are picked up only once from the directory.

A common problem with reading files is that a file may be detected before it is ready. The default AcceptOnceFileListFilter does not prevent this. In most cases, this can be prevented if the file-writing process renames each file as soon as it is ready for reading. A pattern-matching filter that accepts only files that are ready (e.g. based on a known suffix), composed with the default AcceptOnceFileListFilter would allow for this.

If a external DirectoryScanner is used, then the FileLocker and FileListFilter objects should be set on the external DirectoryScanner, not the instance of FileReadingMessageSource. An IllegalStateException will result otherwise.

A Comparator can be used to ensure internal ordering of the Files in a PriorityBlockingQueue. This does not provide the same guarantees as a ResequencingMessageGroupProcessor, but in cases where writing files and failure downstream are rare it might be sufficient.

FileReadingMessageSource is fully thread-safe under concurrent receive() invocations and message delivery callbacks.

  • Nested Class Summary

    Nested Classes 
    Modifier and Type Class Description
    static class  FileReadingMessageSource.WatchEventType  

    Nested classes/interfaces inherited from interface org.springframework.integration.support.management.IntegrationManagement

    org.springframework.integration.support.management.IntegrationManagement.ManagementOverrides

  • Field Summary

    Fields inherited from class org.springframework.integration.util.AbstractExpressionEvaluator

    EXPRESSION_PARSER, logger

    Fields inherited from interface org.springframework.integration.support.management.IntegrationManagement

    METER_PREFIX, RECEIVE_COUNTER_NAME, SEND_TIMER_NAME

  • Constructor Summary

    Constructors 
    Constructor Description
    FileReadingMessageSource()
    Create a FileReadingMessageSource with a naturally ordered queue of unbounded capacity.
    FileReadingMessageSource​(int internalQueueCapacity)
    Create a FileReadingMessageSource with a bounded queue of the given capacity.
    FileReadingMessageSource​(java.util.Comparator<java.io.File> receptionOrderComparator)
    Create a FileReadingMessageSource with a PriorityBlockingQueue ordered with the passed in Comparator.

  • Method Summary

    Modifier and Type Method Description
    protected org.springframework.integration.support.AbstractIntegrationMessageBuilder<java.io.File> doReceive()  
    java.lang.String getComponentType()  
    DirectoryScanner getScanner()
    The scanner property accessor to allow to modify its options (filter, locker etc.) at runtime using the FileReadingMessageSource bean.
    boolean isRunning()  
    boolean isUseWatchService()  
    void onFailure​(org.springframework.messaging.Message<java.io.File> failedMessage)
    Adds the failed message back to the 'toBeReceived' queue if there is room.
    protected void onInit()  
    void setAutoCreateDirectory​(boolean autoCreateDirectory)
    Specify whether to create the source directory automatically if it does not yet exist upon initialization.
    void setDirectory​(java.io.File directory)
    Specify the input directory.
    void setFilter​(FileListFilter<java.io.File> filter)
    Set a FileListFilter.
    void setLocker​(FileLocker locker)
    Set a FileLocker to be used to guard files against duplicate processing.
    void setScanEachPoll​(boolean scanEachPoll)
    Set this flag if you want to make sure the internal queue is refreshed with the latest content of the input directory on each poll.
    void setScanner​(DirectoryScanner scanner)
    Optionally specify a custom scanner, for example the FileReadingMessageSource.WatchServiceDirectoryScanner.
    void setUseWatchService​(boolean useWatchService)
    Switch this FileReadingMessageSource to use its internal FileReadingMessageSource.WatchServiceDirectoryScanner.
    void setWatchEvents​(FileReadingMessageSource.WatchEventType... watchEvents)
    The WatchService event types.
    void start()  
    void stop()  

    Methods inherited from class org.springframework.integration.endpoint.AbstractMessageSource

    buildMessage, destroy, getBeanName, getComponentName, getManagedName, getManagedType, getOverrides, isLoggingEnabled, receive, registerMetricsCaptor, setBeanName, setHeaderExpressions, setLoggingEnabled, setManagedName, setManagedType

    Methods inherited from class org.springframework.integration.util.AbstractExpressionEvaluator

    afterPropertiesSet, evaluateExpression, evaluateExpression, evaluateExpression, evaluateExpression, evaluateExpression, evaluateExpression, evaluateExpression, getBeanFactory, getEvaluationContext, getEvaluationContext, getMessageBuilderFactory, setBeanFactory, setConversionService

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.springframework.integration.support.management.IntegrationManagement

    getThisAs

    Methods inherited from interface org.springframework.integration.core.MessageSource

    getIntegrationPatternType

  • Constructor Details

    • FileReadingMessageSource

      public FileReadingMessageSource()
      Create a FileReadingMessageSource with a naturally ordered queue of unbounded capacity.

    • FileReadingMessageSource

      public FileReadingMessageSource​(int internalQueueCapacity)
      Create a FileReadingMessageSource with a bounded queue of the given capacity. This can be used to reduce the memory footprint of this component when reading from a large directory.
      Parameters:
      internalQueueCapacity - the size of the queue used to cache files to be received internally. This queue can be made larger to optimize the directory scanning. With scanEachPoll set to false and the queue to a large size, it will be filled once and then completely emptied before a new directory listing is done. This is particularly useful to reduce scans of large numbers of files in a directory.

    • FileReadingMessageSource

      public FileReadingMessageSource​(@Nullable java.util.Comparator<java.io.File> receptionOrderComparator)
      Create a FileReadingMessageSource with a PriorityBlockingQueue ordered with the passed in Comparator.

      The size of the queue used should be large enough to hold all the files in the input directory in order to sort all of them, so restricting the size of the queue is mutually exclusive with ordering. No guarantees about file delivery order can be made under concurrent access.

      Parameters:
      receptionOrderComparator - the comparator to be used to order the files in the internal queue

  • Method Details

    • setDirectory

      public void setDirectory​(java.io.File directory)
      Specify the input directory.
      Parameters:
      directory - to monitor

    • setScanner

      public void setScanner​(DirectoryScanner scanner)
      Optionally specify a custom scanner, for example the FileReadingMessageSource.WatchServiceDirectoryScanner.
      Parameters:
      scanner - scanner implementation

    • getScanner

      public DirectoryScanner getScanner()
      The scanner property accessor to allow to modify its options (filter, locker etc.) at runtime using the FileReadingMessageSource bean.
      Returns:
      the DirectoryScanner of this FileReadingMessageSource.
      Since:
      4.2

    • setAutoCreateDirectory

      public void setAutoCreateDirectory​(boolean autoCreateDirectory)
      Specify whether to create the source directory automatically if it does not yet exist upon initialization. By default, this value is true. If set to false and the source directory does not exist, an Exception will be thrown upon initialization.
      Parameters:
      autoCreateDirectory - should the directory to be monitored be created when this component starts up?

    • setFilter

      public void setFilter​(FileListFilter<java.io.File> filter)
      Set a FileListFilter. By default a AcceptOnceFileListFilter with no bounds is used. In most cases a customized FileListFilter will be needed to deal with modification and duplication concerns. If multiple filters are required a CompositeFileListFilter can be used to group them together.

      The supplied filter must be thread safe..

      Parameters:
      filter - a filter

    • setLocker

      public void setLocker​(FileLocker locker)
      Set a FileLocker to be used to guard files against duplicate processing.

      The supplied FileLocker must be thread safe

      Parameters:
      locker - a locker

    • setScanEachPoll

      public void setScanEachPoll​(boolean scanEachPoll)
      Set this flag if you want to make sure the internal queue is refreshed with the latest content of the input directory on each poll.

      By default this implementation will empty its queue before looking at the directory again. In cases where order is relevant it is important to consider the effects of setting this flag. The internal BlockingQueue that this class is keeping will more likely be out of sync with the file system if this flag is set to false, but it will change more often (causing expensive reordering) if it is set to true.

      Parameters:
      scanEachPoll - whether or not the component should re-scan (as opposed to not rescanning until the entire backlog has been delivered)

    • setUseWatchService

      public void setUseWatchService​(boolean useWatchService)
      Switch this FileReadingMessageSource to use its internal FileReadingMessageSource.WatchServiceDirectoryScanner.
      Parameters:
      useWatchService - the boolean flag to switch to FileReadingMessageSource.WatchServiceDirectoryScanner on true.
      Since:
      4.3
      See Also:
      setWatchEvents(org.springframework.integration.file.FileReadingMessageSource.WatchEventType...)

    • isUseWatchService

      public boolean isUseWatchService()

    • setWatchEvents

      public void setWatchEvents​(FileReadingMessageSource.WatchEventType... watchEvents)
      The WatchService event types. If setUseWatchService(boolean) isn't true, this option is ignored.
      Parameters:
      watchEvents - the set of FileReadingMessageSource.WatchEventType.
      Since:
      4.3
      See Also:
      setUseWatchService(boolean)

    • getComponentType

      public java.lang.String getComponentType()
      Specified by:
      getComponentType in interface org.springframework.integration.support.context.NamedComponent

    • start

      public void start()
      Specified by:
      start in interface org.springframework.context.Lifecycle
      Specified by:
      start in interface org.springframework.integration.support.management.ManageableLifecycle

    • stop

      public void stop()
      Specified by:
      stop in interface org.springframework.context.Lifecycle
      Specified by:
      stop in interface org.springframework.integration.support.management.ManageableLifecycle

    • isRunning

      public boolean isRunning()
      Specified by:
      isRunning in interface org.springframework.context.Lifecycle
      Specified by:
      isRunning in interface org.springframework.integration.support.management.ManageableLifecycle

    • onInit

      protected void onInit()
      Overrides:
      onInit in class org.springframework.integration.util.AbstractExpressionEvaluator

    • doReceive

      protected org.springframework.integration.support.AbstractIntegrationMessageBuilder<java.io.File> doReceive()
      Specified by:
      doReceive in class org.springframework.integration.endpoint.AbstractMessageSource<java.io.File>

    • onFailure

      public void onFailure​(org.springframework.messaging.Message<java.io.File> failedMessage)
      Adds the failed message back to the 'toBeReceived' queue if there is room.
      Parameters:
      failedMessage - the Message that failed