Package org.apache.camel.component.file
Class GenericFileConsumer<T>
java.lang.Object
org.apache.camel.support.service.BaseService
org.apache.camel.support.service.ServiceSupport
org.apache.camel.support.DefaultConsumer
org.apache.camel.support.ScheduledPollConsumer
org.apache.camel.support.ScheduledBatchPollingConsumer
org.apache.camel.component.file.GenericFileConsumer<T>
- All Implemented Interfaces:
AutoCloseable
,Runnable
,org.apache.camel.BatchConsumer
,org.apache.camel.Consumer
,org.apache.camel.EndpointAware
,org.apache.camel.health.HealthCheckAware
,org.apache.camel.PollingConsumerPollingStrategy
,org.apache.camel.RouteAware
,org.apache.camel.Service
,org.apache.camel.ShutdownableService
,org.apache.camel.spi.RouteIdAware
,org.apache.camel.spi.ShutdownAware
,org.apache.camel.spi.ShutdownPrepared
,org.apache.camel.StatefulService
,org.apache.camel.Suspendable
,org.apache.camel.SuspendableService
- Direct Known Subclasses:
FileConsumer
public abstract class GenericFileConsumer<T>
extends org.apache.camel.support.ScheduledBatchPollingConsumer
Base class for file consumers.
-
Field Summary
Modifier and TypeFieldDescriptionprotected org.apache.camel.Processor
protected boolean
protected GenericFileEndpoint
<T> protected GenericFileOperations
<T> protected int
protected boolean
protected GenericFileProcessStrategy
<T> protected org.apache.camel.ShutdownRunningTask
Fields inherited from class org.apache.camel.support.ScheduledBatchPollingConsumer
maxMessagesPerPoll
Fields inherited from class org.apache.camel.support.service.BaseService
BUILT, FAILED, INITIALIZED, INITIALIZING, lock, NEW, SHUTDOWN, SHUTTING_DOWN, STARTED, STARTING, status, STOPPED, STOPPING, SUSPENDED, SUSPENDING
-
Constructor Summary
ModifierConstructorDescriptionprotected
GenericFileConsumer
(GenericFileEndpoint<T> endpoint, org.apache.camel.Processor processor, GenericFileOperations<T> operations, GenericFileProcessStrategy<T> processStrategy) -
Method Summary
Modifier and TypeMethodDescriptionvoid
long
beforePoll
(long timeout) boolean
canPollMoreFiles
(List<?> fileList) Whether or not we can continue polling for more filesprotected abstract org.apache.camel.Exchange
createExchange
(GenericFile<T> file) Creates the exchange from the polled fileprotected boolean
customProcessExchange
(org.apache.camel.Exchange exchange, org.apache.camel.Processor processor) Processes the exchange using a custom processor.protected void
doInit()
protected void
doStart()
protected void
doStop()
protected String
evaluateFileExpression
(org.apache.camel.Exchange exchange) org.apache.camel.Processor
protected boolean
ignoreCannotRetrieveFile
(String name, org.apache.camel.Exchange exchange, Exception cause) Whether to ignore if the file cannot be retrieved.boolean
protected boolean
isMatched
(GenericFile<T> file, boolean isDirectory, T[] files) Strategy to perform file matching based on endpoint configuration.protected abstract boolean
isMatched
(GenericFile<T> file, String doneFileName, T[] files) Strategy to perform file matching based on endpoint configuration in terms of done file name.protected boolean
isMatchedHiddenFile
(GenericFile<T> file, boolean isDirectory) Strategy to perform hidden file matching based on endpoint configuration.protected boolean
Override if required.protected boolean
isValidFile
(GenericFile<T> file, boolean isDirectory, T[] files) Strategy for validating if the given remote file should be included or notvoid
onInit()
int
poll()
Poll for filesprotected abstract boolean
pollDirectory
(String fileName, List<GenericFile<T>> fileList, int depth) Polls the given directory for files to processprotected void
postPollCheck
(int polledMessages) Override if required.protected boolean
Override if required.int
processBatch
(Queue<Object> exchanges) protected boolean
processExchange
(org.apache.camel.Exchange exchange) Processes the exchangeprotected void
removeExcessiveInProgressFiles
(Deque<org.apache.camel.Exchange> exchanges, int limit) Drain any in progress files as we are done with this batchprotected void
removeExcessiveInProgressFiles
(List<GenericFile<T>> files) Drain any in progress files as we are done with the filesvoid
setCustomProcessor
(org.apache.camel.Processor processor) Use a custom processor to process the exchange.void
setEagerLimitMaxMessagesPerPoll
(boolean eagerLimitMaxMessagesPerPoll) void
setOperations
(GenericFileOperations<T> operations) Sets the operations to be used.protected abstract void
updateFileHeaders
(GenericFile<T> file, org.apache.camel.Message message) Updates the information onMessage
after we have acquired read-lock and can begin process the file.Methods inherited from class org.apache.camel.support.ScheduledBatchPollingConsumer
deferShutdown, getMaxMessagesPerPoll, getPendingExchangesSize, isBatchAllowed, prepareShutdown, processEmptyMessage, setMaxMessagesPerPoll
Methods inherited from class org.apache.camel.support.ScheduledPollConsumer
addLastErrorDetail, afterConfigureScheduler, doBuild, doShutdown, doSuspend, forceConsumerAsReady, getBackoffCounter, getBackoffErrorThreshold, getBackoffIdleThreshold, getBackoffMultiplier, getCounter, getDelay, getErrorCounter, getInitialDelay, getLastError, getLastErrorDetails, getPollStrategy, getRepeatCount, getRunLoggingLevel, getScheduledExecutorService, getScheduler, getSchedulerProperties, getSuccessCounter, getTimeUnit, initialHealthCheckState, isConsumerReady, isFirstPollDone, isGreedy, isPollAllowed, isPolling, isSchedulerStarted, isSendEmptyMessageWhenIdle, isStartScheduler, isUseFixedDelay, run, setBackoffErrorThreshold, setBackoffIdleThreshold, setBackoffMultiplier, setDelay, setGreedy, setInitialDelay, setPollStrategy, setRepeatCount, setRunLoggingLevel, setScheduledExecutorService, setScheduler, setSchedulerProperties, setSendEmptyMessageWhenIdle, setStartScheduler, setTimeUnit, setUseFixedDelay, startScheduler
Methods inherited from class org.apache.camel.support.DefaultConsumer
createExchange, createUoW, defaultConsumerCallback, doneUoW, getAsyncProcessor, getEndpoint, getExceptionHandler, getHealthCheck, getProcessor, getRoute, getRouteId, handleException, handleException, handleException, releaseExchange, setExceptionHandler, setHealthCheck, setRoute, setRouteId, toString
Methods inherited from class org.apache.camel.support.service.BaseService
build, doFail, doLifecycleChange, doResume, fail, getStatus, init, isBuild, isInit, isNew, isRunAllowed, isShutdown, isStarted, isStarting, isStartingOrStarted, isStopped, isStopping, isStoppingOrStopped, isSuspended, isSuspending, isSuspendingOrSuspended, resume, shutdown, start, stop, suspend
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
Methods inherited from interface org.apache.camel.Consumer
createExchange, defaultConsumerCallback, getProcessor, releaseExchange
Methods inherited from interface org.apache.camel.EndpointAware
getEndpoint
Methods inherited from interface org.apache.camel.health.HealthCheckAware
getHealthCheck, setHealthCheck
Methods inherited from interface org.apache.camel.Service
build, close, init, start, stop
Methods inherited from interface org.apache.camel.ShutdownableService
shutdown
Methods inherited from interface org.apache.camel.StatefulService
getStatus, isRunAllowed, isStarted, isStarting, isStopped, isStopping, isSuspending
Methods inherited from interface org.apache.camel.SuspendableService
isSuspended, resume, suspend
-
Field Details
-
endpoint
-
operations
-
processStrategy
-
shutdownRunningTask
protected volatile org.apache.camel.ShutdownRunningTask shutdownRunningTask -
pendingExchanges
protected volatile int pendingExchanges -
customProcessor
protected org.apache.camel.Processor customProcessor -
eagerLimitMaxMessagesPerPoll
protected boolean eagerLimitMaxMessagesPerPoll -
prepareOnStartup
protected volatile boolean prepareOnStartup
-
-
Constructor Details
-
GenericFileConsumer
protected GenericFileConsumer(GenericFileEndpoint<T> endpoint, org.apache.camel.Processor processor, GenericFileOperations<T> operations, GenericFileProcessStrategy<T> processStrategy)
-
-
Method Details
-
getCustomProcessor
public org.apache.camel.Processor getCustomProcessor() -
setCustomProcessor
public void setCustomProcessor(org.apache.camel.Processor processor) Use a custom processor to process the exchange. Only set this if you need to do custom processing, instead of the regular processing. This is for example used to browse file endpoints by leveraging the file consumer to poll the directory to gather the list of exchanges. But to avoid processing the files regularly we can use a custom processor.- Parameters:
processor
- a custom processor
-
isEagerLimitMaxMessagesPerPoll
public boolean isEagerLimitMaxMessagesPerPoll() -
setEagerLimitMaxMessagesPerPoll
public void setEagerLimitMaxMessagesPerPoll(boolean eagerLimitMaxMessagesPerPoll) -
createExchange
Creates the exchange from the polled file -
poll
Poll for files- Specified by:
poll
in classorg.apache.camel.support.ScheduledPollConsumer
- Throws:
Exception
-
processBatch
-
removeExcessiveInProgressFiles
protected void removeExcessiveInProgressFiles(Deque<org.apache.camel.Exchange> exchanges, int limit) Drain any in progress files as we are done with this batch- Parameters:
exchanges
- the exchangeslimit
- the limit
-
removeExcessiveInProgressFiles
Drain any in progress files as we are done with the files- Parameters:
files
- the files
-
canPollMoreFiles
Whether or not we can continue polling for more files- Parameters:
fileList
- the current list of gathered files- Returns:
- true to continue, false to stop due hitting maxMessagesPerPoll limit
-
prePollCheck
Override if required. Perform some checks (and perhaps actions) before we poll.- Returns:
- true to poll, false to skip this poll.
- Throws:
Exception
-
postPollCheck
protected void postPollCheck(int polledMessages) Override if required. Perform some checks (and perhaps actions) after we have polled.- Parameters:
polledMessages
- number of polled messages
-
pollDirectory
Polls the given directory for files to process- Parameters:
fileName
- current directory or filefileList
- current list of files gathereddepth
- the current depth of the directory (will start from 0)- Returns:
- whether or not to continue polling, false means the maxMessagesPerPoll limit has been hit
-
setOperations
Sets the operations to be used. Can be used to set a fresh operations in case of recovery attempts- Parameters:
operations
- the operations
-
ignoreCannotRetrieveFile
protected boolean ignoreCannotRetrieveFile(String name, org.apache.camel.Exchange exchange, Exception cause) Whether to ignore if the file cannot be retrieved. By default anGenericFileOperationFailedException
is thrown if the file cannot be retrieved. This method allows to suppress this and just ignore that.- Parameters:
name
- the file nameexchange
- the exchangecause
- optional exception occurred during retrieving file- Returns:
- true to ignore, false is the default.
-
processExchange
protected boolean processExchange(org.apache.camel.Exchange exchange) Processes the exchange- Parameters:
exchange
- the exchange- Returns:
- true if the file was started to be processed, false if the file was not started to be processed, for some reason (not found, or aborted etc)
-
updateFileHeaders
Updates the information onMessage
after we have acquired read-lock and can begin process the file.- Parameters:
file
- the filemessage
- the Camel message to update its headers
-
isRetrieveFile
protected boolean isRetrieveFile()Override if required. Files are retrieved / returns true by default- Returns:
- true to retrieve files, false to skip retrieval of files.
-
customProcessExchange
protected boolean customProcessExchange(org.apache.camel.Exchange exchange, org.apache.camel.Processor processor) Processes the exchange using a custom processor.- Parameters:
exchange
- the exchangeprocessor
- the custom processor
-
isValidFile
Strategy for validating if the given remote file should be included or not- Parameters:
file
- the fileisDirectory
- whether the file is a directory or a filefiles
- files in the directory- Returns:
- true to include the file, false to skip it
-
isMatchedHiddenFile
Strategy to perform hidden file matching based on endpoint configuration. Will always return false for certain files/folders:- Starting with a dot (hidden)
-
isMatched
Strategy to perform file matching based on endpoint configuration. Will always return false for certain files/folders:- Starting with a dot
- lock files
- Parameters:
file
- the fileisDirectory
- whether the file is a directory or a filefiles
- files in the directory- Returns:
- true if the file is matched, false if not
-
isMatched
Strategy to perform file matching based on endpoint configuration in terms of done file name.- Parameters:
file
- the filedoneFileName
- the done file name (without any paths)files
- files in the directory- Returns:
- true if the file is matched, false if not
-
evaluateFileExpression
-
doInit
- Overrides:
doInit
in classorg.apache.camel.support.ScheduledPollConsumer
- Throws:
Exception
-
doStart
- Overrides:
doStart
in classorg.apache.camel.support.ScheduledPollConsumer
- Throws:
Exception
-
doStop
- Overrides:
doStop
in classorg.apache.camel.support.ScheduledPollConsumer
- Throws:
Exception
-
onInit
- Specified by:
onInit
in interfaceorg.apache.camel.PollingConsumerPollingStrategy
- Overrides:
onInit
in classorg.apache.camel.support.ScheduledPollConsumer
- Throws:
Exception
-
beforePoll
- Specified by:
beforePoll
in interfaceorg.apache.camel.PollingConsumerPollingStrategy
- Overrides:
beforePoll
in classorg.apache.camel.support.ScheduledPollConsumer
- Throws:
Exception
-
afterPoll
- Specified by:
afterPoll
in interfaceorg.apache.camel.PollingConsumerPollingStrategy
- Overrides:
afterPoll
in classorg.apache.camel.support.ScheduledPollConsumer
- Throws:
Exception
-