Package org.apache.camel.component.file
Class GenericFileConsumer<T>
- java.lang.Object
-
- org.apache.camel.support.service.BaseService
-
- org.apache.camel.support.service.ServiceSupport
-
- org.apache.camel.support.DefaultConsumer
-
- org.apache.camel.support.ScheduledPollConsumer
-
- org.apache.camel.support.ScheduledBatchPollingConsumer
-
- org.apache.camel.component.file.GenericFileConsumer<T>
-
- All Implemented Interfaces:
AutoCloseable
,Runnable
,org.apache.camel.BatchConsumer
,org.apache.camel.Consumer
,org.apache.camel.EndpointAware
,org.apache.camel.health.HealthCheckAware
,org.apache.camel.PollingConsumerPollingStrategy
,org.apache.camel.RouteAware
,org.apache.camel.Service
,org.apache.camel.ShutdownableService
,org.apache.camel.spi.RouteIdAware
,org.apache.camel.spi.ShutdownAware
,org.apache.camel.spi.ShutdownPrepared
,org.apache.camel.StatefulService
,org.apache.camel.Suspendable
,org.apache.camel.SuspendableService
- Direct Known Subclasses:
FileConsumer
public abstract class GenericFileConsumer<T> extends org.apache.camel.support.ScheduledBatchPollingConsumer
Base class for file consumers.
-
-
Field Summary
Fields Modifier and Type Field Description protected org.apache.camel.Processor
customProcessor
protected boolean
eagerLimitMaxMessagesPerPoll
protected GenericFileEndpoint<T>
endpoint
protected GenericFileOperations<T>
operations
protected int
pendingExchanges
protected boolean
prepareOnStartup
protected GenericFileProcessStrategy<T>
processStrategy
protected org.apache.camel.ShutdownRunningTask
shutdownRunningTask
-
Constructor Summary
Constructors Constructor Description GenericFileConsumer(GenericFileEndpoint<T> endpoint, org.apache.camel.Processor processor, GenericFileOperations<T> operations, GenericFileProcessStrategy<T> processStrategy)
-
Method Summary
All Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description void
afterPoll()
long
beforePoll(long timeout)
boolean
canPollMoreFiles(List<?> fileList)
Whether or not we can continue polling for more filesprotected abstract org.apache.camel.Exchange
createExchange(GenericFile<T> file)
Creates the exchange from the polled fileprotected boolean
customProcessExchange(org.apache.camel.Exchange exchange, org.apache.camel.Processor processor)
Processes the exchange using a custom processor.protected void
doInit()
protected void
doStart()
protected void
doStop()
protected String
evaluateFileExpression(org.apache.camel.Exchange exchange)
org.apache.camel.Processor
getCustomProcessor()
protected boolean
ignoreCannotRetrieveFile(String name, org.apache.camel.Exchange exchange, Exception cause)
Whether to ignore if the file cannot be retrieved.boolean
isEagerLimitMaxMessagesPerPoll()
protected boolean
isMatched(GenericFile<T> file, boolean isDirectory, T[] files)
Strategy to perform file matching based on endpoint configuration.protected abstract boolean
isMatched(GenericFile<T> file, String doneFileName, T[] files)
Strategy to perform file matching based on endpoint configuration in terms of done file name.protected boolean
isRetrieveFile()
Override if required.protected boolean
isValidFile(GenericFile<T> file, boolean isDirectory, T[] files)
Strategy for validating if the given remote file should be included or notvoid
onInit()
int
poll()
Poll for filesprotected abstract boolean
pollDirectory(String fileName, List<GenericFile<T>> fileList, int depth)
Polls the given directory for files to processprotected void
postPollCheck(int polledMessages)
Override if required.protected boolean
prePollCheck()
Override if required.int
processBatch(Queue<Object> exchanges)
protected boolean
processExchange(org.apache.camel.Exchange exchange)
Processes the exchangeprotected void
removeExcessiveInProgressFiles(Deque<org.apache.camel.Exchange> exchanges, int limit)
Drain any in progress files as we are done with this batchprotected void
removeExcessiveInProgressFiles(List<GenericFile<T>> files)
Drain any in progress files as we are done with the filesvoid
setCustomProcessor(org.apache.camel.Processor processor)
Use a custom processor to process the exchange.void
setEagerLimitMaxMessagesPerPoll(boolean eagerLimitMaxMessagesPerPoll)
void
setOperations(GenericFileOperations<T> operations)
Sets the operations to be used.protected abstract void
updateFileHeaders(GenericFile<T> file, org.apache.camel.Message message)
Updates the information onMessage
after we have acquired read-lock and can begin process the file.-
Methods inherited from class org.apache.camel.support.ScheduledBatchPollingConsumer
deferShutdown, getMaxMessagesPerPoll, getPendingExchangesSize, isBatchAllowed, prepareShutdown, processEmptyMessage, setMaxMessagesPerPoll
-
Methods inherited from class org.apache.camel.support.ScheduledPollConsumer
addLastErrorDetail, afterConfigureScheduler, doBuild, doShutdown, doSuspend, getBackoffCounter, getBackoffErrorThreshold, getBackoffIdleThreshold, getBackoffMultiplier, getCounter, getDelay, getErrorCounter, getInitialDelay, getLastError, getLastErrorDetails, getPollStrategy, getRepeatCount, getRunLoggingLevel, getScheduledExecutorService, getScheduler, getSchedulerProperties, getSuccessCounter, getTimeUnit, initialHealthCheckState, isFirstPollDone, isGreedy, isPollAllowed, isPolling, isSchedulerStarted, isSendEmptyMessageWhenIdle, isStartScheduler, isUseFixedDelay, run, setBackoffErrorThreshold, setBackoffIdleThreshold, setBackoffMultiplier, setDelay, setGreedy, setInitialDelay, setPollStrategy, setRepeatCount, setRunLoggingLevel, setScheduledExecutorService, setScheduler, setSchedulerProperties, setSendEmptyMessageWhenIdle, setStartScheduler, setTimeUnit, setUseFixedDelay, startScheduler
-
Methods inherited from class org.apache.camel.support.DefaultConsumer
createExchange, createUoW, defaultConsumerCallback, doneUoW, getAsyncProcessor, getEndpoint, getExceptionHandler, getHealthCheck, getProcessor, getRoute, getRouteId, handleException, handleException, releaseExchange, setExceptionHandler, setHealthCheck, setRoute, setRouteId, toString
-
Methods inherited from class org.apache.camel.support.service.BaseService
build, doFail, doLifecycleChange, doResume, fail, getStatus, init, isBuild, isInit, isNew, isRunAllowed, isShutdown, isStarted, isStarting, isStartingOrStarted, isStopped, isStopping, isStoppingOrStopped, isSuspended, isSuspending, isSuspendingOrSuspended, resume, shutdown, start, stop, suspend
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
-
Methods inherited from interface org.apache.camel.Consumer
createExchange, defaultConsumerCallback, getProcessor, releaseExchange
-
Methods inherited from interface org.apache.camel.health.HealthCheckAware
getHealthCheck, setHealthCheck
-
-
-
-
Field Detail
-
endpoint
protected GenericFileEndpoint<T> endpoint
-
operations
protected GenericFileOperations<T> operations
-
processStrategy
protected GenericFileProcessStrategy<T> processStrategy
-
shutdownRunningTask
protected volatile org.apache.camel.ShutdownRunningTask shutdownRunningTask
-
pendingExchanges
protected volatile int pendingExchanges
-
customProcessor
protected org.apache.camel.Processor customProcessor
-
eagerLimitMaxMessagesPerPoll
protected boolean eagerLimitMaxMessagesPerPoll
-
prepareOnStartup
protected volatile boolean prepareOnStartup
-
-
Constructor Detail
-
GenericFileConsumer
public GenericFileConsumer(GenericFileEndpoint<T> endpoint, org.apache.camel.Processor processor, GenericFileOperations<T> operations, GenericFileProcessStrategy<T> processStrategy)
-
-
Method Detail
-
getCustomProcessor
public org.apache.camel.Processor getCustomProcessor()
-
setCustomProcessor
public void setCustomProcessor(org.apache.camel.Processor processor)
Use a custom processor to process the exchange. Only set this if you need to do custom processing, instead of the regular processing. This is for example used to browse file endpoints by leveraging the file consumer to poll the directory to gather the list of exchanges. But to avoid processing the files regularly we can use a custom processor.- Parameters:
processor
- a custom processor
-
isEagerLimitMaxMessagesPerPoll
public boolean isEagerLimitMaxMessagesPerPoll()
-
setEagerLimitMaxMessagesPerPoll
public void setEagerLimitMaxMessagesPerPoll(boolean eagerLimitMaxMessagesPerPoll)
-
createExchange
protected abstract org.apache.camel.Exchange createExchange(GenericFile<T> file)
Creates the exchange from the polled file
-
poll
public int poll() throws Exception
Poll for files- Specified by:
poll
in classorg.apache.camel.support.ScheduledPollConsumer
- Throws:
Exception
-
removeExcessiveInProgressFiles
protected void removeExcessiveInProgressFiles(Deque<org.apache.camel.Exchange> exchanges, int limit)
Drain any in progress files as we are done with this batch- Parameters:
exchanges
- the exchangeslimit
- the limit
-
removeExcessiveInProgressFiles
protected void removeExcessiveInProgressFiles(List<GenericFile<T>> files)
Drain any in progress files as we are done with the files- Parameters:
files
- the files
-
canPollMoreFiles
public boolean canPollMoreFiles(List<?> fileList)
Whether or not we can continue polling for more files- Parameters:
fileList
- the current list of gathered files- Returns:
- true to continue, false to stop due hitting maxMessagesPerPoll limit
-
prePollCheck
protected boolean prePollCheck() throws Exception
Override if required. Perform some checks (and perhaps actions) before we poll.- Returns:
- true to poll, false to skip this poll.
- Throws:
Exception
-
postPollCheck
protected void postPollCheck(int polledMessages)
Override if required. Perform some checks (and perhaps actions) after we have polled.- Parameters:
polledMessages
- number of polled messages
-
pollDirectory
protected abstract boolean pollDirectory(String fileName, List<GenericFile<T>> fileList, int depth)
Polls the given directory for files to process- Parameters:
fileName
- current directory or filefileList
- current list of files gathereddepth
- the current depth of the directory (will start from 0)- Returns:
- whether or not to continue polling, false means the maxMessagesPerPoll limit has been hit
-
setOperations
public void setOperations(GenericFileOperations<T> operations)
Sets the operations to be used. Can be used to set a fresh operations in case of recovery attempts- Parameters:
operations
- the operations
-
ignoreCannotRetrieveFile
protected boolean ignoreCannotRetrieveFile(String name, org.apache.camel.Exchange exchange, Exception cause)
Whether to ignore if the file cannot be retrieved. By default anGenericFileOperationFailedException
is thrown if the file cannot be retrieved. This method allows to suppress this and just ignore that.- Parameters:
name
- the file nameexchange
- the exchangecause
- optional exception occurred during retrieving file- Returns:
- true to ignore, false is the default.
-
processExchange
protected boolean processExchange(org.apache.camel.Exchange exchange)
Processes the exchange- Parameters:
exchange
- the exchange- Returns:
- true if the file was started to be processed, false if the file was not started to be processed, for some reason (not found, or aborted etc)
-
updateFileHeaders
protected abstract void updateFileHeaders(GenericFile<T> file, org.apache.camel.Message message)
Updates the information onMessage
after we have acquired read-lock and can begin process the file.- Parameters:
file
- the filemessage
- the Camel message to update its headers
-
isRetrieveFile
protected boolean isRetrieveFile()
Override if required. Files are retrieved / returns true by default- Returns:
- true to retrieve files, false to skip retrieval of files.
-
customProcessExchange
protected boolean customProcessExchange(org.apache.camel.Exchange exchange, org.apache.camel.Processor processor)
Processes the exchange using a custom processor.- Parameters:
exchange
- the exchangeprocessor
- the custom processor
-
isValidFile
protected boolean isValidFile(GenericFile<T> file, boolean isDirectory, T[] files)
Strategy for validating if the given remote file should be included or not- Parameters:
file
- the fileisDirectory
- whether the file is a directory or a filefiles
- files in the directory- Returns:
- true to include the file, false to skip it
-
isMatched
protected boolean isMatched(GenericFile<T> file, boolean isDirectory, T[] files)
Strategy to perform file matching based on endpoint configuration. Will always return false for certain files/folders:- Starting with a dot
- lock files
- Parameters:
file
- the fileisDirectory
- whether the file is a directory or a filefiles
- files in the directory- Returns:
- true if the file is matched, false if not
-
isMatched
protected abstract boolean isMatched(GenericFile<T> file, String doneFileName, T[] files)
Strategy to perform file matching based on endpoint configuration in terms of done file name.- Parameters:
file
- the filedoneFileName
- the done file name (without any paths)files
- files in the directory- Returns:
- true if the file is matched, false if not
-
evaluateFileExpression
protected String evaluateFileExpression(org.apache.camel.Exchange exchange)
-
doInit
protected void doInit() throws Exception
- Overrides:
doInit
in classorg.apache.camel.support.ScheduledPollConsumer
- Throws:
Exception
-
doStart
protected void doStart() throws Exception
- Overrides:
doStart
in classorg.apache.camel.support.ScheduledPollConsumer
- Throws:
Exception
-
doStop
protected void doStop() throws Exception
- Overrides:
doStop
in classorg.apache.camel.support.ScheduledPollConsumer
- Throws:
Exception
-
onInit
public void onInit() throws Exception
- Specified by:
onInit
in interfaceorg.apache.camel.PollingConsumerPollingStrategy
- Overrides:
onInit
in classorg.apache.camel.support.ScheduledPollConsumer
- Throws:
Exception
-
beforePoll
public long beforePoll(long timeout) throws Exception
- Specified by:
beforePoll
in interfaceorg.apache.camel.PollingConsumerPollingStrategy
- Overrides:
beforePoll
in classorg.apache.camel.support.ScheduledPollConsumer
- Throws:
Exception
-
-