org.apache.camel.component.file

Class GenericFileConsumer<T>

java.lang.Object

org.apache.camel.support.service.BaseService

org.apache.camel.support.service.ServiceSupport

org.apache.camel.support.DefaultConsumer

org.apache.camel.support.ScheduledPollConsumer

org.apache.camel.support.ScheduledBatchPollingConsumer

org.apache.camel.component.file.GenericFileConsumer<T>

All Implemented Interfaces:

AutoCloseable, Runnable, org.apache.camel.BatchConsumer, org.apache.camel.Consumer, org.apache.camel.EndpointAware, org.apache.camel.PollingConsumerPollingStrategy, org.apache.camel.RouteAware, org.apache.camel.Service, org.apache.camel.ShutdownableService, org.apache.camel.spi.RouteIdAware, org.apache.camel.spi.ShutdownAware, org.apache.camel.spi.ShutdownPrepared, org.apache.camel.StatefulService, org.apache.camel.Suspendable, org.apache.camel.SuspendableService

Direct Known Subclasses:

FileConsumer

public abstract class GenericFileConsumer<T>
extends org.apache.camel.support.ScheduledBatchPollingConsumer

Base class for file consumers.

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.camel.Processor	customProcessor
protected boolean	eagerLimitMaxMessagesPerPoll
protected GenericFileEndpoint<T>	endpoint
protected GenericFileOperations<T>	operations
protected int	pendingExchanges
protected boolean	prepareOnStartup
protected GenericFileProcessStrategy<T>	processStrategy
protected org.apache.camel.ShutdownRunningTask	shutdownRunningTask

Fields inherited from class org.apache.camel.support.ScheduledBatchPollingConsumer

maxMessagesPerPoll

Fields inherited from class org.apache.camel.support.service.BaseService

BUILT, FAILED, INITIALIZED, INITIALIZING, lock, NEW, SHUTDOWN, SHUTTING_DOWN, STARTED, STARTING, status, STOPPED, STOPPING, SUSPENDED, SUSPENDING

Constructor Summary

Constructors

Constructor and Description
GenericFileConsumer(GenericFileEndpoint<T> endpoint, org.apache.camel.Processor processor, GenericFileOperations<T> operations, GenericFileProcessStrategy<T> processStrategy)

Method Summary

Modifier and Type	Method and Description
void	afterPoll()
long	beforePoll(long timeout)
boolean	canPollMoreFiles(List<?> fileList) Whether or not we can continue polling for more files
protected boolean	customProcessExchange(org.apache.camel.Exchange exchange, org.apache.camel.Processor processor) Processes the exchange using a custom processor.
protected void	doInit()
protected void	doStart()
protected void	doStop()
protected String	evaluateFileExpression(org.apache.camel.Exchange exchange)
org.apache.camel.Processor	getCustomProcessor()
protected boolean	ignoreCannotRetrieveFile(String name, org.apache.camel.Exchange exchange, Exception cause) Whether to ignore if the file cannot be retrieved.
boolean	isEagerLimitMaxMessagesPerPoll()
protected boolean	isMatched(GenericFile<T> file, boolean isDirectory, List<T> files) Strategy to perform file matching based on endpoint configuration.
protected abstract boolean	isMatched(GenericFile<T> file, String doneFileName, List<T> files) Strategy to perform file matching based on endpoint configuration in terms of done file name.
protected boolean	isRetrieveFile() Override if required.
protected boolean	isValidFile(GenericFile<T> file, boolean isDirectory, List<T> files) Strategy for validating if the given remote file should be included or not
void	onInit()
int	poll() Poll for files
protected abstract boolean	pollDirectory(String fileName, List<GenericFile<T>> fileList, int depth) Polls the given directory for files to process
protected void	postPollCheck(int polledMessages) Override if required.
protected boolean	prePollCheck() Override if required.
int	processBatch(Queue<Object> exchanges)
protected boolean	processExchange(org.apache.camel.Exchange exchange) Processes the exchange
protected void	removeExcessiveInProgressFiles(Deque<org.apache.camel.Exchange> exchanges, int limit) Drain any in progress files as we are done with this batch
protected void	removeExcessiveInProgressFiles(List<GenericFile<T>> files) Drain any in progress files as we are done with the files
void	setCustomProcessor(org.apache.camel.Processor processor) Use a custom processor to process the exchange.
void	setEagerLimitMaxMessagesPerPoll(boolean eagerLimitMaxMessagesPerPoll)
void	setOperations(GenericFileOperations<T> operations) Sets the operations to be used.
protected abstract void	updateFileHeaders(GenericFile<T> file, org.apache.camel.Message message) Updates the information on Message after we have acquired read-lock and can begin process the file.

Methods inherited from class org.apache.camel.support.ScheduledBatchPollingConsumer

deferShutdown, getMaxMessagesPerPoll, getPendingExchangesSize, isBatchAllowed, prepareShutdown, processEmptyMessage, setMaxMessagesPerPoll

Methods inherited from class org.apache.camel.support.ScheduledPollConsumer

doShutdown, doSuspend, getBackoffCounter, getBackoffErrorThreshold, getBackoffIdleThreshold, getBackoffMultiplier, getDelay, getInitialDelay, getPollStrategy, getRepeatCount, getRunLoggingLevel, getScheduledExecutorService, getScheduler, getSchedulerProperties, getTimeUnit, isGreedy, isPollAllowed, isPolling, isSchedulerStarted, isSendEmptyMessageWhenIdle, isStartScheduler, isUseFixedDelay, run, setBackoffErrorThreshold, setBackoffIdleThreshold, setBackoffMultiplier, setDelay, setGreedy, setInitialDelay, setPollStrategy, setRepeatCount, setRunLoggingLevel, setScheduledExecutorService, setScheduler, setSchedulerProperties, setSendEmptyMessageWhenIdle, setStartScheduler, setTimeUnit, setUseFixedDelay, startScheduler

Methods inherited from class org.apache.camel.support.DefaultConsumer

createUoW, doneUoW, getAsyncProcessor, getEndpoint, getExceptionHandler, getProcessor, getRoute, getRouteId, handleException, handleException, setExceptionHandler, setRoute, setRouteId, toString

Methods inherited from class org.apache.camel.support.service.BaseService

build, doBuild, doFail, doLifecycleChange, doResume, fail, getStatus, init, isBuild, isInit, isNew, isRunAllowed, isShutdown, isStarted, isStarting, isStartingOrStarted, isStopped, isStopping, isStoppingOrStopped, isSuspended, isSuspending, isSuspendingOrSuspended, resume, shutdown, start, stop, suspend

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.camel.Consumer

getProcessor

Methods inherited from interface org.apache.camel.Service

build, close, init, start, stop

Methods inherited from interface org.apache.camel.EndpointAware

getEndpoint

Methods inherited from interface org.apache.camel.StatefulService

getStatus, isRunAllowed, isStarted, isStarting, isStopped, isStopping, isSuspending

Methods inherited from interface org.apache.camel.SuspendableService

isSuspended, resume, suspend

Methods inherited from interface org.apache.camel.ShutdownableService

shutdown

Field Detail

endpoint

protected GenericFileEndpoint<T> endpoint

operations

protected GenericFileOperations<T> operations

processStrategy

protected GenericFileProcessStrategy<T> processStrategy

shutdownRunningTask

protected volatile org.apache.camel.ShutdownRunningTask shutdownRunningTask

pendingExchanges

protected volatile int pendingExchanges

customProcessor

protected org.apache.camel.Processor customProcessor

eagerLimitMaxMessagesPerPoll

protected boolean eagerLimitMaxMessagesPerPoll

prepareOnStartup

protected volatile boolean prepareOnStartup

Constructor Detail

GenericFileConsumer

public GenericFileConsumer(GenericFileEndpoint<T> endpoint,
                           org.apache.camel.Processor processor,
                           GenericFileOperations<T> operations,
                           GenericFileProcessStrategy<T> processStrategy)

Method Detail

getCustomProcessor

public org.apache.camel.Processor getCustomProcessor()

setCustomProcessor

public void setCustomProcessor(org.apache.camel.Processor processor)

Use a custom processor to process the exchange.

Only set this if you need to do custom processing, instead of the regular processing.

This is for example used to browse file endpoints by leveraging the file consumer to poll the directory to gather the list of exchanges. But to avoid processing the files regularly we can use a custom processor.

Parameters:

processor - a custom processor

isEagerLimitMaxMessagesPerPoll

public boolean isEagerLimitMaxMessagesPerPoll()

setEagerLimitMaxMessagesPerPoll

public void setEagerLimitMaxMessagesPerPoll(boolean eagerLimitMaxMessagesPerPoll)

poll

public int poll()
         throws Exception

Poll for files

Specified by:

poll in class org.apache.camel.support.ScheduledPollConsumer

Throws:

Exception

processBatch

public int processBatch(Queue<Object> exchanges)

removeExcessiveInProgressFiles

protected void removeExcessiveInProgressFiles(Deque<org.apache.camel.Exchange> exchanges,
                                              int limit)

Drain any in progress files as we are done with this batch

Parameters:

exchanges - the exchanges

limit - the limit

removeExcessiveInProgressFiles

protected void removeExcessiveInProgressFiles(List<GenericFile<T>> files)

Drain any in progress files as we are done with the files

Parameters:

files - the files

canPollMoreFiles

public boolean canPollMoreFiles(List<?> fileList)

Whether or not we can continue polling for more files

Parameters:

fileList - the current list of gathered files

Returns:

true to continue, false to stop due hitting maxMessagesPerPoll limit

prePollCheck

protected boolean prePollCheck()
                        throws Exception

Override if required. Perform some checks (and perhaps actions) before we poll.

Returns:

true to poll, false to skip this poll.

Throws:

Exception

postPollCheck

protected void postPollCheck(int polledMessages)

Override if required. Perform some checks (and perhaps actions) after we have polled.

Parameters:

polledMessages - number of polled messages

pollDirectory

protected abstract boolean pollDirectory(String fileName,
                                         List<GenericFile<T>> fileList,
                                         int depth)

Polls the given directory for files to process

Parameters:

fileName - current directory or file

fileList - current list of files gathered

depth - the current depth of the directory (will start from 0)

Returns:

whether or not to continue polling, false means the maxMessagesPerPoll limit has been hit

setOperations

public void setOperations(GenericFileOperations<T> operations)

Sets the operations to be used.

Can be used to set a fresh operations in case of recovery attempts

Parameters:

operations - the operations

ignoreCannotRetrieveFile

protected boolean ignoreCannotRetrieveFile(String name,
                                           org.apache.camel.Exchange exchange,
                                           Exception cause)

Whether to ignore if the file cannot be retrieved.

By default an GenericFileOperationFailedException is thrown if the file cannot be retrieved.

This method allows to suppress this and just ignore that.

Parameters:

name - the file name

exchange - the exchange

cause - optional exception occurred during retrieving file

Returns:

true to ignore, false is the default.

processExchange

protected boolean processExchange(org.apache.camel.Exchange exchange)

Processes the exchange

Parameters:

exchange - the exchange

Returns:

true if the file was started to be processed, false if the file was not started to be processed, for some reason (not found, or aborted etc)

updateFileHeaders

protected abstract void updateFileHeaders(GenericFile<T> file,
                                          org.apache.camel.Message message)

Updates the information on Message after we have acquired read-lock and can begin process the file.

Parameters:

file - the file

message - the Camel message to update its headers

isRetrieveFile

protected boolean isRetrieveFile()

Override if required. Files are retrieved / returns true by default

Returns:

true to retrieve files, false to skip retrieval of files.

customProcessExchange

protected boolean customProcessExchange(org.apache.camel.Exchange exchange,
                                        org.apache.camel.Processor processor)

Processes the exchange using a custom processor.

Parameters:

exchange - the exchange

processor - the custom processor

isValidFile

protected boolean isValidFile(GenericFile<T> file,
                              boolean isDirectory,
                              List<T> files)

Strategy for validating if the given remote file should be included or not

Parameters:

file - the file

isDirectory - whether the file is a directory or a file

files - files in the directory

Returns:

true to include the file, false to skip it

isMatched

protected boolean isMatched(GenericFile<T> file,
                            boolean isDirectory,
                            List<T> files)

Strategy to perform file matching based on endpoint configuration.

Will always return false for certain files/folders:

• Starting with a dot
• lock files

And then true for directories.

Parameters:

file - the file

isDirectory - whether the file is a directory or a file

files - files in the directory

Returns:

true if the file is matched, false if not

isMatched

protected abstract boolean isMatched(GenericFile<T> file,
                                     String doneFileName,
                                     List<T> files)

Strategy to perform file matching based on endpoint configuration in terms of done file name.

Parameters:

file - the file

doneFileName - the done file name (without any paths)

files - files in the directory

Returns:

true if the file is matched, false if not

evaluateFileExpression

protected String evaluateFileExpression(org.apache.camel.Exchange exchange)

doInit

protected void doInit()
               throws Exception

Overrides:

doInit in class org.apache.camel.support.ScheduledPollConsumer

Throws:

Exception

doStart

protected void doStart()
                throws Exception

Overrides:

doStart in class org.apache.camel.support.ScheduledPollConsumer

Throws:

Exception

doStop

protected void doStop()
               throws Exception

Overrides:

doStop in class org.apache.camel.support.ScheduledPollConsumer

Throws:

Exception

onInit

public void onInit()
            throws Exception

Specified by:

onInit in interface org.apache.camel.PollingConsumerPollingStrategy

Overrides:

onInit in class org.apache.camel.support.ScheduledPollConsumer

Throws:

Exception

beforePoll

public long beforePoll(long timeout)
                throws Exception

Specified by:

beforePoll in interface org.apache.camel.PollingConsumerPollingStrategy

Overrides:

beforePoll in class org.apache.camel.support.ScheduledPollConsumer

Throws:

Exception

afterPoll

public void afterPoll()
               throws Exception

Specified by:

afterPoll in interface org.apache.camel.PollingConsumerPollingStrategy

Overrides:

afterPoll in class org.apache.camel.support.ScheduledPollConsumer

Throws:

Exception