Package org.apache.camel.component.file
Class GenericFileEndpoint<T>
- java.lang.Object
-
- org.apache.camel.support.service.BaseService
-
- org.apache.camel.support.service.ServiceSupport
-
- org.apache.camel.support.DefaultEndpoint
-
- org.apache.camel.support.ScheduledPollEndpoint
-
- org.apache.camel.component.file.GenericFileEndpoint<T>
-
- All Implemented Interfaces:
AutoCloseable
,org.apache.camel.CamelContextAware
,org.apache.camel.Endpoint
,org.apache.camel.IsSingleton
,org.apache.camel.Service
,org.apache.camel.ShutdownableService
,org.apache.camel.spi.BrowsableEndpoint
,org.apache.camel.spi.HasId
,org.apache.camel.StatefulService
,org.apache.camel.SuspendableService
- Direct Known Subclasses:
FileEndpoint
public abstract class GenericFileEndpoint<T> extends org.apache.camel.support.ScheduledPollEndpoint implements org.apache.camel.spi.BrowsableEndpoint
Base class for file endpoints
-
-
Field Summary
Fields Modifier and Type Field Description protected boolean
allowNullBody
protected String
antExclude
protected AntPathMatcherGenericFileFilter<T>
antFilter
protected boolean
antFilterCaseSensitive
protected String
antInclude
protected String
appendChars
protected boolean
autoCreate
protected int
bufferSize
protected String
charset
protected GenericFileConfiguration
configuration
protected static int
DEFAULT_IDEMPOTENT_CACHE_SIZE
protected static int
DEFAULT_IN_PROGRESS_CACHE_SIZE
protected boolean
delete
protected String
doneFileName
protected boolean
eagerDeleteTargetFile
protected boolean
eagerMaxMessagesPerPoll
protected String
exclude
protected String
excludeExt
protected GenericFileExclusiveReadLockStrategy<T>
exclusiveReadLockStrategy
protected GenericFileExist
fileExist
protected org.apache.camel.Expression
fileName
protected GenericFileFilter<T>
filter
protected org.apache.camel.Predicate
filterDirectory
protected org.apache.camel.Predicate
filterFile
protected boolean
flatten
protected Boolean
idempotent
protected org.apache.camel.Expression
idempotentKey
protected org.apache.camel.spi.IdempotentRepository
idempotentRepository
protected String
include
protected String
includeExt
protected org.apache.camel.spi.IdempotentRepository
inProgressRepository
protected boolean
jailStartingDirectory
protected boolean
keepLastModified
protected String
localWorkDirectory
protected int
maxDepth
protected int
maxMessagesPerPoll
protected int
minDepth
protected org.apache.camel.Expression
move
protected org.apache.camel.Expression
moveExisting
protected FileMoveExistingStrategy
moveExistingFileStrategy
protected org.apache.camel.Expression
moveFailed
protected boolean
noop
protected org.apache.camel.spi.ExceptionHandler
onCompletionExceptionHandler
protected org.apache.camel.Expression
preMove
protected boolean
preSort
protected GenericFileProcessStrategy<T>
processStrategy
protected String
readLock
protected long
readLockCheckInterval
protected boolean
readLockDeleteOrphanLockFiles
protected boolean
readLockIdempotentReleaseAsync
protected int
readLockIdempotentReleaseAsyncPoolSize
protected int
readLockIdempotentReleaseDelay
protected ScheduledExecutorService
readLockIdempotentReleaseExecutorService
protected org.apache.camel.LoggingLevel
readLockLoggingLevel
protected boolean
readLockMarkerFile
protected long
readLockMinAge
protected long
readLockMinLength
protected boolean
readLockRemoveOnCommit
protected boolean
readLockRemoveOnRollback
protected long
readLockTimeout
protected boolean
recursive
protected boolean
shuffle
protected Comparator<org.apache.camel.Exchange>
sortBy
protected Comparator<GenericFile<T>>
sorter
protected org.apache.camel.Expression
tempFileName
protected String
tempPrefix
-
Constructor Summary
Constructors Constructor Description GenericFileEndpoint()
GenericFileEndpoint(String endpointUri, org.apache.camel.Component component)
-
Method Summary
All Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description void
configureExchange(org.apache.camel.Exchange exchange)
Set up the exchange properties with the options of the file endpointvoid
configureMessage(GenericFile<T> file, org.apache.camel.Message message)
Configures the given message with the file which sets the body to the file object.protected String
configureMoveOrPreMoveExpression(String expression)
Strategy to configure the move, preMove, or moveExisting option based on a String input.abstract GenericFileConsumer<T>
createConsumer(org.apache.camel.Processor processor)
protected String
createDoneFileName(String fileName)
Creates the associated name of the done file based on the given file name.abstract org.apache.camel.Exchange
createExchange(GenericFile<T> file)
protected abstract GenericFileProcessStrategy<T>
createGenericFileStrategy()
A strategy method to lazily create the file strategyabstract GenericFileProducer<T>
createProducer()
protected void
doInit()
protected void
doStart()
protected void
doStop()
String
getAntExclude()
GenericFileFilter<T>
getAntFilter()
String
getAntInclude()
String
getAppendChars()
int
getBufferSize()
String
getCharset()
GenericFileConfiguration
getConfiguration()
String
getDoneFileName()
List<org.apache.camel.Exchange>
getExchanges()
This implementation will not load the file content.String
getExclude()
String
getExcludeExt()
Pattern
getExcludePattern()
GenericFileExclusiveReadLockStrategy<T>
getExclusiveReadLockStrategy()
GenericFileExist
getFileExist()
org.apache.camel.Expression
getFileName()
abstract char
getFileSeparator()
GenericFileFilter<T>
getFilter()
org.apache.camel.Predicate
getFilterDirectory()
org.apache.camel.Predicate
getFilterFile()
String
getGeneratedFileName(org.apache.camel.Message message)
Return the file name that will be auto-generated for the given message if none is providedBoolean
getIdempotent()
org.apache.camel.Expression
getIdempotentKey()
org.apache.camel.spi.IdempotentRepository
getIdempotentRepository()
String
getInclude()
String
getIncludeExt()
Pattern
getIncludePattern()
org.apache.camel.spi.IdempotentRepository
getInProgressRepository()
String
getLocalWorkDirectory()
int
getMaxDepth()
int
getMaxMessagesPerPoll()
int
getMinDepth()
org.apache.camel.Expression
getMove()
org.apache.camel.Expression
getMoveExisting()
FileMoveExistingStrategy
getMoveExistingFileStrategy()
org.apache.camel.Expression
getMoveFailed()
org.apache.camel.spi.ExceptionHandler
getOnCompletionExceptionHandler()
protected Map<String,Object>
getParamsAsMap()
org.apache.camel.Expression
getPreMove()
GenericFileProcessStrategy<T>
getProcessStrategy()
String
getReadLock()
long
getReadLockCheckInterval()
int
getReadLockIdempotentReleaseAsyncPoolSize()
int
getReadLockIdempotentReleaseDelay()
ScheduledExecutorService
getReadLockIdempotentReleaseExecutorService()
org.apache.camel.LoggingLevel
getReadLockLoggingLevel()
long
getReadLockMinAge()
long
getReadLockMinLength()
long
getReadLockTimeout()
abstract String
getScheme()
Comparator<org.apache.camel.Exchange>
getSortBy()
Comparator<GenericFile<T>>
getSorter()
org.apache.camel.Expression
getTempFileName()
String
getTempPrefix()
abstract boolean
isAbsolute(String name)
boolean
isAllowNullBody()
boolean
isAntFilterCaseSensitive()
boolean
isAutoCreate()
boolean
isDelete()
protected boolean
isDoneFile(String fileName)
Is the given file a done file?boolean
isEagerDeleteTargetFile()
boolean
isEagerMaxMessagesPerPoll()
boolean
isFlatten()
Boolean
isIdempotent()
protected boolean
isIdempotentSet()
boolean
isJailStartingDirectory()
boolean
isKeepLastModified()
boolean
isNoop()
boolean
isPreSort()
boolean
isReadLockDeleteOrphanLockFiles()
boolean
isReadLockIdempotentReleaseAsync()
boolean
isReadLockMarkerFile()
boolean
isReadLockRemoveOnCommit()
boolean
isReadLockRemoveOnRollback()
boolean
isRecursive()
boolean
isShuffle()
boolean
isSynchronous()
void
setAllowNullBody(boolean allowNullBody)
Used to specify if a null body is allowed during file writing.void
setAntExclude(String antExclude)
Ant style filter exclusion.void
setAntFilterCaseSensitive(boolean antFilterCaseSensitive)
Sets case sensitive flag on ant filtervoid
setAntInclude(String antInclude)
Ant style filter inclusion.void
setAppendChars(String appendChars)
Used to append characters (text) after writing files.void
setAutoCreate(boolean autoCreate)
Automatically create missing directories in the file's pathname.void
setBufferSize(int bufferSize)
Buffer size in bytes used for writing files (or in case of FTP for downloading and uploading files).void
setCharset(String charset)
This option is used to specify the encoding of the file.void
setConfiguration(GenericFileConfiguration configuration)
void
setDelete(boolean delete)
If true, the file will be deleted after it is processed successfully.void
setDoneFileName(String doneFileName)
Producer: If provided, then Camel will write a 2nd done file when the original file has been written.void
setEagerDeleteTargetFile(boolean eagerDeleteTargetFile)
Whether or not to eagerly delete any existing target file.void
setEagerMaxMessagesPerPoll(boolean eagerMaxMessagesPerPoll)
Allows for controlling whether the limit from maxMessagesPerPoll is eager or not.void
setExclude(String exclude)
Is used to exclude files, if filename matches the regex pattern (matching is case in-senstive).void
setExcludeExt(String excludeExt)
Is used to exclude files matching file extension name (case insensitive).void
setExclusiveReadLockStrategy(GenericFileExclusiveReadLockStrategy<T> exclusiveReadLockStrategy)
Pluggable read-lock as a org.apache.camel.component.file.GenericFileExclusiveReadLockStrategy implementation.void
setFileExist(GenericFileExist fileExist)
What to do if a file already exists with the same name.void
setFileName(String fileLanguageExpression)
void
setFileName(org.apache.camel.Expression fileName)
Use Expression such as File Language to dynamically set the filename.void
setFilter(GenericFileFilter<T> filter)
Pluggable filter as a org.apache.camel.component.file.GenericFileFilter class.void
setFilterDirectory(String expression)
Filters the directory based on Simple language.void
setFilterDirectory(org.apache.camel.Predicate filterDirectory)
Filters the directory based on Simple language.void
setFilterFile(String expression)
Filters the file based on Simple language.void
setFilterFile(org.apache.camel.Predicate filterFile)
Filters the file based on Simple language.void
setFlatten(boolean flatten)
Flatten is used to flatten the file name path to strip any leading paths, so it's just the file name.void
setIdempotent(Boolean idempotent)
Option to use the Idempotent Consumer EIP pattern to let Camel skip already processed files.void
setIdempotentKey(String expression)
void
setIdempotentKey(org.apache.camel.Expression idempotentKey)
To use a custom idempotent key.void
setIdempotentRepository(org.apache.camel.spi.IdempotentRepository idempotentRepository)
A pluggable repository org.apache.camel.spi.IdempotentRepository which by default use MemoryIdempotentRepository if none is specified and idempotent is true.void
setInclude(String include)
Is used to include files, if filename matches the regex pattern (matching is case in-sensitive).void
setIncludeExt(String includeExt)
Is used to include files matching file extension name (case insensitive).void
setInProgressRepository(org.apache.camel.spi.IdempotentRepository inProgressRepository)
A pluggable in-progress repository org.apache.camel.spi.IdempotentRepository.void
setJailStartingDirectory(boolean jailStartingDirectory)
Used for jailing (restricting) writing files to the starting directory (and sub) only.void
setKeepLastModified(boolean keepLastModified)
Will keep the last modified timestamp from the source file (if any).void
setLocalWorkDirectory(String localWorkDirectory)
When consuming, a local work directory can be used to store the remote file content directly in local files, to avoid loading the content into memory.void
setMaxDepth(int maxDepth)
The maximum depth to traverse when recursively processing a directory.void
setMaxMessagesPerPoll(int maxMessagesPerPoll)
To define a maximum messages to gather per poll.void
setMinDepth(int minDepth)
The minimum depth to start processing when recursively processing a directory.void
setMove(String fileLanguageExpression)
void
setMove(org.apache.camel.Expression move)
Expression (such as Simple Language) used to dynamically set the filename when moving it after processing.void
setMoveExisting(String fileLanguageExpression)
void
setMoveExisting(org.apache.camel.Expression moveExisting)
Expression (such as File Language) used to compute file name to use when fileExist=Move is configured.void
setMoveExistingFileStrategy(FileMoveExistingStrategy moveExistingFileStrategy)
Strategy (Custom Strategy) used to move file with special naming token to use when fileExist=Move is configured.void
setMoveFailed(String fileLanguageExpression)
void
setMoveFailed(org.apache.camel.Expression moveFailed)
Sets the move failure expression based on Simple language.void
setNoop(boolean noop)
If true, the file is not moved or deleted in any way.void
setOnCompletionExceptionHandler(org.apache.camel.spi.ExceptionHandler onCompletionExceptionHandler)
To use a customExceptionHandler
to handle any thrown exceptions that happens during the file on completion process where the consumer does either a commit or rollback.void
setPreMove(String fileLanguageExpression)
void
setPreMove(org.apache.camel.Expression preMove)
Expression (such as File Language) used to dynamically set the filename when moving it before processing.void
setPreSort(boolean preSort)
When pre-sort is enabled then the consumer will sort the file and directory names during polling, that was retrieved from the file system.void
setProcessStrategy(GenericFileProcessStrategy<T> processStrategy)
A pluggable org.apache.camel.component.file.GenericFileProcessStrategy allowing you to implement your own readLock option or similar.void
setReadLock(String readLock)
Used by consumer, to only poll the files if it has exclusive read-lock on the file (i.e.void
setReadLockCheckInterval(long readLockCheckInterval)
Interval in millis for the read-lock, if supported by the read lock.void
setReadLockDeleteOrphanLockFiles(boolean readLockDeleteOrphanLockFiles)
Whether or not read lock with marker files should upon startup delete any orphan read lock files, which may have been left on the file system, if Camel was not properly shutdown (such as a JVM crash).void
setReadLockIdempotentReleaseAsync(boolean readLockIdempotentReleaseAsync)
Whether the delayed release task should be synchronous or asynchronous.void
setReadLockIdempotentReleaseAsyncPoolSize(int readLockIdempotentReleaseAsyncPoolSize)
The number of threads in the scheduled thread pool when using asynchronous release tasks.void
setReadLockIdempotentReleaseDelay(int readLockIdempotentReleaseDelay)
Whether to delay the release task for a period of millis.void
setReadLockIdempotentReleaseExecutorService(ScheduledExecutorService readLockIdempotentReleaseExecutorService)
To use a custom and shared thread pool for asynchronous release tasks.void
setReadLockLoggingLevel(org.apache.camel.LoggingLevel readLockLoggingLevel)
Logging level used when a read lock could not be acquired.void
setReadLockMarkerFile(boolean readLockMarkerFile)
Whether to use marker file with the changed, rename, or exclusive read lock types.void
setReadLockMinAge(long readLockMinAge)
This option is applied only for readLock=changed.void
setReadLockMinLength(long readLockMinLength)
This option is applied only for readLock=changed.void
setReadLockRemoveOnCommit(boolean readLockRemoveOnCommit)
This option is applied only for readLock=idempotent.void
setReadLockRemoveOnRollback(boolean readLockRemoveOnRollback)
This option is applied only for readLock=idempotent.void
setReadLockTimeout(long readLockTimeout)
Optional timeout in millis for the read-lock, if supported by the read-lock.void
setRecursive(boolean recursive)
If a directory, will look for files in all the sub-directories as well.void
setShuffle(boolean shuffle)
To shuffle the list of files (sort in random order)void
setSortBy(String expression)
void
setSortBy(String expression, boolean reverse)
void
setSortBy(Comparator<org.apache.camel.Exchange> sortBy)
Built-in sort by using the File Language.void
setSorter(Comparator<GenericFile<T>> sorter)
Pluggable sorter as a java.util.Comparatorclass. void
setSynchronous(boolean synchronous)
void
setTempFileName(String tempFileNameExpression)
void
setTempFileName(org.apache.camel.Expression tempFileName)
The same as tempPrefix option but offering a more fine grained control on the naming of the temporary filename as it uses the File Language.void
setTempPrefix(String tempPrefix)
This option is used to write the file using a temporary name and then, after the write is complete, rename it to the real name.-
Methods inherited from class org.apache.camel.support.ScheduledPollEndpoint
configureConsumer, configureProperties, configureScheduledPollConsumerProperties, doConfigureConsumer, getBackoffErrorThreshold, getBackoffIdleThreshold, getBackoffMultiplier, getDefaultDelay, getDelay, getInitialDelay, getPollStrategy, getRepeatCount, getRunLoggingLevel, getScheduledExecutorService, getScheduler, getSchedulerProperties, getTimeUnit, isGreedy, isSendEmptyMessageWhenIdle, isStartScheduler, isUseFixedDelay, setBackoffErrorThreshold, setBackoffIdleThreshold, setBackoffMultiplier, setDelay, setGreedy, setInitialDelay, setPollStrategy, setRepeatCount, setRunLoggingLevel, setScheduledExecutorService, setScheduler, setSchedulerProperties, setSendEmptyMessageWhenIdle, setStartScheduler, setTimeUnit, setUseFixedDelay
-
Methods inherited from class org.apache.camel.support.DefaultEndpoint
configurePollingConsumer, createAsyncProducer, createEndpointUri, createExchange, createExchange, createPollingConsumer, equals, getCamelContext, getComponent, getEndpointKey, getEndpointUri, getExceptionHandler, getExchangePattern, getId, getPollingConsumerBlockTimeout, getPollingConsumerQueueSize, hashCode, isAutowiredEnabled, isBridgeErrorHandler, isLazyStartProducer, isLenientProperties, isPollingConsumerBlockWhenFull, isPollingConsumerCopy, isSingleton, setAutowiredEnabled, setBridgeErrorHandler, setCamelContext, setComponent, setEndpointUri, setEndpointUriIfNotSpecified, setExceptionHandler, setExchangePattern, setLazyStartProducer, setPollingConsumerBlockTimeout, setPollingConsumerBlockWhenFull, setPollingConsumerCopy, setPollingConsumerQueueSize, setProperties, toString
-
Methods inherited from class org.apache.camel.support.service.BaseService
build, doBuild, doFail, doLifecycleChange, doResume, doShutdown, doSuspend, fail, getStatus, init, isBuild, isInit, isNew, isRunAllowed, isShutdown, isStarted, isStarting, isStartingOrStarted, isStopped, isStopping, isStoppingOrStopped, isSuspended, isSuspending, isSuspendingOrSuspended, resume, shutdown, start, stop, suspend
-
Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait
-
Methods inherited from interface org.apache.camel.Endpoint
configureProperties, createAsyncProducer, createExchange, createExchange, createPollingConsumer, getCamelContext, getEndpointBaseUri, getEndpointKey, getEndpointUri, getExchangePattern, isLenientProperties, isSingletonProducer, setCamelContext
-
-
-
-
Field Detail
-
DEFAULT_IDEMPOTENT_CACHE_SIZE
protected static final int DEFAULT_IDEMPOTENT_CACHE_SIZE
- See Also:
- Constant Field Values
-
DEFAULT_IN_PROGRESS_CACHE_SIZE
protected static final int DEFAULT_IN_PROGRESS_CACHE_SIZE
- See Also:
- Constant Field Values
-
autoCreate
@UriParam(label="advanced", defaultValue="true", description="Automatically create missing directories in the file\'s pathname. For the file consumer, that means creating the starting directory. For the file producer, it means the directory the files should be written to.") protected boolean autoCreate
-
bufferSize
@UriParam(label="advanced", defaultValue="131072", description="Buffer size in bytes used for writing files (or in case of FTP for downloading and uploading files).") protected int bufferSize
-
charset
@UriParam(description="This option is used to specify the encoding of the file. You can use this on the consumer, to specify the encodings of the files, which allow Camel to know the charset it should load the file content in case the file content is being accessed. Likewise when writing a file, you can use this option to specify which charset to write the file as well. Do mind that when writing the file Camel may have to read the message content into memory to be able to convert the data into the configured charset, so do not use this if you have big messages.") protected String charset
-
fileName
@UriParam(javaType="java.lang.String", description="Use Expression such as File Language to dynamically set the filename. For consumers, it\'s used as a filename filter. For producers, it\'s used to evaluate the filename to write. If an expression is set, it take precedence over the CamelFileName header. (Note: The header itself can also be an Expression). The expression options support both String and Expression types. If the expression is a String type, it is always evaluated using the File Language. If the expression is an Expression type, the specified Expression type is used - this allows you, for instance, to use OGNL expressions. For the consumer, you can use it to filter filenames, so you can for instance consume today\'s file using the File Language syntax: mydata-${date:now:yyyyMMdd}.txt. The producers support the CamelOverruleFileName header which takes precedence over any existing CamelFileName header; the CamelOverruleFileName is a header that is used only once, and makes it easier as this avoids to temporary store CamelFileName and have to restore it afterwards.") protected org.apache.camel.Expression fileName
-
doneFileName
@UriParam(description="Producer: If provided, then Camel will write a 2nd done file when the original file has been written. The done file will be empty. This option configures what file name to use. Either you can specify a fixed name. Or you can use dynamic placeholders. The done file will always be written in the same folder as the original file.<p/> Consumer: If provided, Camel will only consume files if a done file exists. This option configures what file name to use. Either you can specify a fixed name. Or you can use dynamic placeholders.The done file is always expected in the same folder as the original file.<p/> Only ${file.name} and ${file.name.next} is supported as dynamic placeholders.") protected String doneFileName
-
flatten
@UriParam(label="producer", description="Flatten is used to flatten the file name path to strip any leading paths, so it\'s just the file name. This allows you to consume recursively into sub-directories, but when you eg write the files to another directory they will be written in a single directory. Setting this to true on the producer enforces that any file name in CamelFileName header will be stripped for any leading paths.") protected boolean flatten
-
fileExist
@UriParam(label="producer", defaultValue="Override", description="What to do if a file already exists with the same name. Override, which is the default, replaces the existing file.<p/> - Append - adds content to the existing file.<p/> - Fail - throws a GenericFileOperationException, indicating that there is already an existing file.<p/> - Ignore - silently ignores the problem and does not override the existing file, but assumes everything is okay.<p/> - Move - option requires to use the moveExisting option to be configured as well. The option eagerDeleteTargetFile can be used to control what to do if an moving the file, and there exists already an existing file, otherwise causing the move operation to fail. The Move option will move any existing files, before writing the target file.<p/> - TryRename is only applicable if tempFileName option is in use. This allows to try renaming the file from the temporary name to the actual name, without doing any exists check. This check may be faster on some file systems and especially FTP servers.") protected GenericFileExist fileExist
-
tempPrefix
@UriParam(label="producer", description="This option is used to write the file using a temporary name and then, after the write is complete, rename it to the real name. Can be used to identify files being written and also avoid consumers (not using exclusive read locks) reading in progress files. Is often used by FTP when uploading big files.") protected String tempPrefix
-
tempFileName
@UriParam(label="producer", javaType="java.lang.String", description="The same as tempPrefix option but offering a more fine grained control on the naming of the temporary filename as it uses the File Language. The location for tempFilename is relative to the final file location in the option \'fileName\', not the target directory in the base uri. For example if option fileName includes a directory prefix: dir/finalFilename then tempFileName is relative to that subdirectory dir.") protected org.apache.camel.Expression tempFileName
-
eagerDeleteTargetFile
@UriParam(label="producer,advanced", defaultValue="true", description="Whether or not to eagerly delete any existing target file. This option only applies when you use fileExists=Override and the tempFileName option as well. You can use this to disable (set it to false) deleting the target file before the temp file is written. For example you may write big files and want the target file to exists during the temp file is being written. This ensure the target file is only deleted until the very last moment, just before the temp file is being renamed to the target filename. This option is also used to control whether to delete any existing files when fileExist=Move is enabled, and an existing file exists. If this option copyAndDeleteOnRenameFails false, then an exception will be thrown if an existing file existed, if its true, then the existing file is deleted before the move operation.") protected boolean eagerDeleteTargetFile
-
keepLastModified
@UriParam(label="producer,advanced", description="Will keep the last modified timestamp from the source file (if any). Will use the FileConstants.FILE_LAST_MODIFIED header to located the timestamp. This header can contain either a java.util.Date or long with the timestamp. If the timestamp exists and the option is enabled it will set this timestamp on the written file. Note: This option only applies to the file producer. You cannot use this option with any of the ftp producers.") protected boolean keepLastModified
-
allowNullBody
@UriParam(label="producer,advanced", description="Used to specify if a null body is allowed during file writing. If set to true then an empty file will be created, when set to false, and attempting to send a null body to the file component, a GenericFileWriteException of \'Cannot write null body to file.\' will be thrown. If the `fileExist` option is set to \'Override\', then the file will be truncated, and if set to `append` the file will remain unchanged.") protected boolean allowNullBody
-
jailStartingDirectory
@UriParam(label="producer", defaultValue="true", description="Used for jailing (restricting) writing files to the starting directory (and sub) only. This is enabled by default to not allow Camel to write files to outside directories (to be more secured out of the box). You can turn this off to allow writing files to directories outside the starting directory, such as parent or root folders.") protected boolean jailStartingDirectory
-
appendChars
@UriParam(label="producer", description="Used to append characters (text) after writing files. This can for example be used to add new lines or other separators when writing and appending new files or existing files. <p/> To specify new-line (slash-n or slash-r) or tab (slash-t) characters then escape with an extra slash, eg slash-slash-n.") protected String appendChars
-
configuration
@UriParam protected GenericFileConfiguration configuration
-
processStrategy
@UriParam(label="consumer,advanced", description="A pluggable org.apache.camel.component.file.GenericFileProcessStrategy allowing you to implement your own readLock option or similar. Can also be used when special conditions must be met before a file can be consumed, such as a special ready file exists. If this option is set then the readLock option does not apply.") protected GenericFileProcessStrategy<T> processStrategy
-
inProgressRepository
@UriParam(label="consumer,advanced", description="A pluggable in-progress repository org.apache.camel.spi.IdempotentRepository. The in-progress repository is used to account the current in progress files being consumed. By default a memory based repository is used.") protected org.apache.camel.spi.IdempotentRepository inProgressRepository
-
localWorkDirectory
@UriParam(label="consumer,advanced", description="When consuming, a local work directory can be used to store the remote file content directly in local files, to avoid loading the content into memory. This is beneficial, if you consume a very big remote file and thus can conserve memory.") protected String localWorkDirectory
-
noop
@UriParam(label="consumer", description="If true, the file is not moved or deleted in any way. This option is good for readonly data, or for ETL type requirements. If noop=true, Camel will set idempotent=true as well, to avoid consuming the same files over and over again.") protected boolean noop
-
recursive
@UriParam(label="consumer", description="If a directory, will look for files in all the sub-directories as well.") protected boolean recursive
-
delete
@UriParam(label="consumer", description="If true, the file will be deleted after it is processed successfully.") protected boolean delete
-
preSort
@UriParam(label="consumer", description="When pre-sort is enabled then the consumer will sort the file and directory names during polling, that was retrieved from the file system. You may want to do this in case you need to operate on the files in a sorted order. The pre-sort is executed before the consumer starts to filter, and accept files to process by Camel. This option is default=false meaning disabled.") protected boolean preSort
-
maxMessagesPerPoll
@UriParam(label="consumer,filter", description="To define a maximum messages to gather per poll. By default no maximum is set. Can be used to set a limit of e.g. 1000 to avoid when starting up the server that there are thousands of files. Set a value of 0 or negative to disabled it. Notice: If this option is in use then the File and FTP components will limit before any sorting. For example if you have 100000 files and use maxMessagesPerPoll=500, then only the first 500 files will be picked up, and then sorted. You can use the eagerMaxMessagesPerPoll option and set this to false to allow to scan all files first and then sort afterwards.") protected int maxMessagesPerPoll
-
eagerMaxMessagesPerPoll
@UriParam(label="consumer,filter", defaultValue="true", description="Allows for controlling whether the limit from maxMessagesPerPoll is eager or not. If eager then the limit is during the scanning of files. Where as false would scan all files, and then perform sorting. Setting this option to false allows for sorting all files first, and then limit the poll. Mind that this requires a higher memory usage as all file details are in memory to perform the sorting.") protected boolean eagerMaxMessagesPerPoll
-
maxDepth
@UriParam(label="consumer,filter", defaultValue="2147483647", description="The maximum depth to traverse when recursively processing a directory.") protected int maxDepth
-
minDepth
@UriParam(label="consumer,filter", description="The minimum depth to start processing when recursively processing a directory. Using minDepth=1 means the base directory. Using minDepth=2 means the first sub directory.") protected int minDepth
-
include
@UriParam(label="consumer,filter", description="Is used to include files, if filename matches the regex pattern (matching is case in-sensitive). <p/> Notice if you use symbols such as plus sign and others you would need to configure this using the RAW() syntax if configuring this as an endpoint uri. See more details at <a href=\"http://camel.apache.org/how-do-i-configure-endpoints.html\">configuring endpoint uris</a>") protected String include
-
exclude
@UriParam(label="consumer,filter", description="Is used to exclude files, if filename matches the regex pattern (matching is case in-sensitive). <p/> Notice if you use symbols such as plus sign and others you would need to configure this using the RAW() syntax if configuring this as an endpoint uri. See more details at <a href=\"http://camel.apache.org/how-do-i-configure-endpoints.html\">configuring endpoint uris</a>") protected String exclude
-
includeExt
@UriParam(label="consumer,filter", description="Is used to include files matching file extension name (case insensitive). For example to include txt files, then use includeExt=txt. Multiple extensions can be separated by comma, for example to include txt and xml files, use includeExt=txt,xml. Note that the file extension includes all parts, for example having a file named mydata.tar.gz will have extension as tar.gz. For more flexibility then use the include/exclude options.") protected String includeExt
-
excludeExt
@UriParam(label="consumer,filter", description="Is used to exclude files matching file extension name (case insensitive). For example to exclude bak files, then use excludeExt=bak. Multiple extensions can be separated by comma, for example to exclude bak and dat files, use excludeExt=bak,dat. Note that the file extension includes all parts, for example having a file named mydata.tar.gz will have extension as tar.gz. For more flexibility then use the include/exclude options.") protected String excludeExt
-
move
@UriParam(label="consumer,filter", javaType="java.lang.String", description="Expression (such as Simple Language) used to dynamically set the filename when moving it after processing. To move files into a .done subdirectory just enter .done.") protected org.apache.camel.Expression move
-
moveFailed
@UriParam(label="consumer", javaType="java.lang.String", description="Sets the move failure expression based on Simple language. For example, to move files into a .error subdirectory use: .error. Note: When moving the files to the fail location Camel will handle the error and will not pick up the file again.") protected org.apache.camel.Expression moveFailed
-
preMove
@UriParam(label="consumer", javaType="java.lang.String", description="Expression (such as File Language) used to dynamically set the filename when moving it before processing. For example to move in-progress files into the order directory set this value to order.") protected org.apache.camel.Expression preMove
-
moveExisting
@UriParam(label="producer", javaType="java.lang.String", description="Expression (such as File Language) used to compute file name to use when fileExist=Move is configured. To move files into a backup subdirectory just enter backup. This option only supports the following File Language tokens: \"file:name\", \"file:name.ext\", \"file:name.noext\", \"file:onlyname\", \"file:onlyname.noext\", \"file:ext\", and \"file:parent\". Notice the \"file:parent\" is not supported by the FTP component, as the FTP component can only move any existing files to a relative directory based on current dir as base.") protected org.apache.camel.Expression moveExisting
-
moveExistingFileStrategy
@UriParam(label="producer,advanced", description="Strategy (Custom Strategy) used to move file with special naming token to use when fileExist=Move is configured. By default, there is an implementation used if no custom strategy is provided") protected FileMoveExistingStrategy moveExistingFileStrategy
-
idempotent
@UriParam(label="consumer,filter", defaultValue="false", description="Option to use the Idempotent Consumer EIP pattern to let Camel skip already processed files. Will by default use a memory based LRUCache that holds 1000 entries. If noop=true then idempotent will be enabled as well to avoid consuming the same files over and over again.") protected Boolean idempotent
-
idempotentKey
@UriParam(label="consumer,filter", javaType="java.lang.String", description="To use a custom idempotent key. By default the absolute path of the file is used. You can use the File Language, for example to use the file name and file size, you can do: idempotentKey=${file:name}-${file:size}") protected org.apache.camel.Expression idempotentKey
-
idempotentRepository
@UriParam(label="consumer,filter", description="A pluggable repository org.apache.camel.spi.IdempotentRepository which by default use MemoryIdempotentRepository if none is specified and idempotent is true.") protected org.apache.camel.spi.IdempotentRepository idempotentRepository
-
filter
@UriParam(label="consumer,filter", description="Pluggable filter as a org.apache.camel.component.file.GenericFileFilter class. Will skip files if filter returns false in its accept() method.") protected GenericFileFilter<T> filter
-
filterDirectory
@UriParam(label="consumer,filter", javaType="java.lang.String", description="Filters the directory based on Simple language. For example to filter on current date, you can use a simple date pattern such as ${date:now:yyyMMdd}") protected org.apache.camel.Predicate filterDirectory
-
filterFile
@UriParam(label="consumer,filter", javaType="java.lang.String", description="Filters the file based on Simple language. For example to filter on file size, you can use ${file:size} > 5000") protected org.apache.camel.Predicate filterFile
-
antFilterCaseSensitive
@UriParam(label="consumer,filter", defaultValue="true", description="Sets case sensitive flag on ant filter.") protected boolean antFilterCaseSensitive
-
antFilter
protected volatile AntPathMatcherGenericFileFilter<T> antFilter
-
antInclude
@UriParam(label="consumer,filter", description="Ant style filter inclusion. Multiple inclusions may be specified in comma-delimited format.") protected String antInclude
-
antExclude
@UriParam(label="consumer,filter", description="Ant style filter exclusion. If both antInclude and antExclude are used, antExclude takes precedence over antInclude. Multiple exclusions may be specified in comma-delimited format.") protected String antExclude
-
sorter
@UriParam(label="consumer,sort", description="Pluggable sorter as a java.util.Comparator<org.apache.camel.component.file.GenericFile> class.") protected Comparator<GenericFile<T>> sorter
-
sortBy
@UriParam(label="consumer,sort", javaType="java.lang.String", description="Built-in sort by using the File Language. Supports nested sorts, so you can have a sort by file name and as a 2nd group sort by modified date.") protected Comparator<org.apache.camel.Exchange> sortBy
-
shuffle
@UriParam(label="consumer,sort", description="To shuffle the list of files (sort in random order)") protected boolean shuffle
-
readLock
@UriParam(label="consumer,lock", defaultValue="none", enums="none,markerFile,fileLock,rename,changed,idempotent,idempotent-changed,idempotent-rename", description="Used by consumer, to only poll the files if it has exclusive read-lock on the file (i.e. the file is not in-progress or being written). Camel will wait until the file lock is granted. This option provides the build in strategies:<p/> - none - No read lock is in use<p/> - markerFile - Camel creates a marker file (fileName.camelLock) and then holds a lock on it. This option is not available for the FTP component<p/> - changed - Changed is using file length/modification timestamp to detect whether the file is currently being copied or not. Will at least use 1 sec to determine this, so this option cannot consume files as fast as the others, but can be more reliable as the JDK IO API cannot always determine whether a file is currently being used by another process. The option readLockCheckInterval can be used to set the check frequency.<p/> - fileLock - is for using java.nio.channels.FileLock. This option is not avail for Windows OS and the FTP component. This approach should be avoided when accessing a remote file system via a mount/share unless that file system supports distributed file locks.<p/> - rename - rename is for using a try to rename the file as a test if we can get exclusive read-lock.<p/> - idempotent - (only for file component) idempotent is for using a idempotentRepository as the read-lock. This allows to use read locks that supports clustering if the idempotent repository implementation supports that.<p/> - idempotent-changed - (only for file component) idempotent-changed is for using a idempotentRepository and changed as the combined read-lock. This allows to use read locks that supports clustering if the idempotent repository implementation supports that.<p/> - idempotent-rename - (only for file component) idempotent-rename is for using a idempotentRepository and rename as the combined read-lock. This allows to use read locks that supports clustering if the idempotent repository implementation supports that.<p/>Notice: The various read locks is not all suited to work in clustered mode, where concurrent consumers on different nodes is competing for the same files on a shared file system. The markerFile using a close to atomic operation to create the empty marker file, but its not guaranteed to work in a cluster. The fileLock may work better but then the file system need to support distributed file locks, and so on. Using the idempotent read lock can support clustering if the idempotent repository supports clustering, such as Hazelcast Component or Infinispan.") protected String readLock
-
readLockCheckInterval
@UriParam(label="consumer,lock", defaultValue="1000", description="Interval in millis for the read-lock, if supported by the read lock. This interval is used for sleeping between attempts to acquire the read lock. For example when using the changed read lock, you can set a higher interval period to cater for slow writes. The default of 1 sec. may be too fast if the producer is very slow writing the file. <p/>Notice: For FTP the default readLockCheckInterval is 5000. <p/> The readLockTimeout value must be higher than readLockCheckInterval, but a rule of thumb is to have a timeout that is at least 2 or more times higher than the readLockCheckInterval. This is needed to ensure that amble time is allowed for the read lock process to try to grab the lock before the timeout was hit.") protected long readLockCheckInterval
-
readLockTimeout
@UriParam(label="consumer,lock", defaultValue="10000", description="Optional timeout in millis for the read-lock, if supported by the read-lock. If the read-lock could not be granted and the timeout triggered, then Camel will skip the file. At next poll Camel, will try the file again, and this time maybe the read-lock could be granted. Use a value of 0 or lower to indicate forever. Currently fileLock, changed and rename support the timeout. <p/> Notice: For FTP the default readLockTimeout value is 20000 instead of 10000. <p/> The readLockTimeout value must be higher than readLockCheckInterval, but a rule of thumb is to have a timeout that is at least 2 or more times higher than the readLockCheckInterval. This is needed to ensure that amble time is allowed for the read lock process to try to grab the lock before the timeout was hit.") protected long readLockTimeout
-
readLockMarkerFile
@UriParam(label="consumer,lock", defaultValue="true", description="Whether to use marker file with the changed, rename, or exclusive read lock types. By default a marker file is used as well to guard against other processes picking up the same files. This behavior can be turned off by setting this option to false. For example if you do not want to write marker files to the file systems by the Camel application.") protected boolean readLockMarkerFile
-
readLockDeleteOrphanLockFiles
@UriParam(label="consumer,lock", defaultValue="true", description="Whether or not read lock with marker files should upon startup delete any orphan read lock files, which may have been left on the file system, if Camel was not properly shutdown (such as a JVM crash). <p/> If turning this option to <tt>false</tt> then any orphaned lock file will cause Camel to not attempt to pickup that file, this could also be due another node is concurrently reading files from the same shared directory.") protected boolean readLockDeleteOrphanLockFiles
-
readLockLoggingLevel
@UriParam(label="consumer,lock", defaultValue="DEBUG", description="Logging level used when a read lock could not be acquired. By default a DEBUG is logged. You can change this level, for example to OFF to not have any logging. This option is only applicable for readLock of types: changed, fileLock, idempotent, idempotent-changed, idempotent-rename, rename.") protected org.apache.camel.LoggingLevel readLockLoggingLevel
-
readLockMinLength
@UriParam(label="consumer,lock", defaultValue="1", description="This option is applied only for readLock=changed. It allows you to configure a minimum file length. By default Camel expects the file to contain data, and thus the default value is 1. You can set this option to zero, to allow consuming zero-length files.") protected long readLockMinLength
-
readLockMinAge
@UriParam(label="consumer,lock", defaultValue="0", description="This option is applied only for readLock=changed. It allows to specify a minimum age the file must be before attempting to acquire the read lock. For example use readLockMinAge=300s to require the file is at last 5 minutes old. This can speedup the changed read lock as it will only attempt to acquire files which are at least that given age.") protected long readLockMinAge
-
readLockRemoveOnRollback
@UriParam(label="consumer,lock", defaultValue="true", description="This option is applied only for readLock=idempotent. It allows to specify whether to remove the file name entry from the idempotent repository when processing the file failed and a rollback happens. If this option is false, then the file name entry is confirmed (as if the file did a commit).") protected boolean readLockRemoveOnRollback
-
readLockRemoveOnCommit
@UriParam(label="consumer,lock", description="This option is applied only for readLock=idempotent. It allows to specify whether to remove the file name entry from the idempotent repository when processing the file is succeeded and a commit happens. <p/> By default the file is not removed which ensures that any race-condition do not occur so another active node may attempt to grab the file. Instead the idempotent repository may support eviction strategies that you can configure to evict the file name entry after X minutes - this ensures no problems with race conditions. <p/> See more details at the readLockIdempotentReleaseDelay option.") protected boolean readLockRemoveOnCommit
-
readLockIdempotentReleaseDelay
@UriParam(label="consumer,lock", description="Whether to delay the release task for a period of millis. <p/> This can be used to delay the release tasks to expand the window when a file is regarded as read-locked, in an active/active cluster scenario with a shared idempotent repository, to ensure other nodes cannot potentially scan and acquire the same file, due to race-conditions. By expanding the time-window of the release tasks helps prevents these situations. Note delaying is only needed if you have configured readLockRemoveOnCommit to true.") protected int readLockIdempotentReleaseDelay
-
readLockIdempotentReleaseAsync
@UriParam(label="consumer,lock", description="Whether the delayed release task should be synchronous or asynchronous. <p/> See more details at the readLockIdempotentReleaseDelay option.") protected boolean readLockIdempotentReleaseAsync
-
readLockIdempotentReleaseAsyncPoolSize
@UriParam(label="consumer,lock", description="The number of threads in the scheduled thread pool when using asynchronous release tasks. Using a default of 1 core threads should be sufficient in almost all use-cases, only set this to a higher value if either updating the idempotent repository is slow, or there are a lot of files to process. This option is not in-use if you use a shared thread pool by configuring the readLockIdempotentReleaseExecutorService option. <p/> See more details at the readLockIdempotentReleaseDelay option.") protected int readLockIdempotentReleaseAsyncPoolSize
-
readLockIdempotentReleaseExecutorService
@UriParam(label="consumer,lock", description="To use a custom and shared thread pool for asynchronous release tasks. <p/> See more details at the readLockIdempotentReleaseDelay option.") protected ScheduledExecutorService readLockIdempotentReleaseExecutorService
-
exclusiveReadLockStrategy
@UriParam(label="consumer,lock", description="Pluggable read-lock as a org.apache.camel.component.file.GenericFileExclusiveReadLockStrategy implementation.") protected GenericFileExclusiveReadLockStrategy<T> exclusiveReadLockStrategy
-
onCompletionExceptionHandler
@UriParam(label="consumer,advanced", description="To use a custom {@link org.apache.camel.spi.ExceptionHandler} to handle any thrown exceptions that happens during the file on completion process where the consumer does either a commit or rollback. The default implementation will log any exception at WARN level and ignore.") protected org.apache.camel.spi.ExceptionHandler onCompletionExceptionHandler
-
-
Constructor Detail
-
GenericFileEndpoint
public GenericFileEndpoint()
-
GenericFileEndpoint
public GenericFileEndpoint(String endpointUri, org.apache.camel.Component component)
-
-
Method Detail
-
createConsumer
public abstract GenericFileConsumer<T> createConsumer(org.apache.camel.Processor processor) throws Exception
- Specified by:
createConsumer
in interfaceorg.apache.camel.Endpoint
- Throws:
Exception
-
createProducer
public abstract GenericFileProducer<T> createProducer() throws Exception
- Specified by:
createProducer
in interfaceorg.apache.camel.Endpoint
- Throws:
Exception
-
createExchange
public abstract org.apache.camel.Exchange createExchange(GenericFile<T> file)
-
getScheme
public abstract String getScheme()
-
getFileSeparator
public abstract char getFileSeparator()
-
isAbsolute
public abstract boolean isAbsolute(String name)
-
getGeneratedFileName
public String getGeneratedFileName(org.apache.camel.Message message)
Return the file name that will be auto-generated for the given message if none is provided
-
getExchanges
public List<org.apache.camel.Exchange> getExchanges()
This implementation will not load the file content. Any file locking is neither in use by this implementation..- Specified by:
getExchanges
in interfaceorg.apache.camel.spi.BrowsableEndpoint
-
createGenericFileStrategy
protected abstract GenericFileProcessStrategy<T> createGenericFileStrategy()
A strategy method to lazily create the file strategy
-
isNoop
public boolean isNoop()
-
setNoop
public void setNoop(boolean noop)
If true, the file is not moved or deleted in any way. This option is good for readonly data, or for ETL type requirements. If noop=true, Camel will set idempotent=true as well, to avoid consuming the same files over and over again.
-
isRecursive
public boolean isRecursive()
-
setRecursive
public void setRecursive(boolean recursive)
If a directory, will look for files in all the sub-directories as well.
-
getInclude
public String getInclude()
-
setInclude
public void setInclude(String include)
Is used to include files, if filename matches the regex pattern (matching is case in-sensitive). Notice if you use symbols such as plus sign and others you would need to configure this using the RAW() syntax if configuring this as an endpoint uri. See more details at configuring endpoint uris
-
getIncludePattern
public Pattern getIncludePattern()
-
getExclude
public String getExclude()
-
setExclude
public void setExclude(String exclude)
Is used to exclude files, if filename matches the regex pattern (matching is case in-senstive). Notice if you use symbols such as plus sign and others you would need to configure this using the RAW() syntax if configuring this as an endpoint uri. See more details at configuring endpoint uris
-
getExcludePattern
public Pattern getExcludePattern()
-
getAntInclude
public String getAntInclude()
-
setAntInclude
public void setAntInclude(String antInclude)
Ant style filter inclusion. Multiple inclusions may be specified in comma-delimited format.
-
getAntExclude
public String getAntExclude()
-
setAntExclude
public void setAntExclude(String antExclude)
Ant style filter exclusion. If both antInclude and antExclude are used, antExclude takes precedence over antInclude. Multiple exclusions may be specified in comma-delimited format.
-
isAntFilterCaseSensitive
public boolean isAntFilterCaseSensitive()
-
setAntFilterCaseSensitive
public void setAntFilterCaseSensitive(boolean antFilterCaseSensitive)
Sets case sensitive flag on ant filter
-
getAntFilter
public GenericFileFilter<T> getAntFilter()
-
getIncludeExt
public String getIncludeExt()
-
setIncludeExt
public void setIncludeExt(String includeExt)
Is used to include files matching file extension name (case insensitive). For example to include txt files, then use includeExt=txt. Multiple extensions can be separated by comma, for example to include txt and xml files, use includeExt=txt,xml
-
getExcludeExt
public String getExcludeExt()
-
setExcludeExt
public void setExcludeExt(String excludeExt)
Is used to exclude files matching file extension name (case insensitive). For example to exclude bak files, then use excludeExt=bak. Multiple extensions can be separated by comma, for example to exclude bak and dat files, use excludeExt=bak,dat.
-
isPreSort
public boolean isPreSort()
-
setPreSort
public void setPreSort(boolean preSort)
When pre-sort is enabled then the consumer will sort the file and directory names during polling, that was retrieved from the file system. You may want to do this in case you need to operate on the files in a sorted order. The pre-sort is executed before the consumer starts to filter, and accept files to process by Camel. This option is default=false meaning disabled.
-
isDelete
public boolean isDelete()
-
setDelete
public void setDelete(boolean delete)
If true, the file will be deleted after it is processed successfully.
-
isFlatten
public boolean isFlatten()
-
setFlatten
public void setFlatten(boolean flatten)
Flatten is used to flatten the file name path to strip any leading paths, so it's just the file name. This allows you to consume recursively into sub-directories, but when you eg write the files to another directory they will be written in a single directory. Setting this to true on the producer enforces that any file name in CamelFileName header will be stripped for any leading paths.
-
getMove
public org.apache.camel.Expression getMove()
-
setMove
public void setMove(org.apache.camel.Expression move)
Expression (such as Simple Language) used to dynamically set the filename when moving it after processing. To move files into a .done subdirectory just enter .done.
-
setMove
public void setMove(String fileLanguageExpression)
- See Also:
setMove(org.apache.camel.Expression)
-
getMoveFailed
public org.apache.camel.Expression getMoveFailed()
-
setMoveFailed
public void setMoveFailed(org.apache.camel.Expression moveFailed)
Sets the move failure expression based on Simple language. For example, to move files into a .error subdirectory use: .error. Note: When moving the files to the fail location Camel will handle the error and will not pick up the file again.
-
setMoveFailed
public void setMoveFailed(String fileLanguageExpression)
-
getFilterDirectory
public org.apache.camel.Predicate getFilterDirectory()
-
setFilterDirectory
public void setFilterDirectory(org.apache.camel.Predicate filterDirectory)
Filters the directory based on Simple language. For example to filter on current date, you can use a simple date pattern such as ${date:now:yyyMMdd}
-
setFilterDirectory
public void setFilterDirectory(String expression)
Filters the directory based on Simple language. For example to filter on current date, you can use a simple date pattern such as ${date:now:yyyMMdd}- See Also:
setFilterDirectory(Predicate)
-
getFilterFile
public org.apache.camel.Predicate getFilterFile()
-
setFilterFile
public void setFilterFile(org.apache.camel.Predicate filterFile)
Filters the file based on Simple language. For example to filter on file size, you can use ${file:size} > 5000
-
setFilterFile
public void setFilterFile(String expression)
Filters the file based on Simple language. For example to filter on file size, you can use ${file:size} > 5000- See Also:
setFilterFile(Predicate)
-
getPreMove
public org.apache.camel.Expression getPreMove()
-
setPreMove
public void setPreMove(org.apache.camel.Expression preMove)
Expression (such as File Language) used to dynamically set the filename when moving it before processing. For example to move in-progress files into the order directory set this value to order.
-
setPreMove
public void setPreMove(String fileLanguageExpression)
-
getMoveExisting
public org.apache.camel.Expression getMoveExisting()
-
setMoveExisting
public void setMoveExisting(org.apache.camel.Expression moveExisting)
Expression (such as File Language) used to compute file name to use when fileExist=Move is configured. To move files into a backup subdirectory just enter backup. This option only supports the following File Language tokens: "file:name", "file:name.ext", "file:name.noext", "file:onlyname", "file:onlyname.noext", "file:ext", and "file:parent". Notice the "file:parent" is not supported by the FTP component, as the FTP component can only move any existing files to a relative directory based on current dir as base.
-
getMoveExistingFileStrategy
public FileMoveExistingStrategy getMoveExistingFileStrategy()
-
setMoveExistingFileStrategy
public void setMoveExistingFileStrategy(FileMoveExistingStrategy moveExistingFileStrategy)
Strategy (Custom Strategy) used to move file with special naming token to use when fileExist=Move is configured. By default, there is an implementation used if no custom strategy is provided
-
setMoveExisting
public void setMoveExisting(String fileLanguageExpression)
-
getFileName
public org.apache.camel.Expression getFileName()
-
setFileName
public void setFileName(org.apache.camel.Expression fileName)
Use Expression such as File Language to dynamically set the filename. For consumers, it's used as a filename filter. For producers, it's used to evaluate the filename to write. If an expression is set, it take precedence over the CamelFileName header. (Note: The header itself can also be an Expression). The expression options support both String and Expression types. If the expression is a String type, it is always evaluated using the File Language. If the expression is an Expression type, the specified Expression type is used - this allows you, for instance, to use OGNL expressions. For the consumer, you can use it to filter filenames, so you can for instance consume today's file using the File Language syntax: mydata-${date:now:yyyyMMdd}.txt. The producers support the CamelOverruleFileName header which takes precedence over any existing CamelFileName header; the CamelOverruleFileName is a header that is used only once, and makes it easier as this avoids to temporary store CamelFileName and have to restore it afterwards.
-
setFileName
public void setFileName(String fileLanguageExpression)
-
getDoneFileName
public String getDoneFileName()
-
setDoneFileName
public void setDoneFileName(String doneFileName)
Producer: If provided, then Camel will write a 2nd done file when the original file has been written. The done file will be empty. This option configures what file name to use. Either you can specify a fixed name. Or you can use dynamic placeholders. The done file will always be written in the same folder as the original file. Consumer: If provided, Camel will only consume files if a done file exists. This option configures what file name to use. Either you can specify a fixed name. Or you can use dynamic placeholders.The done file is always expected in the same folder as the original file. Only ${file.name} and ${file.name.noext} is supported as dynamic placeholders.
-
isIdempotent
public Boolean isIdempotent()
-
getCharset
public String getCharset()
-
setCharset
public void setCharset(String charset)
This option is used to specify the encoding of the file. You can use this on the consumer, to specify the encodings of the files, which allow Camel to know the charset it should load the file content in case the file content is being accessed. Likewise when writing a file, you can use this option to specify which charset to write the file as well. Do mind that when writing the file Camel may have to read the message content into memory to be able to convert the data into the configured charset, so do not use this if you have big messages.
-
isIdempotentSet
protected boolean isIdempotentSet()
-
getIdempotent
public Boolean getIdempotent()
-
setIdempotent
public void setIdempotent(Boolean idempotent)
Option to use the Idempotent Consumer EIP pattern to let Camel skip already processed files. Will by default use a memory based LRUCache that holds 1000 entries. If noop=true then idempotent will be enabled as well to avoid consuming the same files over and over again.
-
getIdempotentKey
public org.apache.camel.Expression getIdempotentKey()
-
setIdempotentKey
public void setIdempotentKey(org.apache.camel.Expression idempotentKey)
To use a custom idempotent key. By default the absolute path of the file is used. You can use the File Language, for example to use the file name and file size, you can do: idempotentKey=${file:name}-${file:size}
-
setIdempotentKey
public void setIdempotentKey(String expression)
-
getIdempotentRepository
public org.apache.camel.spi.IdempotentRepository getIdempotentRepository()
-
setIdempotentRepository
public void setIdempotentRepository(org.apache.camel.spi.IdempotentRepository idempotentRepository)
A pluggable repository org.apache.camel.spi.IdempotentRepository which by default use MemoryIdempotentRepository if none is specified and idempotent is true.
-
getFilter
public GenericFileFilter<T> getFilter()
-
setFilter
public void setFilter(GenericFileFilter<T> filter)
Pluggable filter as a org.apache.camel.component.file.GenericFileFilter class. Will skip files if filter returns false in its accept() method.
-
getSorter
public Comparator<GenericFile<T>> getSorter()
-
setSorter
public void setSorter(Comparator<GenericFile<T>> sorter)
Pluggable sorter as a java.util.Comparatorclass.
-
getSortBy
public Comparator<org.apache.camel.Exchange> getSortBy()
-
setSortBy
public void setSortBy(Comparator<org.apache.camel.Exchange> sortBy)
Built-in sort by using the File Language. Supports nested sorts, so you can have a sort by file name and as a 2nd group sort by modified date.
-
setSortBy
public void setSortBy(String expression)
-
setSortBy
public void setSortBy(String expression, boolean reverse)
-
isShuffle
public boolean isShuffle()
-
setShuffle
public void setShuffle(boolean shuffle)
To shuffle the list of files (sort in random order)
-
getTempPrefix
public String getTempPrefix()
-
setTempPrefix
public void setTempPrefix(String tempPrefix)
This option is used to write the file using a temporary name and then, after the write is complete, rename it to the real name. Can be used to identify files being written and also avoid consumers (not using exclusive read locks) reading in progress files. Is often used by FTP when uploading big files.
-
getTempFileName
public org.apache.camel.Expression getTempFileName()
-
setTempFileName
public void setTempFileName(org.apache.camel.Expression tempFileName)
The same as tempPrefix option but offering a more fine grained control on the naming of the temporary filename as it uses the File Language. The location for tempFilename is relative to the final file location in the option 'fileName', not the target directory in the base uri. For example if option fileName includes a directory prefix: dir/finalFilename then tempFileName is relative to that subdirectory dir.
-
setTempFileName
public void setTempFileName(String tempFileNameExpression)
-
isEagerDeleteTargetFile
public boolean isEagerDeleteTargetFile()
-
setEagerDeleteTargetFile
public void setEagerDeleteTargetFile(boolean eagerDeleteTargetFile)
Whether or not to eagerly delete any existing target file. This option only applies when you use fileExists=Override and the tempFileName option as well. You can use this to disable (set it to false) deleting the target file before the temp file is written. For example you may write big files and want the target file to exists during the temp file is being written. This ensure the target file is only deleted until the very last moment, just before the temp file is being renamed to the target filename. This option is also used to control whether to delete any existing files when fileExist=Move is enabled, and an existing file exists. If this option copyAndDeleteOnRenameFails false, then an exception will be thrown if an existing file existed, if its true, then the existing file is deleted before the move operation.
-
getConfiguration
public GenericFileConfiguration getConfiguration()
-
setConfiguration
public void setConfiguration(GenericFileConfiguration configuration)
-
getExclusiveReadLockStrategy
public GenericFileExclusiveReadLockStrategy<T> getExclusiveReadLockStrategy()
-
setExclusiveReadLockStrategy
public void setExclusiveReadLockStrategy(GenericFileExclusiveReadLockStrategy<T> exclusiveReadLockStrategy)
Pluggable read-lock as a org.apache.camel.component.file.GenericFileExclusiveReadLockStrategy implementation.
-
getReadLock
public String getReadLock()
-
setReadLock
public void setReadLock(String readLock)
Used by consumer, to only poll the files if it has exclusive read-lock on the file (i.e. the file is not in-progress or being written). Camel will wait until the file lock is granted. This option provides the build in strategies:\n\n - none - No read lock is in use\n - markerFile - Camel creates a marker file (fileName.camelLock) and then holds a lock on it. This option is not available for the FTP component\n - changed - Changed is using file length/modification timestamp to detect whether the file is currently being copied or not. Will at least use 1 sec to determine this, so this option cannot consume files as fast as the others, but can be more reliable as the JDK IO API cannot always determine whether a file is currently being used by another process. The option readLockCheckInterval can be used to set the check frequency.\n - fileLock - is for using java.nio.channels.FileLock. This option is not avail for Windows OS and the FTP component. This approach should be avoided when accessing a remote file system via a mount/share unless that file system supports distributed file locks.\n - rename - rename is for using a try to rename the file as a test if we can get exclusive read-lock.\n - idempotent - (only for file component) idempotent is for using a idempotentRepository as the read-lock. This allows to use read locks that supports clustering if the idempotent repository implementation supports that.\n - idempotent-changed - (only for file component) idempotent-changed is for using a idempotentRepository and changed as the combined read-lock. This allows to use read locks that supports clustering if the idempotent repository implementation supports that.\n - idempotent-rename - (only for file component) idempotent-rename is for using a idempotentRepository and rename as the combined read-lock. This allows to use read locks that supports clustering if the idempotent repository implementation supports that.\n \nNotice: The various read locks is not all suited to work in clustered mode, where concurrent consumers on different nodes is competing for the same files on a shared file system. The markerFile using a close to atomic operation to create the empty marker file, but its not guaranteed to work in a cluster. The fileLock may work better but then the file system need to support distributed file locks, and so on. Using the idempotent read lock can support clustering if the idempotent repository supports clustering, such as Hazelcast Component or Infinispan.
-
getReadLockCheckInterval
public long getReadLockCheckInterval()
-
setReadLockCheckInterval
public void setReadLockCheckInterval(long readLockCheckInterval)
Interval in millis for the read-lock, if supported by the read lock. This interval is used for sleeping between attempts to acquire the read lock. For example when using the changed read lock, you can set a higher interval period to cater for slow writes. The default of 1 sec. may be too fast if the producer is very slow writing the file. Notice: For FTP the default readLockCheckInterval is 5000. The readLockTimeout value must be higher than readLockCheckInterval, but a rule of thumb is to have a timeout that is at least 2 or more times higher than the readLockCheckInterval. This is needed to ensure that amble time is allowed for the read lock process to try to grab the lock before the timeout was hit.
-
getReadLockTimeout
public long getReadLockTimeout()
-
setReadLockTimeout
public void setReadLockTimeout(long readLockTimeout)
Optional timeout in millis for the read-lock, if supported by the read-lock. If the read-lock could not be granted and the timeout triggered, then Camel will skip the file. At next poll Camel, will try the file again, and this time maybe the read-lock could be granted. Use a value of 0 or lower to indicate forever. Currently fileLock, changed and rename support the timeout. Notice: For FTP the default readLockTimeout value is 20000 instead of 10000. The readLockTimeout value must be higher than readLockCheckInterval, but a rule of thumb is to have a timeout that is at least 2 or more times higher than the readLockCheckInterval. This is needed to ensure that amble time is allowed for the read lock process to try to grab the lock before the timeout was hit.
-
isReadLockMarkerFile
public boolean isReadLockMarkerFile()
-
setReadLockMarkerFile
public void setReadLockMarkerFile(boolean readLockMarkerFile)
Whether to use marker file with the changed, rename, or exclusive read lock types. By default a marker file is used as well to guard against other processes picking up the same files. This behavior can be turned off by setting this option to false. For example if you do not want to write marker files to the file systems by the Camel application.
-
isReadLockDeleteOrphanLockFiles
public boolean isReadLockDeleteOrphanLockFiles()
-
setReadLockDeleteOrphanLockFiles
public void setReadLockDeleteOrphanLockFiles(boolean readLockDeleteOrphanLockFiles)
Whether or not read lock with marker files should upon startup delete any orphan read lock files, which may have been left on the file system, if Camel was not properly shutdown (such as a JVM crash). If turning this option to false then any orphaned lock file will cause Camel to not attempt to pickup that file, this could also be due another node is concurrently reading files from the same shared directory.
-
getReadLockLoggingLevel
public org.apache.camel.LoggingLevel getReadLockLoggingLevel()
-
setReadLockLoggingLevel
public void setReadLockLoggingLevel(org.apache.camel.LoggingLevel readLockLoggingLevel)
Logging level used when a read lock could not be acquired. By default a DEBUG is logged. You can change this level, for example to OFF to not have any logging. This option is only applicable for readLock of types: changed, fileLock, idempotent, idempotent-changed, idempotent-rename, rename.
-
getReadLockMinLength
public long getReadLockMinLength()
-
setReadLockMinLength
public void setReadLockMinLength(long readLockMinLength)
This option is applied only for readLock=changed. It allows you to configure a minimum file length. By default Camel expects the file to contain data, and thus the default value is 1. You can set this option to zero, to allow consuming zero-length files.
-
getReadLockMinAge
public long getReadLockMinAge()
-
setReadLockMinAge
public void setReadLockMinAge(long readLockMinAge)
This option is applied only for readLock=changed. It allows to specify a minimum age the file must be before attempting to acquire the read lock. For example use readLockMinAge=300s to require the file is at last 5 minutes old. This can speedup the changed read lock as it will only attempt to acquire files which are at least that given age.
-
isReadLockRemoveOnRollback
public boolean isReadLockRemoveOnRollback()
-
setReadLockRemoveOnRollback
public void setReadLockRemoveOnRollback(boolean readLockRemoveOnRollback)
This option is applied only for readLock=idempotent. It allows to specify whether to remove the file name entry from the idempotent repository when processing the file failed and a rollback happens. If this option is false, then the file name entry is confirmed (as if the file did a commit).
-
isReadLockRemoveOnCommit
public boolean isReadLockRemoveOnCommit()
-
setReadLockRemoveOnCommit
public void setReadLockRemoveOnCommit(boolean readLockRemoveOnCommit)
This option is applied only for readLock=idempotent. It allows to specify whether to remove the file name entry from the idempotent repository when processing the file is succeeded and a commit happens. By default the file is not removed which ensures that any race-condition do not occur so another active node may attempt to grab the file. Instead the idempotent repository may support eviction strategies that you can configure to evict the file name entry after X minutes - this ensures no problems with race conditions. See more details at the readLockIdempotentReleaseDelay option.
-
getReadLockIdempotentReleaseDelay
public int getReadLockIdempotentReleaseDelay()
-
setReadLockIdempotentReleaseDelay
public void setReadLockIdempotentReleaseDelay(int readLockIdempotentReleaseDelay)
Whether to delay the release task for a period of millis. This can be used to delay the release tasks to expand the window when a file is regarded as read-locked, in an active/active cluster scenario with a shared idempotent repository, to ensure other nodes cannot potentially scan and acquire the same file, due to race-conditions. By expanding the time-window of the release tasks helps prevents these situations. Note delaying is only needed if you have configured readLockRemoveOnCommit to true.
-
isReadLockIdempotentReleaseAsync
public boolean isReadLockIdempotentReleaseAsync()
-
setReadLockIdempotentReleaseAsync
public void setReadLockIdempotentReleaseAsync(boolean readLockIdempotentReleaseAsync)
Whether the delayed release task should be synchronous or asynchronous. See more details at the readLockIdempotentReleaseDelay option.
-
getReadLockIdempotentReleaseAsyncPoolSize
public int getReadLockIdempotentReleaseAsyncPoolSize()
-
setReadLockIdempotentReleaseAsyncPoolSize
public void setReadLockIdempotentReleaseAsyncPoolSize(int readLockIdempotentReleaseAsyncPoolSize)
The number of threads in the scheduled thread pool when using asynchronous release tasks. Using a default of 1 core threads should be sufficient in almost all use-cases, only set this to a higher value if either updating the idempotent repository is slow, or there are a lot of files to process. This option is not in-use if you use a shared thread pool by configuring the readLockIdempotentReleaseExecutorService option. See more details at the readLockIdempotentReleaseDelay option.
-
getReadLockIdempotentReleaseExecutorService
public ScheduledExecutorService getReadLockIdempotentReleaseExecutorService()
-
setReadLockIdempotentReleaseExecutorService
public void setReadLockIdempotentReleaseExecutorService(ScheduledExecutorService readLockIdempotentReleaseExecutorService)
To use a custom and shared thread pool for asynchronous release tasks. See more details at the readLockIdempotentReleaseDelay option.
-
getBufferSize
public int getBufferSize()
-
setBufferSize
public void setBufferSize(int bufferSize)
Buffer size in bytes used for writing files (or in case of FTP for downloading and uploading files).
-
getFileExist
public GenericFileExist getFileExist()
-
setFileExist
public void setFileExist(GenericFileExist fileExist)
What to do if a file already exists with the same name. Override, which is the default, replaces the existing file. \n\n - Append - adds content to the existing file.\n - Fail - throws a GenericFileOperationException, indicating that there is already an existing file.\n - Ignore - silently ignores the problem and does not override the existing file, but assumes everything is okay.\n - Move - option requires to use the moveExisting option to be configured as well. The option eagerDeleteTargetFile can be used to control what to do if an moving the file, and there exists already an existing file, otherwise causing the move operation to fail. The Move option will move any existing files, before writing the target file.\n - TryRename is only applicable if tempFileName option is in use. This allows to try renaming the file from the temporary name to the actual name, without doing any exists check. This check may be faster on some file systems and especially FTP servers.
-
isAutoCreate
public boolean isAutoCreate()
-
setAutoCreate
public void setAutoCreate(boolean autoCreate)
Automatically create missing directories in the file's pathname. For the file consumer, that means creating the starting directory. For the file producer, it means the directory the files should be written to.
-
getProcessStrategy
public GenericFileProcessStrategy<T> getProcessStrategy()
-
setProcessStrategy
public void setProcessStrategy(GenericFileProcessStrategy<T> processStrategy)
A pluggable org.apache.camel.component.file.GenericFileProcessStrategy allowing you to implement your own readLock option or similar. Can also be used when special conditions must be met before a file can be consumed, such as a special ready file exists. If this option is set then the readLock option does not apply.
-
getLocalWorkDirectory
public String getLocalWorkDirectory()
-
setLocalWorkDirectory
public void setLocalWorkDirectory(String localWorkDirectory)
When consuming, a local work directory can be used to store the remote file content directly in local files, to avoid loading the content into memory. This is beneficial, if you consume a very big remote file and thus can conserve memory.
-
getMaxMessagesPerPoll
public int getMaxMessagesPerPoll()
-
setMaxMessagesPerPoll
public void setMaxMessagesPerPoll(int maxMessagesPerPoll)
To define a maximum messages to gather per poll. By default no maximum is set. Can be used to set a limit of e.g. 1000 to avoid when starting up the server that there are thousands of files. Set a value of 0 or negative to disabled it. Notice: If this option is in use then the File and FTP components will limit before any sorting. For example if you have 100000 files and use maxMessagesPerPoll=500, then only the first 500 files will be picked up, and then sorted. You can use the eagerMaxMessagesPerPoll option and set this to false to allow to scan all files first and then sort afterwards.
-
isEagerMaxMessagesPerPoll
public boolean isEagerMaxMessagesPerPoll()
-
setEagerMaxMessagesPerPoll
public void setEagerMaxMessagesPerPoll(boolean eagerMaxMessagesPerPoll)
Allows for controlling whether the limit from maxMessagesPerPoll is eager or not. If eager then the limit is during the scanning of files. Where as false would scan all files, and then perform sorting. Setting this option to false allows for sorting all files first, and then limit the poll. Mind that this requires a higher memory usage as all file details are in memory to perform the sorting.
-
getMaxDepth
public int getMaxDepth()
-
setMaxDepth
public void setMaxDepth(int maxDepth)
The maximum depth to traverse when recursively processing a directory.
-
getMinDepth
public int getMinDepth()
-
setMinDepth
public void setMinDepth(int minDepth)
The minimum depth to start processing when recursively processing a directory. Using minDepth=1 means the base directory. Using minDepth=2 means the first sub directory.
-
getInProgressRepository
public org.apache.camel.spi.IdempotentRepository getInProgressRepository()
-
setInProgressRepository
public void setInProgressRepository(org.apache.camel.spi.IdempotentRepository inProgressRepository)
A pluggable in-progress repository org.apache.camel.spi.IdempotentRepository. The in-progress repository is used to account the current in progress files being consumed. By default a memory based repository is used.
-
isKeepLastModified
public boolean isKeepLastModified()
-
setKeepLastModified
public void setKeepLastModified(boolean keepLastModified)
Will keep the last modified timestamp from the source file (if any). Will use the FileConstants.FILE_LAST_MODIFIED header to located the timestamp. This header can contain either a java.util.Date or long with the timestamp. If the timestamp exists and the option is enabled it will set this timestamp on the written file. Note: This option only applies to the file producer. You cannot use this option with any of the ftp producers.
-
isAllowNullBody
public boolean isAllowNullBody()
-
setAllowNullBody
public void setAllowNullBody(boolean allowNullBody)
Used to specify if a null body is allowed during file writing. If set to true then an empty file will be created, when set to false, and attempting to send a null body to the file component, a GenericFileWriteException of 'Cannot write null body to file.' will be thrown. If the `fileExist` option is set to 'Override', then the file will be truncated, and if set to `append` the file will remain unchanged.
-
isJailStartingDirectory
public boolean isJailStartingDirectory()
-
setJailStartingDirectory
public void setJailStartingDirectory(boolean jailStartingDirectory)
Used for jailing (restricting) writing files to the starting directory (and sub) only. This is enabled by default to not allow Camel to write files to outside directories (to be more secured out of the box). You can turn this off to allow writing files to directories outside the starting directory, such as parent or root folders.
-
getAppendChars
public String getAppendChars()
-
setAppendChars
public void setAppendChars(String appendChars)
Used to append characters (text) after writing files. This can for example be used to add new lines or other separators when writing and appending to new files or existing files. To specify new-line (slash-n or slash-r) or tab (slash-t) characters then escape with an extra slash, eg slash-slash-n
-
getOnCompletionExceptionHandler
public org.apache.camel.spi.ExceptionHandler getOnCompletionExceptionHandler()
-
setOnCompletionExceptionHandler
public void setOnCompletionExceptionHandler(org.apache.camel.spi.ExceptionHandler onCompletionExceptionHandler)
To use a customExceptionHandler
to handle any thrown exceptions that happens during the file on completion process where the consumer does either a commit or rollback. The default implementation will log any exception at WARN level and ignore.
-
isSynchronous
public boolean isSynchronous()
-
setSynchronous
public void setSynchronous(boolean synchronous)
-
configureMessage
public void configureMessage(GenericFile<T> file, org.apache.camel.Message message)
Configures the given message with the file which sets the body to the file object.
-
configureExchange
public void configureExchange(org.apache.camel.Exchange exchange)
Set up the exchange properties with the options of the file endpoint- Specified by:
configureExchange
in interfaceorg.apache.camel.Endpoint
- Overrides:
configureExchange
in classorg.apache.camel.support.DefaultEndpoint
-
configureMoveOrPreMoveExpression
protected String configureMoveOrPreMoveExpression(String expression)
Strategy to configure the move, preMove, or moveExisting option based on a String input.- Parameters:
expression
- the original string input- Returns:
- configured string or the original if no modifications is needed
-
createDoneFileName
protected String createDoneFileName(String fileName)
Creates the associated name of the done file based on the given file name. This method should only be invoked if a done filename property has been set on this endpoint.- Parameters:
fileName
- the file name- Returns:
- name of the associated done file name
-
isDoneFile
protected boolean isDoneFile(String fileName)
Is the given file a done file? This method should only be invoked if a done filename property has been set on this endpoint.- Parameters:
fileName
- the file name- Returns:
- true if its a done file, false otherwise
-
doInit
protected void doInit() throws Exception
- Overrides:
doInit
in classorg.apache.camel.support.DefaultEndpoint
- Throws:
Exception
-
doStart
protected void doStart() throws Exception
- Overrides:
doStart
in classorg.apache.camel.support.ScheduledPollEndpoint
- Throws:
Exception
-
-