Package org.apache.druid.data.input
Interface InputSource
-
- All Known Subinterfaces:
SplittableInputSource<T>
- All Known Implementing Classes:
AbstractInputSource
,CloudObjectInputSource
,CombiningInputSource
,HttpInputSource
,InlineInputSource
,LocalInputSource
public interface InputSource
InputSource abstracts the storage system where input data is stored. It creates anInputSourceReader
to read data from the given input source. The most common use case would be:InputSourceReader reader = inputSource.reader(); try (CloseableIterator<InputRow> iterator = reader.read()) { while (iterator.hasNext()) { InputRow row = iterator.next(); processRow(row); } }
-
-
Field Summary
Fields Modifier and Type Field Description static String
TYPE_PROPERTY
-
Method Summary
All Methods Instance Methods Abstract Methods Default Methods Modifier and Type Method Description default Set<String>
getTypes()
The types of input sources uses.boolean
isSplittable()
Returns true if this inputSource can be processed in parallel using ParallelIndexSupervisorTask.boolean
needsFormat()
Returns true if this inputSource supports differentInputFormat
s.InputSourceReader
reader(InputRowSchema inputRowSchema, InputFormat inputFormat, File temporaryDirectory)
Creates anInputSourceReader
.
-
-
-
Field Detail
-
TYPE_PROPERTY
static final String TYPE_PROPERTY
- See Also:
- Constant Field Values
-
-
Method Detail
-
isSplittable
boolean isSplittable()
Returns true if this inputSource can be processed in parallel using ParallelIndexSupervisorTask. It must be castable to SplittableInputSource and the various SplittableInputSource methods must work as documented.
-
needsFormat
boolean needsFormat()
Returns true if this inputSource supports differentInputFormat
s. Some inputSources such asLocalInputSource
can store files of any format. These storage types require anInputFormat
to be passed so thatInputSourceReader
can parse data properly. However, some storage types have a fixed format. For example, druid inputSource always reads segments. These inputSources should return false for this method.
-
reader
InputSourceReader reader(InputRowSchema inputRowSchema, @Nullable InputFormat inputFormat, File temporaryDirectory)
Creates anInputSourceReader
.- Parameters:
inputRowSchema
- forInputRow
inputFormat
- to parse data. It can be null ifneedsFormat()
= truetemporaryDirectory
- to store temp data. It will be cleaned up automatically once the task is finished.
-
-