T - the type of the elements read from the sourcepublic abstract class FileBasedReader<T> extends Reader<T>
| Modifier and Type | Class and Description |
|---|---|
static interface |
FileBasedReader.DecompressingStreamFactory
Factory interface for creating a decompressing
InputStream. |
protected class |
FileBasedReader.FileBasedIterator
Abstract base class for file-based source iterators.
|
protected static class |
FileBasedReader.FilenameBasedStreamFactory
Factory for creating decompressing input streams based on a filename and
a
TextIO.CompressionType. |
Reader.AbstractReaderIterator<T>, Reader.DynamicSplitRequest, Reader.DynamicSplitResult, Reader.DynamicSplitResultWithPosition, Reader.Position, Reader.Progress, Reader.ReaderIterator<T>| Modifier and Type | Field and Description |
|---|---|
protected static int |
BUF_SIZE |
protected Coder<T> |
coder |
protected Long |
endPosition |
protected String |
filename |
protected Long |
startPosition |
protected boolean |
useDefaultBufferSize |
stateSampler, stateSamplerOperationName| Modifier | Constructor and Description |
|---|---|
protected |
FileBasedReader(String filename,
Long startPosition,
Long endPosition,
Coder<T> coder,
boolean useDefaultBufferSize) |
| Modifier and Type | Method and Description |
|---|---|
Reader.ReaderIterator<T> |
iterator()
Returns a ReaderIterator that allows reading from this source.
|
protected abstract Reader.ReaderIterator<T> |
newReaderIteratorForFiles(IOChannelFactory factory,
Collection<String> files)
Returns a new iterator for elements in the given files.
|
protected abstract Reader.ReaderIterator<T> |
newReaderIteratorForRangeInFile(IOChannelFactory factory,
String oneFile,
long startPosition,
Long endPosition)
Returns a new iterator for elements in the given range in the
given file.
|
notifyElementRead, setStateSamplerAndOperationName, supportsRestartaddObserver, clearChanged, countObservers, deleteObserver, deleteObservers, hasChanged, notifyObservers, notifyObservers, setChangedprotected static final int BUF_SIZE
protected final String filename
protected final boolean useDefaultBufferSize
protected abstract Reader.ReaderIterator<T> newReaderIteratorForRangeInFile(IOChannelFactory factory, String oneFile, long startPosition, @Nullable Long endPosition) throws IOException
endPosition - offset of the end position; null means end-of-fileIOExceptionprotected abstract Reader.ReaderIterator<T> newReaderIteratorForFiles(IOChannelFactory factory, Collection<String> files) throws IOException
IOExceptionpublic Reader.ReaderIterator<T> iterator() throws IOException
Readeriterator in class Reader<T>IOException