@InterfaceAudience.Public @InterfaceStability.Stable public class SequenceFileInputFilter<K,V> extends SequenceFileInputFormat<K,V>
Modifier and Type | Class and Description |
---|---|
static interface |
SequenceFileInputFilter.Filter
filter interface
|
static class |
SequenceFileInputFilter.FilterBase
base class for Filters
|
static class |
SequenceFileInputFilter.MD5Filter
This class returns a set of records by examing the MD5 digest of its
key against a filtering frequency f.
|
static class |
SequenceFileInputFilter.PercentFilter
This class returns a percentage of records
The percentage is determined by a filtering frequency f using
the criteria record# % f == 0.
|
static class |
SequenceFileInputFilter.RegexFilter
Records filter by matching key to regex
|
FileInputFormat.Counter
Modifier and Type | Field and Description |
---|---|
static String |
FILTER_CLASS |
static String |
FILTER_FREQUENCY |
static String |
FILTER_REGEX |
static org.apache.commons.logging.Log |
LOG |
DEFAULT_LIST_STATUS_NUM_THREADS, INPUT_DIR, INPUT_DIR_RECURSIVE, LIST_STATUS_NUM_THREADS, NUM_INPUT_FILES, PATHFILTER_CLASS, SPLIT_MAXSIZE, SPLIT_MINSIZE
Constructor and Description |
---|
SequenceFileInputFilter() |
Modifier and Type | Method and Description |
---|---|
RecordReader<K,V> |
createRecordReader(InputSplit split,
TaskAttemptContext context)
Create a record reader for the given split
|
static void |
setFilterClass(Job job,
Class<?> filterClass)
set the filter class
|
getFormatMinSplitSize, listStatus
addInputPath, addInputPathRecursively, addInputPaths, computeSplitSize, getBlockIndex, getInputDirRecursive, getInputPathFilter, getInputPaths, getMaxSplitSize, getMinSplitSize, getSplits, isSplitable, makeSplit, makeSplit, setInputDirRecursive, setInputPathFilter, setInputPaths, setInputPaths, setMaxInputSplitSize, setMinInputSplitSize
public static final org.apache.commons.logging.Log LOG
public static final String FILTER_CLASS
public static final String FILTER_FREQUENCY
public static final String FILTER_REGEX
public RecordReader<K,V> createRecordReader(InputSplit split, TaskAttemptContext context) throws IOException
createRecordReader
in class SequenceFileInputFormat<K,V>
split
- file splitcontext
- the task-attempt contextIOException
Copyright © 2016 Apache Software Foundation. All Rights Reserved.