Package | Description |
---|---|
org.apache.hadoop.mapred | |
org.apache.hadoop.mapred.lib | |
org.apache.hadoop.mapred.lib.aggregate | |
org.apache.hadoop.mapreduce.task.reduce |
Modifier and Type | Method and Description |
---|---|
Class<? extends Reducer> |
ShuffleConsumerPlugin.Context.getCombinerClass() |
Class<? extends Reducer> |
JobConf.getCombinerClass()
Get the user-defined combiner class used to combine map-outputs
before being sent to the reducers.
|
Class<? extends Reducer> |
JobConf.getReducerClass()
Get the
Reducer class for the job. |
Modifier and Type | Method and Description |
---|---|
void |
JobConf.setCombinerClass(Class<? extends Reducer> theClass)
Set the user-defined combiner class used to combine map-outputs
before being sent to the reducers.
|
void |
JobConf.setReducerClass(Class<? extends Reducer> theClass)
Set the
Reducer class for the job. |
Constructor and Description |
---|
Context(TaskAttemptID reduceId,
JobConf jobConf,
org.apache.hadoop.fs.FileSystem localFS,
TaskUmbilicalProtocol umbilical,
org.apache.hadoop.fs.LocalDirAllocator localDirAllocator,
Reporter reporter,
org.apache.hadoop.io.compress.CompressionCodec codec,
Class<? extends Reducer> combinerClass,
Task.CombineOutputCollector<K,V> combineCollector,
Counters.Counter spilledRecordsCounter,
Counters.Counter reduceCombineInputCounter,
Counters.Counter shuffledMapsCounter,
Counters.Counter reduceShuffleBytes,
Counters.Counter failedShuffleCounter,
Counters.Counter mergedMapOutputsCounter,
TaskStatus status,
org.apache.hadoop.util.Progress copyPhase,
org.apache.hadoop.util.Progress mergePhase,
Task reduceTask,
MapOutputFile mapOutputFile,
Map<TaskAttemptID,MapOutputFile> localMapFiles) |
OldCombinerRunner(Class<? extends Reducer<K,V,K,V>> cls,
JobConf conf,
Counters.Counter inputCounter,
Task.TaskReporter reporter) |
Modifier and Type | Class and Description |
---|---|
class |
ChainReducer
The ChainReducer class allows to chain multiple Mapper classes after a
Reducer within the Reducer task.
|
class |
FieldSelectionMapReduce<K,V>
This class implements a mapper/reducer class that can be used to perform
field selections in a manner similar to unix cut.
|
class |
IdentityReducer<K,V>
Performs no reduction, writing all input values directly to the output.
|
class |
LongSumReducer<K>
A
Reducer that sums long values. |
Modifier and Type | Method and Description |
---|---|
static <K1,V1,K2,V2> |
ChainReducer.setReducer(JobConf job,
Class<? extends Reducer<K1,V1,K2,V2>> klass,
Class<? extends K1> inputKeyClass,
Class<? extends V1> inputValueClass,
Class<? extends K2> outputKeyClass,
Class<? extends V2> outputValueClass,
boolean byValue,
JobConf reducerConf)
Sets the Reducer class to the chain job's JobConf.
|
Modifier and Type | Class and Description |
---|---|
class |
ValueAggregatorCombiner<K1 extends org.apache.hadoop.io.WritableComparable,V1 extends org.apache.hadoop.io.Writable>
This class implements the generic combiner of Aggregate.
|
class |
ValueAggregatorJobBase<K1 extends org.apache.hadoop.io.WritableComparable,V1 extends org.apache.hadoop.io.Writable>
This abstract class implements some common functionalities of the
the generic mapper, reducer and combiner classes of Aggregate.
|
class |
ValueAggregatorMapper<K1 extends org.apache.hadoop.io.WritableComparable,V1 extends org.apache.hadoop.io.Writable>
This class implements the generic mapper of Aggregate.
|
class |
ValueAggregatorReducer<K1 extends org.apache.hadoop.io.WritableComparable,V1 extends org.apache.hadoop.io.Writable>
This class implements the generic reducer of Aggregate.
|
Constructor and Description |
---|
MergeManagerImpl(TaskAttemptID reduceId,
JobConf jobConf,
org.apache.hadoop.fs.FileSystem localFS,
org.apache.hadoop.fs.LocalDirAllocator localDirAllocator,
Reporter reporter,
org.apache.hadoop.io.compress.CompressionCodec codec,
Class<? extends Reducer> combinerClass,
Task.CombineOutputCollector<K,V> combineCollector,
Counters.Counter spilledRecordsCounter,
Counters.Counter reduceCombineInputCounter,
Counters.Counter mergedMapOutputsCounter,
ExceptionReporter exceptionReporter,
org.apache.hadoop.util.Progress mergePhase,
MapOutputFile mapOutputFile) |
Copyright © 2017 Apache Software Foundation. All Rights Reserved.