com.yahoo.vespa.hadoop.mapreduce

Class VespaOutputFormat

java.lang.Object

org.apache.hadoop.mapreduce.OutputFormat

com.yahoo.vespa.hadoop.mapreduce.VespaOutputFormat

public class VespaOutputFormat
extends org.apache.hadoop.mapreduce.OutputFormat

An output specification for writing to Vespa instances in a Map-Reduce job. Mainly returns an instance of a VespaRecordWriter that does the actual feeding to Vespa.

Author:

lesters

Field Summary

Fields

Modifier and Type	Field and Description
(package private) Properties	configOverride

Constructor Summary

Constructors

Constructor and Description
VespaOutputFormat()
VespaOutputFormat(Properties configOverride)

Method Summary

Modifier and Type	Method and Description
void	checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext jobContext)
org.apache.hadoop.mapreduce.OutputCommitter	getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext taskAttemptContext)
org.apache.hadoop.mapreduce.RecordWriter	getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

configOverride

final Properties configOverride

Constructor Detail

VespaOutputFormat

public VespaOutputFormat()

VespaOutputFormat

public VespaOutputFormat(Properties configOverride)

Method Detail

getRecordWriter

public org.apache.hadoop.mapreduce.RecordWriter getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                         throws IOException,
                                                                InterruptedException

Specified by:

getRecordWriter in class org.apache.hadoop.mapreduce.OutputFormat

Throws:

IOException

InterruptedException

getOutputCommitter

public org.apache.hadoop.mapreduce.OutputCommitter getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext taskAttemptContext)
                                                               throws IOException,
                                                                      InterruptedException

Specified by:

getOutputCommitter in class org.apache.hadoop.mapreduce.OutputFormat

Throws:

IOException

InterruptedException

checkOutputSpecs

public void checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext jobContext)
                      throws IOException,
                             InterruptedException

Specified by:

checkOutputSpecs in class org.apache.hadoop.mapreduce.OutputFormat

Throws:

IOException

InterruptedException