org.apache.accumulo.core.client.mapreduce
Class AccumuloOutputFormat

java.lang.Object
  org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.Text,Mutation>
      org.apache.accumulo.core.client.mapreduce.AccumuloOutputFormat

public class AccumuloOutputFormat
extends org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.Text,Mutation>

This class allows MapReduce jobs to use Accumulo as the sink of data. This output format accepts keys and values of type Text (for a table name) and Mutation from the Map() and Reduce() functions. The user must specify the following via static methods:

• AccumuloOutputFormat.setOutputInfo(job, username, password, createTables, defaultTableName)
• AccumuloOutputFormat.setZooKeeperInstance(job, instanceName, hosts)

Other static methods are optional

Constructor Summary

AccumuloOutputFormat()

Method Summary

protected static boolean	canCreateTables(org.apache.hadoop.mapreduce.JobContext job)
void	checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext job)
protected static java.lang.String	getDefaultTableName(org.apache.hadoop.mapreduce.JobContext job)
protected static Instance	getInstance(org.apache.hadoop.mapreduce.JobContext job)
protected static org.apache.log4j.Level	getLogLevel(org.apache.hadoop.mapreduce.JobContext job)
protected static int	getMaxLatency(org.apache.hadoop.mapreduce.JobContext job)
protected static long	getMaxMutationBufferSize(org.apache.hadoop.mapreduce.JobContext job)
protected static int	getMaxWriteThreads(org.apache.hadoop.mapreduce.JobContext job)
org.apache.hadoop.mapreduce.OutputCommitter	getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
protected static byte[]	getPassword(org.apache.hadoop.mapreduce.JobContext job) WARNING: The password is stored in the Configuration and shared with all MapReduce tasks; It is BASE64 encoded to provide a charset safe conversion to a string, and is not intended to be secure.
org.apache.hadoop.mapreduce.RecordWriter<org.apache.hadoop.io.Text,Mutation>	getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext attempt)
protected static boolean	getSimulationMode(org.apache.hadoop.mapreduce.JobContext job)
protected static java.lang.String	getUsername(org.apache.hadoop.mapreduce.JobContext job)
static void	setLogLevel(org.apache.hadoop.mapreduce.JobContext job, org.apache.log4j.Level level)
static void	setMaxLatency(org.apache.hadoop.mapreduce.JobContext job, int numberOfMilliseconds)
static void	setMaxMutationBufferSize(org.apache.hadoop.mapreduce.JobContext job, long numberOfBytes)
static void	setMaxWriteThreads(org.apache.hadoop.mapreduce.JobContext job, int numberOfThreads)
static void	setOutputInfo(org.apache.hadoop.mapreduce.JobContext job, java.lang.String user, byte[] passwd, boolean createTables, java.lang.String defaultTable)
static void	setSimulationMode(org.apache.hadoop.mapreduce.JobContext job)
static void	setZooKeeperInstance(org.apache.hadoop.mapreduce.JobContext job, java.lang.String instanceName, java.lang.String zooKeepers)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AccumuloOutputFormat

public AccumuloOutputFormat()

Method Detail

setOutputInfo

public static void setOutputInfo(org.apache.hadoop.mapreduce.JobContext job,
                                 java.lang.String user,
                                 byte[] passwd,
                                 boolean createTables,
                                 java.lang.String defaultTable)

setZooKeeperInstance

public static void setZooKeeperInstance(org.apache.hadoop.mapreduce.JobContext job,
                                        java.lang.String instanceName,
                                        java.lang.String zooKeepers)

setMaxMutationBufferSize

public static void setMaxMutationBufferSize(org.apache.hadoop.mapreduce.JobContext job,
                                            long numberOfBytes)

setMaxLatency

public static void setMaxLatency(org.apache.hadoop.mapreduce.JobContext job,
                                 int numberOfMilliseconds)

setMaxWriteThreads

public static void setMaxWriteThreads(org.apache.hadoop.mapreduce.JobContext job,
                                      int numberOfThreads)

setLogLevel

public static void setLogLevel(org.apache.hadoop.mapreduce.JobContext job,
                               org.apache.log4j.Level level)

setSimulationMode

public static void setSimulationMode(org.apache.hadoop.mapreduce.JobContext job)

getUsername

protected static java.lang.String getUsername(org.apache.hadoop.mapreduce.JobContext job)

getPassword

protected static byte[] getPassword(org.apache.hadoop.mapreduce.JobContext job)

WARNING: The password is stored in the Configuration and shared with all MapReduce tasks; It is BASE64 encoded to provide a charset safe conversion to a string, and is not intended to be secure.

canCreateTables

protected static boolean canCreateTables(org.apache.hadoop.mapreduce.JobContext job)

getDefaultTableName

protected static java.lang.String getDefaultTableName(org.apache.hadoop.mapreduce.JobContext job)

getInstance

protected static Instance getInstance(org.apache.hadoop.mapreduce.JobContext job)

getMaxMutationBufferSize

protected static long getMaxMutationBufferSize(org.apache.hadoop.mapreduce.JobContext job)

getMaxLatency

protected static int getMaxLatency(org.apache.hadoop.mapreduce.JobContext job)

getMaxWriteThreads

protected static int getMaxWriteThreads(org.apache.hadoop.mapreduce.JobContext job)

getLogLevel

protected static org.apache.log4j.Level getLogLevel(org.apache.hadoop.mapreduce.JobContext job)

getSimulationMode

protected static boolean getSimulationMode(org.apache.hadoop.mapreduce.JobContext job)

checkOutputSpecs

public void checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext job)
                      throws java.io.IOException

Specified by:

checkOutputSpecs in class org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.Text,Mutation>

Throws:

java.io.IOException

getOutputCommitter

public org.apache.hadoop.mapreduce.OutputCommitter getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context)

Specified by:

getOutputCommitter in class org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.Text,Mutation>

getRecordWriter

public org.apache.hadoop.mapreduce.RecordWriter<org.apache.hadoop.io.Text,Mutation> getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext attempt)
                                                                                             throws java.io.IOException

Specified by:

getRecordWriter in class org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.Text,Mutation>

Throws:

java.io.IOException