org.apache.phoenix.mapreduce

Class AbstractBulkLoadTool

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.phoenix.mapreduce.AbstractBulkLoadTool

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

Direct Known Subclasses:

CsvBulkLoadTool, JsonBulkLoadTool, RegexBulkLoadTool

public abstract class AbstractBulkLoadTool
extends org.apache.hadoop.conf.Configured
implements org.apache.hadoop.util.Tool

Base tool for running MapReduce-based ingests of data.

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.slf4j.Logger	LOG

Constructor Summary

Constructors

Constructor and Description
AbstractBulkLoadTool()

Method Summary

Methods

Modifier and Type	Method and Description
protected abstract void	configureOptions(org.apache.commons.cli.CommandLine cmdLine, List<ColumnInfo> importColumns, org.apache.hadoop.conf.Configuration conf) Set configuration values based on parsed command line options.
protected org.apache.commons.cli.Options	getOptions()
protected org.apache.commons.cli.CommandLine	parseOptions(String[] args) Parses the commandline arguments, throws IllegalStateException if mandatory arguments are missing.
int	run(String[] args)
protected abstract void	setupJob(org.apache.hadoop.mapreduce.Job job)
int	submitJob(org.apache.hadoop.conf.Configuration conf, String qualifiedTableName, String inputPaths, org.apache.hadoop.fs.Path outputPath, List<TargetTableRef> tablesToBeLoaded, boolean hasLocalIndexes) Submits the jobs to the cluster.

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf, setConf

Field Detail

LOG

protected static final org.slf4j.Logger LOG

Constructor Detail

AbstractBulkLoadTool

public AbstractBulkLoadTool()

Method Detail

configureOptions

protected abstract void configureOptions(org.apache.commons.cli.CommandLine cmdLine,
                    List<ColumnInfo> importColumns,
                    org.apache.hadoop.conf.Configuration conf)
                                  throws SQLException

Set configuration values based on parsed command line options.

Parameters:

cmdLine - supplied command line options

importColumns - descriptors of columns to be imported

conf - job configuration

Throws:

SQLException

setupJob

protected abstract void setupJob(org.apache.hadoop.mapreduce.Job job)

getOptions

protected org.apache.commons.cli.Options getOptions()

parseOptions

protected org.apache.commons.cli.CommandLine parseOptions(String[] args)

Parses the commandline arguments, throws IllegalStateException if mandatory arguments are missing.

Parameters:

args - supplied command line arguments

Returns:

the parsed command line

run

public int run(String[] args)
        throws Exception

Specified by:

run in interface org.apache.hadoop.util.Tool

Throws:

Exception

submitJob

public int submitJob(org.apache.hadoop.conf.Configuration conf,
            String qualifiedTableName,
            String inputPaths,
            org.apache.hadoop.fs.Path outputPath,
            List<TargetTableRef> tablesToBeLoaded,
            boolean hasLocalIndexes)
              throws Exception

Submits the jobs to the cluster. Loads the HFiles onto the respective tables.

Throws:

Exception