|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object org.apache.hadoop.mapreduce.InputFormat<K,V> org.apache.accumulo.core.client.mapreduce.AbstractInputFormat<K,V>
public abstract class AbstractInputFormat<K,V>
An abstract input format to provide shared methods common to all other input format classes. At the very least, any classes inheriting from this class will
need to define their own AbstractInputFormat.AbstractRecordReader
.
Nested Class Summary | |
---|---|
protected static class |
AbstractInputFormat.AbstractRecordReader<K,V>
An abstract base class to be used to create RecordReader instances that convert from Accumulo
Key /Value pairs to the user's K/V types. |
Field Summary | |
---|---|
protected static Class<?> |
CLASS
|
protected static org.apache.log4j.Logger |
log
|
Constructor Summary | |
---|---|
AbstractInputFormat()
|
Method Summary | |
---|---|
protected static AuthenticationToken |
getAuthenticationToken(org.apache.hadoop.mapreduce.JobContext context)
Gets the authenticated token from either the specified token file or directly from the configuration, whichever was used when the job was configured. |
protected static InputTableConfig |
getInputTableConfig(org.apache.hadoop.mapreduce.JobContext context,
String tableName)
Fetches a InputTableConfig that has been set on the configuration for a specific table. |
protected static Map<String,InputTableConfig> |
getInputTableConfigs(org.apache.hadoop.mapreduce.JobContext context)
Fetches all InputTableConfig s that have been set on the given job. |
protected static Instance |
getInstance(org.apache.hadoop.mapreduce.JobContext context)
Initializes an Accumulo Instance based on the configuration. |
protected static org.apache.log4j.Level |
getLogLevel(org.apache.hadoop.mapreduce.JobContext context)
Gets the log level from this configuration. |
protected static String |
getPrincipal(org.apache.hadoop.mapreduce.JobContext context)
Gets the user name from the configuration. |
protected static Authorizations |
getScanAuthorizations(org.apache.hadoop.mapreduce.JobContext context)
Gets the authorizations to set for the scans from the configuration. |
List<org.apache.hadoop.mapreduce.InputSplit> |
getSplits(org.apache.hadoop.mapreduce.JobContext context)
Gets the splits of the tables that have been set on the job. |
protected static TabletLocator |
getTabletLocator(org.apache.hadoop.mapreduce.JobContext context,
String table)
Initializes an Accumulo TabletLocator based on the configuration. |
protected static byte[] |
getToken(org.apache.hadoop.mapreduce.JobContext context)
Deprecated. since 1.6.0; Use getAuthenticationToken(JobContext) instead. |
protected static String |
getTokenClass(org.apache.hadoop.mapreduce.JobContext context)
Deprecated. since 1.6.0; Use getAuthenticationToken(JobContext) instead. |
protected static Boolean |
isConnectorInfoSet(org.apache.hadoop.mapreduce.JobContext context)
Determines if the connector has been configured. |
static void |
setConnectorInfo(org.apache.hadoop.mapreduce.Job job,
String principal,
AuthenticationToken token)
Sets the connector information needed to communicate with Accumulo in this job. |
static void |
setConnectorInfo(org.apache.hadoop.mapreduce.Job job,
String principal,
String tokenFile)
Sets the connector information needed to communicate with Accumulo in this job. |
static void |
setLogLevel(org.apache.hadoop.mapreduce.Job job,
org.apache.log4j.Level level)
Sets the log level for this job. |
static void |
setMockInstance(org.apache.hadoop.mapreduce.Job job,
String instanceName)
Configures a MockInstance for this job. |
static void |
setScanAuthorizations(org.apache.hadoop.mapreduce.Job job,
Authorizations auths)
Sets the Authorizations used to scan. |
static void |
setZooKeeperInstance(org.apache.hadoop.mapreduce.Job job,
ClientConfiguration clientConfig)
Configures a ZooKeeperInstance for this job. |
static void |
setZooKeeperInstance(org.apache.hadoop.mapreduce.Job job,
String instanceName,
String zooKeepers)
Deprecated. since 1.6.0; Use setZooKeeperInstance(Job, ClientConfiguration) instead. |
protected static void |
validateOptions(org.apache.hadoop.mapreduce.JobContext context)
Check whether a configuration is fully configured to be used with an Accumulo InputFormat . |
Methods inherited from class org.apache.hadoop.mapreduce.InputFormat |
---|
createRecordReader |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
protected static final Class<?> CLASS
protected static final org.apache.log4j.Logger log
Constructor Detail |
---|
public AbstractInputFormat()
Method Detail |
---|
public static void setConnectorInfo(org.apache.hadoop.mapreduce.Job job, String principal, AuthenticationToken token) throws AccumuloSecurityException
WARNING: The serialized token is stored in the configuration and shared with all MapReduce tasks. It is BASE64 encoded to provide a charset safe conversion to a string, and is not intended to be secure.
job
- the Hadoop job instance to be configuredprincipal
- a valid Accumulo user name (user must have Table.CREATE permission)token
- the user's password
AccumuloSecurityException
public static void setConnectorInfo(org.apache.hadoop.mapreduce.Job job, String principal, String tokenFile) throws AccumuloSecurityException
Stores the password in a file in HDFS and pulls that into the Distributed Cache in an attempt to be more secure than storing it in the Configuration.
job
- the Hadoop job instance to be configuredprincipal
- a valid Accumulo user name (user must have Table.CREATE permission)tokenFile
- the path to the token file
AccumuloSecurityException
protected static Boolean isConnectorInfoSet(org.apache.hadoop.mapreduce.JobContext context)
context
- the Hadoop context for the configured job
setConnectorInfo(Job, String, AuthenticationToken)
protected static String getPrincipal(org.apache.hadoop.mapreduce.JobContext context)
context
- the Hadoop context for the configured job
setConnectorInfo(Job, String, AuthenticationToken)
@Deprecated protected static String getTokenClass(org.apache.hadoop.mapreduce.JobContext context)
getAuthenticationToken(JobContext)
instead.
@Deprecated protected static byte[] getToken(org.apache.hadoop.mapreduce.JobContext context)
getAuthenticationToken(JobContext)
instead.
protected static AuthenticationToken getAuthenticationToken(org.apache.hadoop.mapreduce.JobContext context)
context
- the Hadoop context for the configured job
setConnectorInfo(Job, String, AuthenticationToken)
,
setConnectorInfo(Job, String, String)
@Deprecated public static void setZooKeeperInstance(org.apache.hadoop.mapreduce.Job job, String instanceName, String zooKeepers)
setZooKeeperInstance(Job, ClientConfiguration)
instead.
ZooKeeperInstance
for this job.
job
- the Hadoop job instance to be configuredinstanceName
- the Accumulo instance namezooKeepers
- a comma-separated list of zookeeper serverspublic static void setZooKeeperInstance(org.apache.hadoop.mapreduce.Job job, ClientConfiguration clientConfig)
ZooKeeperInstance
for this job.
job
- the Hadoop job instance to be configuredclientConfig
- client configuration containing connection optionspublic static void setMockInstance(org.apache.hadoop.mapreduce.Job job, String instanceName)
MockInstance
for this job.
job
- the Hadoop job instance to be configuredinstanceName
- the Accumulo instance nameprotected static Instance getInstance(org.apache.hadoop.mapreduce.JobContext context)
Instance
based on the configuration.
context
- the Hadoop context for the configured job
setZooKeeperInstance(Job, String, String)
,
setMockInstance(Job, String)
public static void setLogLevel(org.apache.hadoop.mapreduce.Job job, org.apache.log4j.Level level)
job
- the Hadoop job instance to be configuredlevel
- the logging levelprotected static org.apache.log4j.Level getLogLevel(org.apache.hadoop.mapreduce.JobContext context)
context
- the Hadoop context for the configured job
setLogLevel(Job, Level)
public static void setScanAuthorizations(org.apache.hadoop.mapreduce.Job job, Authorizations auths)
Authorizations
used to scan. Must be a subset of the user's authorization. Defaults to the empty set.
job
- the Hadoop job instance to be configuredauths
- the user's authorizationsprotected static Authorizations getScanAuthorizations(org.apache.hadoop.mapreduce.JobContext context)
context
- the Hadoop context for the configured job
setScanAuthorizations(Job, Authorizations)
protected static Map<String,InputTableConfig> getInputTableConfigs(org.apache.hadoop.mapreduce.JobContext context)
InputTableConfig
s that have been set on the given job.
context
- the Hadoop job instance to be configured
InputTableConfig
objects for the jobprotected static InputTableConfig getInputTableConfig(org.apache.hadoop.mapreduce.JobContext context, String tableName)
InputTableConfig
that has been set on the configuration for a specific table.
null is returned in the event that the table doesn't exist.
context
- the Hadoop job instance to be configuredtableName
- the table name for which to grab the config object
InputTableConfig
for the given tableprotected static TabletLocator getTabletLocator(org.apache.hadoop.mapreduce.JobContext context, String table) throws TableNotFoundException
TabletLocator
based on the configuration.
context
- the Hadoop context for the configured jobtable
- the table for which to initialize the locator
TableNotFoundException
- if the table name set on the configuration doesn't existprotected static void validateOptions(org.apache.hadoop.mapreduce.JobContext context) throws IOException
InputFormat
.
context
- the Hadoop context for the configured job
IOException
- if the context is improperly configuredpublic List<org.apache.hadoop.mapreduce.InputSplit> getSplits(org.apache.hadoop.mapreduce.JobContext context) throws IOException
getSplits
in class org.apache.hadoop.mapreduce.InputFormat<K,V>
context
- the configuration of the job
IOException
- if a table set on the job doesn't exist or an error occurs initializing the tablet locator
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |