org.apache.hadoop.hdfs

Class HAUtil

java.lang.Object

org.apache.hadoop.hdfs.HAUtil

public class HAUtil
extends Object

Method Summary

Methods

Modifier and Type	Method and Description
static org.apache.hadoop.io.Text	buildTokenServiceForLogicalUri(URI uri, String scheme) Get the service name used in the delegation token for the given logical HA service.
static String	buildTokenServicePrefixForLogicalUri(String scheme)
static void	cloneDelegationTokenForLogicalUri(org.apache.hadoop.security.UserGroupInformation ugi, URI haUri, Collection<InetSocketAddress> nnAddrs) Locate a delegation token associated with the given HA cluster URI, and if one is found, clone it to also represent the underlying namenode address.
static InetSocketAddress	getAddressOfActive(org.apache.hadoop.fs.FileSystem fs) Get the internet address of the currently-active NN.
static org.apache.hadoop.conf.Configuration	getConfForOtherNode(org.apache.hadoop.conf.Configuration myConf) Given the configuration for this node, return a Configuration object for the other node in an HA setup.
static String	getNameNodeId(org.apache.hadoop.conf.Configuration conf, String nsId) Get the namenode Id by matching the addressKey with the the address of the local node.
static String	getNameNodeIdFromAddress(org.apache.hadoop.conf.Configuration conf, InetSocketAddress address, String... keys) Similar to DFSUtil.getNameServiceIdFromAddress(Configuration, InetSocketAddress, String...)
static String	getNameNodeIdOfOtherNode(org.apache.hadoop.conf.Configuration conf, String nsId) Get the NN ID of the other node in an HA setup.
static List<org.apache.hadoop.hdfs.protocol.ClientProtocol>	getProxiesForAllNameNodesInNameservice(org.apache.hadoop.conf.Configuration conf, String nsId) Get an RPC proxy for each NN in an HA nameservice.
static <T> List<NameNodeProxies.ProxyAndInfo<T>>	getProxiesForAllNameNodesInNameservice(org.apache.hadoop.conf.Configuration conf, String nsId, Class<T> xface) Get an RPC proxy for each NN in an HA nameservice.
static URI	getServiceUriFromToken(String scheme, org.apache.hadoop.security.token.Token<?> token) Parse the file system URI out of the provided token.
static boolean	isAtLeastOneActive(List<org.apache.hadoop.hdfs.protocol.ClientProtocol> namenodes) Used to ensure that at least one of the given HA NNs is currently in the active state..
static boolean	isClientFailoverConfigured(org.apache.hadoop.conf.Configuration conf, URI nameNodeUri) Check whether the client has a failover proxy provider configured for the namenode/nameservice.
static boolean	isHAEnabled(org.apache.hadoop.conf.Configuration conf, String nsId) Returns true if HA for namenode is configured for the given nameservice
static boolean	isLogicalUri(org.apache.hadoop.conf.Configuration conf, URI nameNodeUri)
static boolean	isTokenForLogicalUri(org.apache.hadoop.security.token.Token<?> token)
static void	setAllowStandbyReads(org.apache.hadoop.conf.Configuration conf, boolean val)
static boolean	shouldAllowStandbyReads(org.apache.hadoop.conf.Configuration conf) This is used only by tests at the moment.
static boolean	useLogicalUri(org.apache.hadoop.conf.Configuration conf, URI nameNodeUri) Check whether logical URI is needed for the namenode and the corresponding failover proxy provider in the config.
static boolean	usesSharedEditsDir(org.apache.hadoop.conf.Configuration conf) Returns true if HA is using a shared edits directory.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

isHAEnabled

public static boolean isHAEnabled(org.apache.hadoop.conf.Configuration conf,
                  String nsId)

Returns true if HA for namenode is configured for the given nameservice

Parameters:

conf - Configuration

nsId - nameservice, or null if no federated NS is configured

Returns:

true if HA is configured in the configuration; else false.

usesSharedEditsDir

public static boolean usesSharedEditsDir(org.apache.hadoop.conf.Configuration conf)

Returns true if HA is using a shared edits directory.

Parameters:

conf - Configuration

Returns:

true if HA config is using a shared edits dir, false otherwise.

getNameNodeId

public static String getNameNodeId(org.apache.hadoop.conf.Configuration conf,
                   String nsId)

Get the namenode Id by matching the addressKey with the the address of the local node. If DFSConfigKeys.DFS_HA_NAMENODE_ID_KEY is not specifically configured, this method determines the namenode Id by matching the local node's address with the configured addresses. When a match is found, it returns the namenode Id from the corresponding configuration key.

Parameters:

conf - Configuration

Returns:

namenode Id on success, null on failure.

Throws:

org.apache.hadoop.HadoopIllegalArgumentException - on error

getNameNodeIdFromAddress

public static String getNameNodeIdFromAddress(org.apache.hadoop.conf.Configuration conf,
                              InetSocketAddress address,
                              String... keys)

Similar to DFSUtil.getNameServiceIdFromAddress(Configuration, InetSocketAddress, String...)

getNameNodeIdOfOtherNode

public static String getNameNodeIdOfOtherNode(org.apache.hadoop.conf.Configuration conf,
                              String nsId)

Get the NN ID of the other node in an HA setup.

Parameters:

conf - the configuration of this node

Returns:

the NN ID of the other node in this nameservice

getConfForOtherNode

public static org.apache.hadoop.conf.Configuration getConfForOtherNode(org.apache.hadoop.conf.Configuration myConf)

Given the configuration for this node, return a Configuration object for the other node in an HA setup.

Parameters:

myConf - the configuration of this node

Returns:

the configuration of the other node in an HA setup

shouldAllowStandbyReads

public static boolean shouldAllowStandbyReads(org.apache.hadoop.conf.Configuration conf)

This is used only by tests at the moment.

Returns:

true if the NN should allow read operations while in standby mode.

setAllowStandbyReads

public static void setAllowStandbyReads(org.apache.hadoop.conf.Configuration conf,
                        boolean val)

isLogicalUri

public static boolean isLogicalUri(org.apache.hadoop.conf.Configuration conf,
                   URI nameNodeUri)

Returns:

true if the given nameNodeUri appears to be a logical URI.

isClientFailoverConfigured

public static boolean isClientFailoverConfigured(org.apache.hadoop.conf.Configuration conf,
                                 URI nameNodeUri)

Check whether the client has a failover proxy provider configured for the namenode/nameservice.

Parameters:

conf - Configuration

nameNodeUri - The URI of namenode

Returns:

true if failover is configured.

useLogicalUri

public static boolean useLogicalUri(org.apache.hadoop.conf.Configuration conf,
                    URI nameNodeUri)
                             throws IOException

Check whether logical URI is needed for the namenode and the corresponding failover proxy provider in the config.

Parameters:

conf - Configuration

nameNodeUri - The URI of namenode

Returns:

true if logical URI is needed. false, if not needed.

Throws:

IOException - most likely due to misconfiguration.

getServiceUriFromToken

public static URI getServiceUriFromToken(String scheme,
                         org.apache.hadoop.security.token.Token<?> token)

Parse the file system URI out of the provided token.

buildTokenServiceForLogicalUri

public static org.apache.hadoop.io.Text buildTokenServiceForLogicalUri(URI uri,
                                                       String scheme)

Get the service name used in the delegation token for the given logical HA service.

Parameters:

uri - the logical URI of the cluster

scheme - the scheme of the corresponding FileSystem

Returns:

the service name

isTokenForLogicalUri

public static boolean isTokenForLogicalUri(org.apache.hadoop.security.token.Token<?> token)

Returns:

true if this token corresponds to a logical nameservice rather than a specific namenode.

buildTokenServicePrefixForLogicalUri

public static String buildTokenServicePrefixForLogicalUri(String scheme)

cloneDelegationTokenForLogicalUri

public static void cloneDelegationTokenForLogicalUri(org.apache.hadoop.security.UserGroupInformation ugi,
                                     URI haUri,
                                     Collection<InetSocketAddress> nnAddrs)

Locate a delegation token associated with the given HA cluster URI, and if one is found, clone it to also represent the underlying namenode address.

Parameters:

ugi - the UGI to modify

haUri - the logical URI for the cluster

nnAddrs - collection of NNs in the cluster to which the token applies

getAddressOfActive

public static InetSocketAddress getAddressOfActive(org.apache.hadoop.fs.FileSystem fs)
                                            throws IOException

Get the internet address of the currently-active NN. This should rarely be used, since callers of this method who connect directly to the NN using the resulting InetSocketAddress will not be able to connect to the active NN if a failover were to occur after this method has been called.

Parameters:

fs - the file system to get the active address of.

Returns:

the internet address of the currently-active NN.

Throws:

IOException - if an error occurs while resolving the active NN.

getProxiesForAllNameNodesInNameservice

public static List<org.apache.hadoop.hdfs.protocol.ClientProtocol> getProxiesForAllNameNodesInNameservice(org.apache.hadoop.conf.Configuration conf,
                                                                                          String nsId)
                                                                                                   throws IOException

Get an RPC proxy for each NN in an HA nameservice. Used when a given RPC call should be made on every NN in an HA nameservice, not just the active.

Parameters:

conf - configuration

nsId - the nameservice to get all of the proxies for.

Returns:

a list of RPC proxies for each NN in the nameservice.

Throws:

IOException - in the event of error.

getProxiesForAllNameNodesInNameservice

public static <T> List<NameNodeProxies.ProxyAndInfo<T>> getProxiesForAllNameNodesInNameservice(org.apache.hadoop.conf.Configuration conf,
                                                                               String nsId,
                                                                               Class<T> xface)
                                                                                    throws IOException

Get an RPC proxy for each NN in an HA nameservice. Used when a given RPC call should be made on every NN in an HA nameservice, not just the active.

Parameters:

conf - configuration

nsId - the nameservice to get all of the proxies for.

xface - the protocol class.

Returns:

a list of RPC proxies for each NN in the nameservice.

Throws:

IOException - in the event of error.

isAtLeastOneActive

public static boolean isAtLeastOneActive(List<org.apache.hadoop.hdfs.protocol.ClientProtocol> namenodes)
                                  throws IOException

Used to ensure that at least one of the given HA NNs is currently in the active state..

Parameters:

namenodes - list of RPC proxies for each NN to check.

Returns:

true if at least one NN is active, false if all are in the standby state.

Throws:

IOException - in the event of error.