AbstractFSWALProvider (Apache HBase - Server 2.0.0-alpha3 API)

java.lang.Object
- org.apache.hadoop.hbase.wal.AbstractFSWALProvider<T>

All Implemented Interfaces:

WALProvider

Direct Known Subclasses:

AsyncFSWALProvider, FSHLogProvider
```
@InterfaceAudience.Private
 @InterfaceStability.Evolving
public abstract class AbstractFSWALProvider<T extends AbstractFSWAL<?>>
extends Object
implements WALProvider
```
Base class of a WAL Provider that returns a single thread safe WAL that writes to Hadoop FS. By default, this implementation picks a directory in Hadoop FS based on a combination of
- the HBase root directory
- HConstants.HREGION_LOGDIR_NAME
- the given factory's factoryId (usually identifying the regionserver by host:port)
It also uses the providerId to differentiate among files.

Nested Class Summary

Nested Classes
Modifier and Type Class and Description

static interface AbstractFSWALProvider.Reader
- Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.wal.WALProvider
  WALProvider.AsyncWriter, WALProvider.Writer, WALProvider.WriterBase

Nested Classes
Modifier and Type	Class and Description
`static interface`	`AbstractFSWALProvider.Reader`

Field Summary

Fields
Modifier and Type	Field and Description
`protected org.apache.hadoop.conf.Configuration`	`conf`
`protected WALFactory`	`factory`
`protected AtomicBoolean`	`initialized`
`protected List<WALActionsListener>`	`listeners`
`protected String`	`logPrefix`
`static String`	`META_WAL_PROVIDER_ID` The hbase:meta region's WAL filename extension
`protected String`	`providerId`
`static String`	`SPLITTING_EXT` File Extension used while splitting an WAL into regions (HBASE-2312)
`protected T`	`wal`
`static String`	`WAL_FILE_NAME_DELIMITER`

Constructor Summary

Constructors
Constructor and Description

AbstractFSWALProvider()

Constructors
Constructor and Description
`AbstractFSWALProvider()`

Method Summary

All Methods Static Methods Instance Methods Abstract Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`close()` shutdown utstanding WALs and clean up any persisted state.
`protected abstract T`	`createWAL()`
`protected abstract void`	`doInit(org.apache.hadoop.conf.Configuration conf)`
`static long`	`extractFileNumFromWAL(WAL wal)` It returns the file create timestamp from the file name.
`static org.apache.hadoop.fs.Path`	`getArchivedLogPath(org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration conf)` Get the archived WAL file path
`static org.apache.hadoop.fs.Path`	`getCurrentFileName(WAL wal)` return the current filename from the current wal.
`long`	`getLogFileSize()` iff the given WALFactory is using the DefaultWALProvider for meta and/or non-meta, count the size of files (only rolled).
`static long`	`getLogFileSize(WAL wal)` returns the size of rolled WAL files.
`long`	`getNumLogFiles()` iff the given WALFactory is using the DefaultWALProvider for meta and/or non-meta, count the number of files (rolled and active).
`static int`	`getNumRolledLogFiles(WAL wal)` returns the number of rolled WAL files.
`static ServerName`	`getServerNameFromWALDirectoryName(org.apache.hadoop.conf.Configuration conf, String path)` Pulls a ServerName out of a Path generated according to our layout rules.
`static ServerName`	`getServerNameFromWALDirectoryName(org.apache.hadoop.fs.Path logFile)` This function returns region server name from a log file name which is in one of the following formats: hdfs://<name node>/hbase/.logs/<server name>-splitting/... hdfs://<name node>/hbase/.logs/<server name>/...
`T`	`getWAL(byte[] identifier, byte[] namespace)`
`static String`	`getWALDirectoryName(String serverName)` Construct the directory name for all WALs on a given server.
`static String`	`getWALPrefixFromWALName(String name)` Get prefix of the log from its name, assuming WAL name in format of log_prefix.filenumber.log_suffix
`List<WAL>`	`getWALs()`
`void`	`init(WALFactory factory, org.apache.hadoop.conf.Configuration conf, List<WALActionsListener> listeners, String providerId)` Set up the provider to create wals.
`static boolean`	`isArchivedLogFile(org.apache.hadoop.fs.Path p)`
`static boolean`	`isMetaFile(org.apache.hadoop.fs.Path p)`
`static boolean`	`isMetaFile(String p)`
`static WAL.Reader`	`openReader(org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration conf)` Opens WAL reader with retries and additional exception handling
`void`	`shutdown()` persist outstanding WALs to storage and stop accepting new appends.
`static boolean`	`validateWALFilename(String filename)` A WAL file name is of the format: <wal-name>`WAL_FILE_NAME_DELIMITER` <file-creation-timestamp>[.meta].

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - wal
```
protected volatile T extends AbstractFSWAL<?> wal
```
  - factory
```
protected WALFactory factory
```
  - conf
```
protected org.apache.hadoop.conf.Configuration conf
```
  - listeners
```
protected List<WALActionsListener> listeners
```
  - providerId
```
protected String providerId
```
  - initialized
```
protected AtomicBoolean initialized
```
  - logPrefix
```
protected String logPrefix
```
  - WAL_FILE_NAME_DELIMITER
```
public static final String WAL_FILE_NAME_DELIMITER
```
    See Also:
    
    Constant Field Values
  - META_WAL_PROVIDER_ID
```
public static final String META_WAL_PROVIDER_ID
```
    The hbase:meta region's WAL filename extension
    
    See Also:
    
    Constant Field Values
  - SPLITTING_EXT
```
public static final String SPLITTING_EXT
```
    File Extension used while splitting an WAL into regions (HBASE-2312)
    
    See Also:
    
    Constant Field Values
- Constructor Detail
  - AbstractFSWALProvider
```
public AbstractFSWALProvider()
```
- Method Detail
  - init
```
public void init(WALFactory factory,
                 org.apache.hadoop.conf.Configuration conf,
                 List<WALActionsListener> listeners,
                 String providerId)
          throws IOException
```
    Description copied from interface: WALProvider
    
    Set up the provider to create wals. will only be called once per instance.
    
    Specified by:
    
    init in interface WALProvider
    
    Parameters:
    
    factory - factory that made us, identity used for FS layout. may not be null
    
    conf - may not be null
    
    listeners - may be null
    
    providerId - differentiate between providers from one factory, used for FS layout. may be null
    
    Throws:
    
    IOException
  - getWALs
```
public List<WAL> getWALs()
```
    Specified by:
    
    getWALs in interface WALProvider
    
    Returns:
    
    the List of WALs that are used by this server
  - getWAL
```
public T getWAL(byte[] identifier,
                byte[] namespace)
         throws IOException
```
    Specified by:
    
    getWAL in interface WALProvider
    
    Parameters:
    
    identifier - may not be null. contents will not be altered.
    
    namespace - could be null, and will use default namespace if null
    
    Returns:
    
    a WAL for writing entries for the given region.
    
    Throws:
    
    IOException
  - createWAL
```
protected abstract T createWAL()
                        throws IOException
```
    Throws:
    
    IOException
  - doInit
```
protected abstract void doInit(org.apache.hadoop.conf.Configuration conf)
                        throws IOException
```
    Throws:
    
    IOException
  - shutdown
```
public void shutdown()
              throws IOException
```
    Description copied from interface: WALProvider
    
    persist outstanding WALs to storage and stop accepting new appends. This method serves as shorthand for sending a sync to every WAL provided by a given implementation. Those WALs will also stop accepting new writes.
    
    Specified by:
    
    shutdown in interface WALProvider
    
    Throws:
    
    IOException
  - close
```
public void close()
           throws IOException
```
    Description copied from interface: WALProvider
    
    shutdown utstanding WALs and clean up any persisted state. Call this method only when you will not need to replay any of the edits to the WALs from this provider. After this call completes, the underlying resources should have been reclaimed.
    
    Specified by:
    
    close in interface WALProvider
    
    Throws:
    
    IOException
  - getNumLogFiles
```
public long getNumLogFiles()
```
    iff the given WALFactory is using the DefaultWALProvider for meta and/or non-meta, count the number of files (rolled and active). if either of them aren't, count 0 for that provider.
    
    Specified by:
    
    getNumLogFiles in interface WALProvider
  - getLogFileSize
```
public long getLogFileSize()
```
    iff the given WALFactory is using the DefaultWALProvider for meta and/or non-meta, count the size of files (only rolled). if either of them aren't, count 0 for that provider.
    
    Specified by:
    
    getLogFileSize in interface WALProvider
  - getNumRolledLogFiles
```
public static int getNumRolledLogFiles(WAL wal)
```
    returns the number of rolled WAL files.
  - getLogFileSize
```
public static long getLogFileSize(WAL wal)
```
    returns the size of rolled WAL files.
  - getCurrentFileName
```
public static org.apache.hadoop.fs.Path getCurrentFileName(WAL wal)
```
    return the current filename from the current wal.
  - extractFileNumFromWAL
```
public static long extractFileNumFromWAL(WAL wal)
```
    It returns the file create timestamp from the file name. For name format see validateWALFilename(String) public until remaining tests move to o.a.h.h.wal
    
    Parameters:
    
    wal - must not be null
    
    Returns:
    
    the file number that is part of the WAL file name
  - validateWALFilename
```
public static boolean validateWALFilename(String filename)
```
    A WAL file name is of the format: <wal-name>WAL_FILE_NAME_DELIMITER <file-creation-timestamp>[.meta]. provider-name is usually made up of a server-name and a provider-id
    
    Parameters:
    
    filename - name of the file to validate
    
    Returns:
    
    true if the filename matches an WAL, false otherwise
  - getWALDirectoryName
```
public static String getWALDirectoryName(String serverName)
```
    Construct the directory name for all WALs on a given server. Dir names currently look like this for WALs: hbase//WALs/kalashnikov.att.net,61634,1486865297088.
    
    Parameters:
    
    serverName - Server name formatted as described in ServerName
    
    Returns:
    
    the relative WAL directory name, e.g. .logs/1.example.org,60030,12345 if serverName passed is 1.example.org,60030,12345
  - getServerNameFromWALDirectoryName
```
public static ServerName getServerNameFromWALDirectoryName(org.apache.hadoop.conf.Configuration conf,
                                                           String path)
                                                    throws IOException
```
    Pulls a ServerName out of a Path generated according to our layout rules. In the below layouts, this method ignores the format of the logfile component. Current format: [base directory for hbase]/hbase/.logs/ServerName/logfile or [base directory for hbase]/hbase/.logs/ServerName-splitting/logfile Expected to work for individual log files and server-specific directories.
    
    Returns:
    
    null if it's not a log file. Returns the ServerName of the region server that created this log file otherwise.
    
    Throws:
    
    IOException
  - getServerNameFromWALDirectoryName
```
public static ServerName getServerNameFromWALDirectoryName(org.apache.hadoop.fs.Path logFile)
```
    This function returns region server name from a log file name which is in one of the following formats:
    - hdfs://<name node>/hbase/.logs/<server name>-splitting/...
    - hdfs://<name node>/hbase/.logs/<server name>/...
    Returns:
    
    null if the passed in logFile isn't a valid WAL file path
  - isMetaFile
```
public static boolean isMetaFile(org.apache.hadoop.fs.Path p)
```
  - isMetaFile
```
public static boolean isMetaFile(String p)
```
  - isArchivedLogFile
```
public static boolean isArchivedLogFile(org.apache.hadoop.fs.Path p)
```
  - getArchivedLogPath
```
public static org.apache.hadoop.fs.Path getArchivedLogPath(org.apache.hadoop.fs.Path path,
                                                           org.apache.hadoop.conf.Configuration conf)
                                                    throws IOException
```
    Get the archived WAL file path
    
    Parameters:
    
    path - - active WAL file path
    
    conf - - configuration
    
    Returns:
    
    archived path if exists, path - otherwise
    
    Throws:
    
    IOException - exception
  - openReader
```
public static WAL.Reader openReader(org.apache.hadoop.fs.Path path,
                                    org.apache.hadoop.conf.Configuration conf)
                             throws IOException
```
    Opens WAL reader with retries and additional exception handling
    
    Parameters:
    
    path - path to WAL file
    
    conf - configuration
    
    Returns:
    
    WAL Reader instance
    
    Throws:
    
    IOException
  - getWALPrefixFromWALName
```
public static String getWALPrefixFromWALName(String name)
```
    Get prefix of the log from its name, assuming WAL name in format of log_prefix.filenumber.log_suffix
    
    Parameters:
    
    name - Name of the WAL to parse
    
    Returns:
    
    prefix of the log
    
    See Also:
    
    AbstractFSWAL.getCurrentFileName()

Class AbstractFSWALProvider<T extends AbstractFSWAL<?>>

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.wal.WALProvider

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

wal

factory

conf

listeners

providerId

initialized

logPrefix

WAL_FILE_NAME_DELIMITER

META_WAL_PROVIDER_ID

SPLITTING_EXT

Constructor Detail

AbstractFSWALProvider

Method Detail

init

getWALs

getWAL

createWAL

doInit

shutdown

close

getNumLogFiles

getLogFileSize

getNumRolledLogFiles

getLogFileSize

getCurrentFileName

extractFileNumFromWAL

validateWALFilename

getWALDirectoryName

getServerNameFromWALDirectoryName

getServerNameFromWALDirectoryName

isMetaFile

isMetaFile

isArchivedLogFile

getArchivedLogPath

openReader

getWALPrefixFromWALName