org.apache.cassandra.db

Class Directories

java.lang.Object

org.apache.cassandra.db.Directories

public class Directories
extends java.lang.Object

Encapsulate handling of paths to the data files.

 
   /<path_to_data_dir>/ks/<cf dir>/ks-cf1-jb-1-Data.db
                         /<cf dir>/la-2-Data.db
                         /<cf dir>/.<index name>/ks-cf1.idx-jb-1-Data.db
                         /<cf dir>/.<index name>/la-1-Data.db
                         ...
  

Until v2.0, <cf dir> is just column family name. Since v2.1, <cf dir> has column family ID(tableId) added to its end. SSTables from secondary indexes were put in the same directory as their parent. Since v2.2, they have their own directory under the parent directory whose name is index name. Upon startup, those secondary index files are moved to new directory when upgrading. For backward compatibility, Directories can use directory without tableId if exists. In addition, more that one 'root' data directory can be specified so that <path_to_data_dir> potentially represents multiple locations. Note that in the case of multiple locations, the manifest for the leveled compaction is only in one of the location. Snapshots (resp. backups) are always created along the sstables there are snapshotted (resp. backuped) but inside a subdirectory named 'snapshots' (resp. backups) (and snapshots are further inside a subdirectory of the name of the snapshot). For secondary indexes, snapshots (backups) are not created in their own directory, but are in their parent's snapshot (backup) directory. This class abstracts all those details from the rest of the code.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Directories.DataDirectories Data directories used to store keyspace data.
static class	Directories.DataDirectory
static class	Directories.FileAction
static class	Directories.FileType The type of files that can be listed by SSTableLister, we never return txn logs, use LifecycleTransaction.getFiles() if you need txn logs.
static class	Directories.OnTxnErr How to handle a failure to read a txn log file.
static class	Directories.SnapshotSizeDetails
static class	Directories.SSTableLister

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	BACKUPS_SUBDIR
static Directories.DataDirectories	dataDirectories The directories used to store keyspaces data.
static java.lang.String	SECONDARY_INDEX_NAME_SEPARATOR
static java.lang.String	SNAPSHOT_SUBDIR
static java.lang.String	TMP_SUBDIR

Constructor Summary

Constructors

Constructor and Description
Directories(TableMetadata metadata)
Directories(TableMetadata metadata, java.util.Collection<Directories.DataDirectory> paths)
Directories(TableMetadata metadata, Directories.DataDirectory[] paths) Create Directories of given ColumnFamily.

Method Summary

Modifier and Type	Method and Description
static void	clearSnapshot(java.lang.String snapshotName, java.util.List<java.io.File> snapshotDirectories, com.google.common.util.concurrent.RateLimiter snapshotRateLimiter)
Descriptor	find(java.lang.String filename)
static java.io.File	getBackupsDirectory(Descriptor desc)
static java.io.File	getBackupsDirectory(java.io.File location)
java.util.List<java.io.File>	getCFDirectories()
Directories.DataDirectory	getDataDirectoryForFile(Descriptor descriptor)
java.io.File	getDirectoryForNewSSTables() Basically the same as calling getWriteableLocationAsFile(long) with an unknown size (-1L), which may return any allowed directory - even a data directory that has no usable space.
static java.util.List<java.io.File>	getKSChildDirectories(java.lang.String ksName)
java.io.File	getLocationForDisk(Directories.DataDirectory dataDirectory) Returns SSTable location which is inside given data directory.
java.io.File	getNewEphemeralSnapshotMarkerFile(java.lang.String snapshotName)
long	getRawDiretoriesSize()
java.util.Map<java.lang.String,Directories.SnapshotSizeDetails>	getSnapshotDetails()
static java.io.File	getSnapshotDirectory(Descriptor desc, java.lang.String snapshotName)
static java.io.File	getSnapshotDirectory(java.io.File location, java.lang.String snapshotName) Returns directory to write snapshot.
java.io.File	getSnapshotManifestFile(java.lang.String snapshotName)
java.io.File	getSnapshotSchemaFile(java.lang.String snapshotName)
java.io.File	getTemporaryWriteableDirectoryAsFile(long writeSize) Returns a temporary subdirectory on allowed data directory that _currently_ has writeSize bytes as usable space.
long	getTrueAllocatedSizeIn(java.io.File input)
Directories.DataDirectory	getWriteableLocation(long writeSize) Returns an allowed data directory that _currently_ has writeSize bytes as usable space.
java.io.File	getWriteableLocationAsFile(long writeSize) Returns an allowed directory that _currently_ has writeSize bytes as usable space.
Directories.DataDirectory[]	getWriteableLocations()
java.io.File	getWriteableLocationToLoadFile(java.io.File sourceFile) Returns a data directory to load the file sourceFile.
boolean	hasAvailableDiskSpace(long estimatedSSTables, long expectedTotalWriteSize)
static boolean	isSecondaryIndexFolder(java.io.File dir)
static boolean	isStoredInLocalSystemKeyspacesDataLocation(java.lang.String keyspace, java.lang.String table) Checks if the specified table should be stored with local system data.
java.util.List<java.lang.String>	listEphemeralSnapshots()
void	removeTemporaryDirectories()
long	snapshotCreationTime(java.lang.String snapshotName)
boolean	snapshotExists(java.lang.String snapshotName)
Directories.SSTableLister	sstableLister(Directories.OnTxnErr onTxnErr)
Directories.SSTableLister	sstableLister(java.io.File directory, Directories.OnTxnErr onTxnErr)
long	trueSnapshotsSize()
static boolean	verifyFullPermissions(java.io.File dir, java.lang.String dataDir) Checks whether Cassandra has RWX permissions to the specified directory.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

BACKUPS_SUBDIR

public static final java.lang.String BACKUPS_SUBDIR

See Also:

Constant Field Values

SNAPSHOT_SUBDIR

public static final java.lang.String SNAPSHOT_SUBDIR

See Also:

Constant Field Values

TMP_SUBDIR

public static final java.lang.String TMP_SUBDIR

See Also:

Constant Field Values

SECONDARY_INDEX_NAME_SEPARATOR

public static final java.lang.String SECONDARY_INDEX_NAME_SEPARATOR

See Also:

Constant Field Values

dataDirectories

public static final Directories.DataDirectories dataDirectories

The directories used to store keyspaces data.

Constructor Detail

Directories

public Directories(TableMetadata metadata)

Directories

public Directories(TableMetadata metadata,
                   java.util.Collection<Directories.DataDirectory> paths)

Directories

public Directories(TableMetadata metadata,
                   Directories.DataDirectory[] paths)

Create Directories of given ColumnFamily. SSTable directories are created under data_directories defined in cassandra.yaml if not exist at this time.

Parameters:

metadata - metadata of ColumnFamily

Method Detail

verifyFullPermissions

public static boolean verifyFullPermissions(java.io.File dir,
                                            java.lang.String dataDir)

Checks whether Cassandra has RWX permissions to the specified directory. Logs an error with the details if it does not.

Parameters:

dir - File object of the directory.

dataDir - String representation of the directory's location

Returns:

status representing Cassandra's RWX permissions to the supplied folder location.

getLocationForDisk

public java.io.File getLocationForDisk(Directories.DataDirectory dataDirectory)

Returns SSTable location which is inside given data directory.

Parameters:

dataDirectory -

Returns:

SSTable location

getDataDirectoryForFile

public Directories.DataDirectory getDataDirectoryForFile(Descriptor descriptor)

find

public Descriptor find(java.lang.String filename)

getDirectoryForNewSSTables

public java.io.File getDirectoryForNewSSTables()

Basically the same as calling getWriteableLocationAsFile(long) with an unknown size (-1L), which may return any allowed directory - even a data directory that has no usable space. Do not use this method in production code.

Throws:

FSWriteError - if all directories are disallowed.

getWriteableLocationAsFile

public java.io.File getWriteableLocationAsFile(long writeSize)

Returns an allowed directory that _currently_ has writeSize bytes as usable space.

Throws:

FSWriteError - if all directories are disallowed.

getWriteableLocationToLoadFile

public java.io.File getWriteableLocationToLoadFile(java.io.File sourceFile)

Returns a data directory to load the file sourceFile. If the sourceFile is on same disk partition as any data directory then use that one as data directory otherwise use getWriteableLocationAsFile(long) to find suitable data directory. Also makes sure returned directory is not disallowed.

Throws:

FSWriteError - if all directories are disallowed.

getTemporaryWriteableDirectoryAsFile

public java.io.File getTemporaryWriteableDirectoryAsFile(long writeSize)

Returns a temporary subdirectory on allowed data directory that _currently_ has writeSize bytes as usable space. This method does not create the temporary directory.

Throws:

java.io.IOError - if all directories are disallowed.

removeTemporaryDirectories

public void removeTemporaryDirectories()

getWriteableLocation

public Directories.DataDirectory getWriteableLocation(long writeSize)

Returns an allowed data directory that _currently_ has writeSize bytes as usable space.

Throws:

FSWriteError - if all directories are disallowed.

hasAvailableDiskSpace

public boolean hasAvailableDiskSpace(long estimatedSSTables,
                                     long expectedTotalWriteSize)

getWriteableLocations

public Directories.DataDirectory[] getWriteableLocations()

getSnapshotDirectory

public static java.io.File getSnapshotDirectory(Descriptor desc,
                                                java.lang.String snapshotName)

getSnapshotDirectory

public static java.io.File getSnapshotDirectory(java.io.File location,
                                                java.lang.String snapshotName)

Returns directory to write snapshot. If directory does not exist, then one is created. If given location indicates secondary index, this will return <cf dir>/snapshots/<snapshot name>/.<index name>. Otherwise, this will return <cf dir>/snapshots/<snapshot name>.

Parameters:

location - base directory

snapshotName - snapshot name

Returns:

directory to write snapshot

getSnapshotManifestFile

public java.io.File getSnapshotManifestFile(java.lang.String snapshotName)

getSnapshotSchemaFile

public java.io.File getSnapshotSchemaFile(java.lang.String snapshotName)

getNewEphemeralSnapshotMarkerFile

public java.io.File getNewEphemeralSnapshotMarkerFile(java.lang.String snapshotName)

getBackupsDirectory

public static java.io.File getBackupsDirectory(Descriptor desc)

getBackupsDirectory

public static java.io.File getBackupsDirectory(java.io.File location)

isStoredInLocalSystemKeyspacesDataLocation

public static boolean isStoredInLocalSystemKeyspacesDataLocation(java.lang.String keyspace,
                                                                 java.lang.String table)

Checks if the specified table should be stored with local system data.

To minimize the risk of failures, SSTables for local system keyspaces must be stored in a single data directory. The only exception to this are some of the system table as the server can continue operating even if those tables loose some data.

Parameters:

keyspace - the keyspace name

table - the table name

Returns:

true if the specified table should be stored with local system data, false otherwise.

sstableLister

public Directories.SSTableLister sstableLister(Directories.OnTxnErr onTxnErr)

sstableLister

public Directories.SSTableLister sstableLister(java.io.File directory,
                                               Directories.OnTxnErr onTxnErr)

getSnapshotDetails

public java.util.Map<java.lang.String,Directories.SnapshotSizeDetails> getSnapshotDetails()

Returns:

Return a map of all snapshots to space being used The pair for a snapshot has size on disk and true size.

listEphemeralSnapshots

public java.util.List<java.lang.String> listEphemeralSnapshots()

snapshotExists

public boolean snapshotExists(java.lang.String snapshotName)

clearSnapshot

public static void clearSnapshot(java.lang.String snapshotName,
                                 java.util.List<java.io.File> snapshotDirectories,
                                 com.google.common.util.concurrent.RateLimiter snapshotRateLimiter)

snapshotCreationTime

public long snapshotCreationTime(java.lang.String snapshotName)

trueSnapshotsSize

public long trueSnapshotsSize()

Returns:

total snapshot size in byte for all snapshots.

getRawDiretoriesSize

public long getRawDiretoriesSize()

Returns:

Raw size on disk for all directories

getTrueAllocatedSizeIn

public long getTrueAllocatedSizeIn(java.io.File input)

getKSChildDirectories

public static java.util.List<java.io.File> getKSChildDirectories(java.lang.String ksName)

isSecondaryIndexFolder

public static boolean isSecondaryIndexFolder(java.io.File dir)

getCFDirectories

public java.util.List<java.io.File> getCFDirectories()