Class Tablespace
- java.lang.Object
-
- org.yamcs.yarch.rocksdb.Tablespace
-
public class Tablespace extends Object
Tablespaces are used to store data by theRdbStorageEngine
. Each tablespace can store data from one or more Yamcs instances.Tablespaces are rocksdb databases normally stored in the yamcs data directory (/storage/yamcs-data). Each corresponds to a directory <tablespace-name>.rdb and has a definition file tablespace-name.tbs.
Tablespaces can also have time based partitions in different RocksDB databases in sub-directories such as <tablespace-name>.rdb/YYYY/
There are two column families in the main database: - the _metadata_ column family - contains metadata. - the default column family - contains data.
The data is partitioned by the first 4 bytes of the key which we call tbsIndex.
One tbsIndex corresponds to a so called tablespace record. For example tbsIndex=5 can correspond to all telemetry packets for packet XYZ.
Except for the 4 bytes tbsIndex, the rest of the key and value are completely dependent on the data type. For example for yarch table data, the rest of key following the 4 bytes tbsIndex represents the key of the row in the table.
The metadata contains two types of records, identified by the first byte of the key:
- key: 0x1
value: 1 byte version number (0x1), 4 bytes max tbsIndex
used to store the max tbsIndex and also stores a version number in case the format will change in the future
- key: 0x2, 1 byte record type, 4 bytes tbsIndex
value: protobuf encoded TablespaceRecord
Used to store the information corresponding to the given tbsIndex. The record type corresponds to the Type enumerations from tablespace.proto
- key: 0x1
-
-
Constructor Summary
Constructors Constructor Description Tablespace(String name)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
close()
Tablespace.TablespaceRecord
createMetadataRecord(String yamcsInstance, Tablespace.TablespaceRecord.Builder trb)
Creates a new tablespace record and adds it to the metadata databasevoid
createTable(String yamcsInstance, TableDefinition tblDef)
void
dispose(YRDB rdb)
List<Tablespace.TablespaceRecord>
filter(Tablespace.TablespaceRecord.Type type, String instanceName, Predicate<Tablespace.TablespaceRecord.Builder> p)
String
getCustomDataDir()
byte[]
getData(byte[] key)
String
getDataDir()
String
getName()
YRDB
getRdb()
Get the main database of the tablespaceYRDB
getRdb(String relativePath)
YRDB
getRdb(String partitionDir, boolean readOnly)
(Creates) and returns a database in the given partition directory.RDBFactory
getRdbFactory()
Sequence
getSequence(String name)
RdbTable
getTable(TableDefinition tblDef)
returns the table associated to this definition or null if this table is not known.List<Tablespace.TablespaceRecord>
getTableHistograms(String instanceName, String tableName)
Returns a list of all records of type HISTOGRAM for a given instance and table If instanceName = tablespace name, it returns also records which do not have an instanceName specified.List<Tablespace.TablespaceRecord>
getTablePartitions(String instanceName, String tableName)
Returns a list of all records of type TABLE_PARTITION for a given instance and table If instanceName = tablespace name, it returns also records which do not have an instanceName specified.void
loadDb(boolean readonly)
TableWalker
newSecondaryIndexTableWalker(YarchDatabaseInstance ydb, TableDefinition tblDef, boolean ascending, boolean follow)
TableWalker
newTableWalker(YarchDatabaseInstance ydb, TableDefinition tblDef, boolean ascending, boolean follow)
RdbTableWriter
newTableWriter(YarchDatabaseInstance ydb, TableDefinition tblDef, TableWriter.InsertMode insertMode)
void
putData(byte[] key, byte[] value)
inserts data into the main partitionvoid
remove(byte[] key)
void
removeTbsIndex(Tablespace.TablespaceRecord.Type type, int tbsIndex)
Removes the tbsIndex from the metadata and all the associated data from the main db (data might still be present in the partitions)void
removeTbsIndices(Tablespace.TablespaceRecord.Type type, IntArray tbsIndexArray)
Removes the tbs indices with ALL the associated data from the main db (data might still be present in the partitions)void
setCustomDataDir(String dataDir)
-
-
-
Constructor Detail
-
Tablespace
public Tablespace(String name)
-
-
Method Detail
-
loadDb
public void loadDb(boolean readonly) throws IOException
- Throws:
IOException
-
getName
public String getName()
-
getTablePartitions
public List<Tablespace.TablespaceRecord> getTablePartitions(String instanceName, String tableName) throws org.rocksdb.RocksDBException, IOException
Returns a list of all records of type TABLE_PARTITION for a given instance and table If instanceName = tablespace name, it returns also records which do not have an instanceName specified.- Throws:
org.rocksdb.RocksDBException
IOException
-
getTableHistograms
public List<Tablespace.TablespaceRecord> getTableHistograms(String instanceName, String tableName) throws org.rocksdb.RocksDBException, IOException
Returns a list of all records of type HISTOGRAM for a given instance and table If instanceName = tablespace name, it returns also records which do not have an instanceName specified.- Throws:
org.rocksdb.RocksDBException
IOException
-
filter
public List<Tablespace.TablespaceRecord> filter(Tablespace.TablespaceRecord.Type type, String instanceName, Predicate<Tablespace.TablespaceRecord.Builder> p) throws YarchException, DatabaseCorruptionException
-
createMetadataRecord
public Tablespace.TablespaceRecord createMetadataRecord(String yamcsInstance, Tablespace.TablespaceRecord.Builder trb) throws org.rocksdb.RocksDBException
Creates a new tablespace record and adds it to the metadata database- Parameters:
trb
- the builder has to have all fields set except for the tbsIndex which will be assigned by this method- Returns:
- a fully built
- Throws:
org.rocksdb.RocksDBException
-
getCustomDataDir
public String getCustomDataDir()
-
getRdb
public YRDB getRdb(String partitionDir, boolean readOnly)
(Creates) and returns a database in the given partition directory. If the directory is null, return then main tablespace db- Parameters:
partitionDir
-readOnly
-
-
getRdb
public YRDB getRdb()
Get the main database of the tablespace
-
dispose
public void dispose(YRDB rdb)
-
setCustomDataDir
public void setCustomDataDir(String dataDir)
-
getDataDir
public String getDataDir()
-
removeTbsIndex
public void removeTbsIndex(Tablespace.TablespaceRecord.Type type, int tbsIndex) throws org.rocksdb.RocksDBException
Removes the tbsIndex from the metadata and all the associated data from the main db (data might still be present in the partitions)- Parameters:
type
-tbsIndex
-- Throws:
org.rocksdb.RocksDBException
-
removeTbsIndices
public void removeTbsIndices(Tablespace.TablespaceRecord.Type type, IntArray tbsIndexArray) throws org.rocksdb.RocksDBException
Removes the tbs indices with ALL the associated data from the main db (data might still be present in the partitions)- Parameters:
type
-tbsIndexArray
-- Throws:
org.rocksdb.RocksDBException
-
getRdbFactory
public RDBFactory getRdbFactory()
-
putData
public void putData(byte[] key, byte[] value) throws org.rocksdb.RocksDBException
inserts data into the main partition- Parameters:
key
-value
-- Throws:
org.rocksdb.RocksDBException
-
getData
public byte[] getData(byte[] key) throws org.rocksdb.RocksDBException
- Throws:
org.rocksdb.RocksDBException
-
remove
public void remove(byte[] key) throws org.rocksdb.RocksDBException
- Throws:
org.rocksdb.RocksDBException
-
createTable
public void createTable(String yamcsInstance, TableDefinition tblDef) throws YarchException, org.rocksdb.RocksDBException
- Throws:
YarchException
org.rocksdb.RocksDBException
-
getTable
public RdbTable getTable(TableDefinition tblDef)
returns the table associated to this definition or null if this table is not known.- Parameters:
tblDef
-- Returns:
-
newTableWalker
public TableWalker newTableWalker(YarchDatabaseInstance ydb, TableDefinition tblDef, boolean ascending, boolean follow)
-
newTableWriter
public RdbTableWriter newTableWriter(YarchDatabaseInstance ydb, TableDefinition tblDef, TableWriter.InsertMode insertMode)
-
close
public void close()
-
getSequence
public Sequence getSequence(String name) throws YarchException, org.rocksdb.RocksDBException
- Throws:
YarchException
org.rocksdb.RocksDBException
-
newSecondaryIndexTableWalker
public TableWalker newSecondaryIndexTableWalker(YarchDatabaseInstance ydb, TableDefinition tblDef, boolean ascending, boolean follow)
-
-