Package org.yamcs.yarch

Class TableDefinition

java.lang.Object

org.yamcs.yarch.TableDefinition

public class TableDefinition
extends Object

A table definition consists of a (key,value) pair of tuple definitions. A tuple has to contain all the columns from the key while it can contain only a few of the columns from the value (basically it's a sparse table).

The key is encoded as a byte array of all the columns in order. The value is encoded as a byte array of all the columns preceded by the id of their data type (1 byte) and their index (3 bytes).

The secondary index key is encoded as a byte array of all the columns in order preceded by the id of their data type with the first bit set to 1 for the columns present and 0 for the column not present (i.e. null).

A table can also be partitioned according to the partitioningSpec.

Field Summary

Fields

Modifier and Type	Field	Description
static int	CURRENT_FORMAT_VERSION

Constructor Summary

Constructors

Constructor	Description
TableDefinition(int formatVersion, List<TableColumnDefinition> key, List<TableColumnDefinition> value)	Used when creating the table from the serialized data on disk
TableDefinition(String name, TupleDefinition tdef, List<String> primaryKey)	Used when creating an empty table via sql.

Method Summary

Modifier and Type	Method	Description
Short	addAndGetEnumValue(String columnName, String value)	get the enum value corresponding to a column, creating it if it does not exist
void	changeDataType(String cname, DataType dataType)	this method is used during table migration between two versions
Tuple	deserialize(byte[] k, byte[] v)
Row	generateRow(Tuple t)	Generate a new table row by transforming the key part of the tuple into a byte array to be written to disk.
TableColumnDefinition	getColumnDefinition(String colName)
<T> ColumnSerializer<T>	getColumnSerializer(String columnName)
com.google.common.collect.BiMap<String,Short>	getEnumValues(String columnName)
int	getFormatVersion()
List<String>	getHistogramColumns()
IndexedList<String,TableColumnDefinition>	getHistoIdx()
List<TableColumnDefinition>	getKeyDefinition()
String	getName()
PartitioningSpec	getPartitioningSpec()
List<String>	getSecondaryIndex()
String	getStorageEngineName()
TupleDefinition	getTupleDefinition()
List<TableColumnDefinition>	getValueDefinition()
boolean	hasHistogram()
boolean	hasKey(String colName)
boolean	hasPartitioning()
boolean	hasSecondaryIndex()
boolean	isCompressed()
boolean	isIndexedByKey(String cname)
boolean	isPartitionedBy(String colName)	Return true if the table is partitioned and the colName is used as partition column (either time or value)
boolean	isPartitionedByTime()
byte[]	serializeValue(Tuple tuple, Row sertuple)	Transform the value part of the tuple into a byte array to be written on disk.
void	serializeValue(Tuple tuple, Row sertuple, ByteArray byteArray)	Same as serializeValue(Tuple, Row) but encodes the output in user provided byte array
void	setCompressed(boolean compressed)
void	setDb(YarchDatabaseInstance ydb)
void	setHistogramColumns(List<String> histoColumns)
void	setName(String name)
void	setPartitioningSpec(PartitioningSpec pspec)	time based partitions can be on the first column of the key (which has to be of type timestamp) value based partitions can be on any other mandatory column
void	setSecondaryIndex(List<String> index)
void	setStorageEngineName(String storageEngineName)
String	toString()
void	validate()	Checks that the table definition is valid: - primary key not string, except for the last in the list (otherwise the binary sorting does not work properly)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

CURRENT_FORMAT_VERSION

public static final int CURRENT_FORMAT_VERSION

See Also:

Constant Field Values

Constructor Detail

TableDefinition

public TableDefinition(String name,
                       TupleDefinition tdef,
                       List<String> primaryKey)
                throws StreamSqlException

Used when creating an empty table via sql.

Parameters:

name -

tdef -

primaryKey -

Throws:

StreamSqlException

TableDefinition

public TableDefinition(int formatVersion,
                       List<TableColumnDefinition> key,
                       List<TableColumnDefinition> value)

Used when creating the table from the serialized data on disk

Method Detail

setDb

public void setDb(YarchDatabaseInstance ydb)

setPartitioningSpec

public void setPartitioningSpec(PartitioningSpec pspec)
                         throws StreamSqlException

time based partitions can be on the first column of the key (which has to be of type timestamp) value based partitions can be on any other mandatory column

Parameters:

pspec -

Throws:

StreamSqlException

getKeyDefinition

public List<TableColumnDefinition> getKeyDefinition()

getValueDefinition

public List<TableColumnDefinition> getValueDefinition()

getName

public String getName()

setName

public void setName(String name)

getTupleDefinition

public TupleDefinition getTupleDefinition()

validate

public void validate()
              throws StreamSqlException

Checks that the table definition is valid: - primary key not string, except for the last in the list (otherwise the binary sorting does not work properly)

Throws:

StreamSqlException

generateRow

public Row generateRow(Tuple t)
                throws YarchException

Generate a new table row by transforming the key part of the tuple into a byte array to be written to disk. The tuple must contain each column from the key and they are written in order (such that sorting is according to the definition of the primary key).

In addition, it stores into the returned row all the values for the columns used in histograms or indices

Parameters:

t -

Returns:

a tuple containing the histogram and secondary index values as well as the generated key

Throws:

YarchException

addAndGetEnumValue

public Short addAndGetEnumValue(String columnName,
                                String value)

get the enum value corresponding to a column, creating it if it does not exist

Returns:

serializeValue

public void serializeValue(Tuple tuple,
                           Row sertuple,
                           ByteArray byteArray)

Same as serializeValue(Tuple, Row) but encodes the output in user provided byte array

Parameters:

tuple -

sertuple -

byteArray -

serializeValue

public byte[] serializeValue(Tuple tuple,
                             Row sertuple)

Transform the value part of the tuple into a byte array to be written on disk. Each column is preceded by a tag (the column index).

If there are columns in the tuple which are not in the valueDef, they are added and the TableDefinition is serialized on disk.

Columns whose values are null are not serialized but their definition is still added to the table definition if not present already.

Parameters:

tuple -

sertuple - - if not null, store all the values of the columns to this tuple as written to the database (possibly after some data casting)

Returns:

the serialized version of the value part of the tuple

deserialize

public Tuple deserialize(byte[] k,
                         byte[] v)

isCompressed

public boolean isCompressed()

isIndexedByKey

public boolean isIndexedByKey(String cname)

Parameters:

cname - the column name

Returns:

true if cname is the first column of the key

getColumnDefinition

public TableColumnDefinition getColumnDefinition(String colName)

hasPartitioning

public boolean hasPartitioning()

getPartitioningSpec

public PartitioningSpec getPartitioningSpec()

setCompressed

public void setCompressed(boolean compressed)

setHistogramColumns

public void setHistogramColumns(List<String> histoColumns)
                         throws StreamSqlException

Throws:

StreamSqlException

setSecondaryIndex

public void setSecondaryIndex(List<String> index)
                       throws StreamSqlException

Throws:

StreamSqlException

hasHistogram

public boolean hasHistogram()

getEnumValues

public com.google.common.collect.BiMap<String,Short> getEnumValues(String columnName)

getHistogramColumns

public List<String> getHistogramColumns()

getSecondaryIndex

public List<String> getSecondaryIndex()

getColumnSerializer

public <T> ColumnSerializer<T> getColumnSerializer(String columnName)

getStorageEngineName

public String getStorageEngineName()

setStorageEngineName

public void setStorageEngineName(String storageEngineName)

getFormatVersion

public int getFormatVersion()

toString

public String toString()

Overrides:

toString in class Object

changeDataType

public void changeDataType(String cname,
                           DataType dataType)

this method is used during table migration between two versions

It should not be used for other things.

hasKey

public boolean hasKey(String colName)

Parameters:

colName -

Returns:

true if the column is part of the primary key

isPartitionedByTime

public boolean isPartitionedByTime()

getHistoIdx

public IndexedList<String,TableColumnDefinition> getHistoIdx()

hasSecondaryIndex

public boolean hasSecondaryIndex()

isPartitionedBy

public boolean isPartitionedBy(String colName)

Return true if the table is partitioned and the colName is used as partition column (either time or value)

Parameters:

colName -

Returns: