Package com.apple.foundationdb.record.provider.common

Class TransformedRecordSerializer<M extends Message>

java.lang.Object

com.apple.foundationdb.record.provider.common.TransformedRecordSerializer<M>

Type Parameters:

M - type of Message that underlying records will use

All Implemented Interfaces:

RecordSerializer<M>

Direct Known Subclasses:

TransformedRecordSerializerJCE

@API(UNSTABLE)
public class TransformedRecordSerializer<M extends Message>
extends Object
implements RecordSerializer<M>

A RecordSerializer implementation that transforms the bytes produced by another serializer before returning (and likewise performs the reverse transformation before deserializing). At the moment, the transformations are encryption and compression, but other transformations could be added in the future.

This serializer will begin each serialized string with a one-byte prefix containing information about which transformations were performed. This way, when deserializing, it can detect which transformations were applied so it knows which ones it needs to use to restore the original record. This also allows it to do things like decide not to compress records, even if compression is turned on, if the compression algorithm actually produces a byte array that is larger than the raw record, for example.

This base class does not itself support encryption: an exception will be thrown when trying to construct a serializer with encryption enabled or when encountering a serialized record that requires decryption. Subclasses, such as TransformedRecordSerializerJCE, implement encryption and have additional state like secret keys.

It is designed to be compatible with the DynamicMessageRecordSerializer insofar as it can deserialize records written by that serializer, though records serialized by this class cannot be read by instances of that class. In the future, we might remove that capability (when all existing data have been migrated to use this class).

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	TransformedRecordSerializer.Builder<M extends Message>	Builder class for the TransformedRecordSerializer class.
protected static class	TransformedRecordSerializer.TransformState

Nested classes/interfaces inherited from interface com.apple.foundationdb.record.provider.common.RecordSerializer

RecordSerializer.Counts, RecordSerializer.Events

Field Summary

Fields

Modifier and Type	Field	Description
protected int	compressionLevel
protected boolean	compressWhenSerializing
protected static int	DEFAULT_COMPRESSION_LEVEL
protected static int	ENCODING_CLEAR
protected static int	ENCODING_COMPRESSED
protected static int	ENCODING_ENCRYPTED
protected static int	ENCODING_PROTO_MESSAGE_FIELD
protected static int	ENCODING_PROTO_TYPE_MASK
protected boolean	encryptWhenSerializing
protected RecordSerializer<M>	inner
protected static int	MAX_COMPRESSION_VERSION
protected static int	MIN_COMPRESSION_VERSION

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	TransformedRecordSerializer(RecordSerializer<M> inner, boolean compressWhenSerializing, int compressionLevel, boolean encryptWhenSerializing)

Method Summary

Modifier and Type	Method	Description
protected void	compress(TransformedRecordSerializer.TransformState state, StoreTimer timer)
protected void	decompress(TransformedRecordSerializer.TransformState state, StoreTimer timer)
protected void	decrypt(TransformedRecordSerializer.TransformState state, StoreTimer timer)
M	deserialize(RecordMetaData metaData, Tuple primaryKey, byte[] serialized, StoreTimer timer)	Convert a byte array to a Protobuf record.
protected void	encrypt(TransformedRecordSerializer.TransformState state, StoreTimer timer)
static <M extends Message> TransformedRecordSerializer.Builder<M>	newBuilder(RecordSerializer<M> inner)	Creates a new TransformedRecordSerializer.Builder instance around the given serializer.
static TransformedRecordSerializer.Builder<Message>	newDefaultBuilder()	Creates a new TransformedRecordSerializer.Builder instance that is backed by the default serializer for Messages, namely a DynamicMessageRecordSerializer.
byte[]	serialize(RecordMetaData metaData, RecordType recordType, M record, StoreTimer timer)	Convert a Protobuf record to bytes.
RecordSerializer<Message>	widen()	Convert this typed record serializer to an untyped one.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ENCODING_ENCRYPTED

protected static final int ENCODING_ENCRYPTED

See Also:

Constant Field Values

ENCODING_CLEAR

protected static final int ENCODING_CLEAR

See Also:

Constant Field Values

ENCODING_COMPRESSED

protected static final int ENCODING_COMPRESSED

See Also:

Constant Field Values

ENCODING_PROTO_MESSAGE_FIELD

protected static final int ENCODING_PROTO_MESSAGE_FIELD

See Also:

Constant Field Values

ENCODING_PROTO_TYPE_MASK

protected static final int ENCODING_PROTO_TYPE_MASK

See Also:

Constant Field Values

DEFAULT_COMPRESSION_LEVEL

protected static final int DEFAULT_COMPRESSION_LEVEL

See Also:

Constant Field Values

MIN_COMPRESSION_VERSION

protected static final int MIN_COMPRESSION_VERSION

See Also:

Constant Field Values

MAX_COMPRESSION_VERSION

protected static final int MAX_COMPRESSION_VERSION

See Also:

Constant Field Values

inner

@Nonnull
protected final RecordSerializer<M extends Message> inner

compressWhenSerializing

protected final boolean compressWhenSerializing

compressionLevel

protected final int compressionLevel

encryptWhenSerializing

protected final boolean encryptWhenSerializing

Constructor Detail

TransformedRecordSerializer

protected TransformedRecordSerializer(@Nonnull
                                      RecordSerializer<M> inner,
                                      boolean compressWhenSerializing,
                                      int compressionLevel,
                                      boolean encryptWhenSerializing)

Method Detail

compress

protected void compress(@Nonnull
                        TransformedRecordSerializer.TransformState state,
                        @Nullable
                        StoreTimer timer)

encrypt

protected void encrypt(@Nonnull
                       TransformedRecordSerializer.TransformState state,
                       @Nullable
                       StoreTimer timer)
                throws GeneralSecurityException

Throws:

GeneralSecurityException

serialize

@Nonnull
public byte[] serialize(@Nonnull
                        RecordMetaData metaData,
                        @Nonnull
                        RecordType recordType,
                        @Nonnull
                        M record,
                        @Nullable
                        StoreTimer timer)

Description copied from interface: RecordSerializer

Convert a Protobuf record to bytes. While Protobuf messages provide their own MessageLite.toByteArray() method for serialization, record stores may elect to first wrap the raw Protobuf record in another wrapping type or perform additional transformations like encrypt or compress records. Implementors of this interface can control how exactly which transformations are done, with the main constraint being that whatever operation is done should be reversible by calling the deserialize() method on those bytes. Implementors should also be careful as they evolve this method that the deserialize method is still able to read older data unless they are certain that any record store that used the older implementation has since been cleared out or migrated to a newer format.

Specified by:

serialize in interface RecordSerializer<M extends Message>

Parameters:

metaData - the store's meta-data

recordType - the record type of the message

record - the Protobuf record to serialize

timer - a timer used to instrument serialization

Returns:

the serialized record

decompress

protected void decompress(@Nonnull
                          TransformedRecordSerializer.TransformState state,
                          @Nullable
                          StoreTimer timer)
                   throws DataFormatException

Throws:

DataFormatException

decrypt

protected void decrypt(@Nonnull
                       TransformedRecordSerializer.TransformState state,
                       @Nullable
                       StoreTimer timer)
                throws GeneralSecurityException

Throws:

GeneralSecurityException

deserialize

@Nonnull
public M deserialize(@Nonnull
                     RecordMetaData metaData,
                     @Nonnull
                     Tuple primaryKey,
                     @Nonnull
                     byte[] serialized,
                     @Nullable
                     StoreTimer timer)

Description copied from interface: RecordSerializer

Convert a byte array to a Protobuf record. This should be the inverse of the serialize() method.

Specified by:

deserialize in interface RecordSerializer<M extends Message>

Parameters:

metaData - the store's meta-data

primaryKey - the primary key of the record

serialized - the serialized bytes

timer - a timer used to instrument deserialization

Returns:

the deserialized record

widen

@Nonnull
public RecordSerializer<Message> widen()

Description copied from interface: RecordSerializer

Convert this typed record serializer to an untyped one.

• If this serializer wraps another serializer, for example, because it does compressions, then widen that serializer and return the result wrapped equivalently.
• If this serializer parses messages in a type-sensitive way, return the closest general purpose way.
• If this serializer is inherently bound to its type parameter, throw an exception.

If a FDBTypedRecordStore is created without calling FDBTypedRecordStore.Builder.setUntypedSerializer(com.apple.foundationdb.record.provider.common.RecordSerializer<com.google.protobuf.Message>), then in order to make the untyped record store that is paired with the typed store (used, for instance, to rebuild indexes), the given typed serializer will be widened by calling this method.

Specified by:

widen in interface RecordSerializer<M extends Message>

Returns:

a new serializer that works the same way but handles all record types.

newDefaultBuilder

public static TransformedRecordSerializer.Builder<Message> newDefaultBuilder()

Creates a new TransformedRecordSerializer.Builder instance that is backed by the default serializer for Messages, namely a DynamicMessageRecordSerializer. Methods on the returned Builder instance can be used to specify which transformations to apply after using the default serializer.

Returns:

Builder instance backed by a DynamicMessageRecordSerializer

newBuilder

public static <M extends Message> TransformedRecordSerializer.Builder<M> newBuilder(@Nonnull
                                                                                    RecordSerializer<M> inner)

Creates a new TransformedRecordSerializer.Builder instance around the given serializer. Methods on the Builder instance can be used to specify which transformations after using the provided serializer.

Type Parameters:

M - type of Message that underlying records will use

Parameters:

inner - RecordSerializer to use before/after applying transformations

Returns:

Builder instance that can be used to specify transformations