Class TransformedRecordSerializer<M extends Message>
- java.lang.Object
-
- com.apple.foundationdb.record.provider.common.TransformedRecordSerializer<M>
-
- Type Parameters:
M
- type ofMessage
that underlying records will use
- All Implemented Interfaces:
RecordSerializer<M>
- Direct Known Subclasses:
TransformedRecordSerializerJCE
@API(UNSTABLE) public class TransformedRecordSerializer<M extends Message> extends Object implements RecordSerializer<M>
ARecordSerializer
implementation that transforms the bytes produced by another serializer before returning (and likewise performs the reverse transformation before deserializing). At the moment, the transformations are encryption and compression, but other transformations could be added in the future.This serializer will begin each serialized string with a one-byte prefix containing information about which transformations were performed. This way, when deserializing, it can detect which transformations were applied so it knows which ones it needs to use to restore the original record. This also allows it to do things like decide not to compress records, even if compression is turned on, if the compression algorithm actually produces a byte array that is larger than the raw record, for example.
This base class does not itself support encryption: an exception will be thrown when trying to construct a serializer with encryption enabled or when encountering a serialized record that requires decryption. Subclasses, such as
TransformedRecordSerializerJCE
, implement encryption and have additional state like secret keys.It is designed to be compatible with the
DynamicMessageRecordSerializer
insofar as it can deserialize records written by that serializer, though records serialized by this class cannot be read by instances of that class. In the future, we might remove that capability (when all existing data have been migrated to use this class).
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
TransformedRecordSerializer.Builder<M extends Message>
Builder class for theTransformedRecordSerializer
class.protected static class
TransformedRecordSerializer.TransformState
-
Nested classes/interfaces inherited from interface com.apple.foundationdb.record.provider.common.RecordSerializer
RecordSerializer.Counts, RecordSerializer.Events
-
-
Field Summary
Fields Modifier and Type Field Description protected int
compressionLevel
protected boolean
compressWhenSerializing
protected static int
DEFAULT_COMPRESSION_LEVEL
protected static int
ENCODING_CLEAR
protected static int
ENCODING_COMPRESSED
protected static int
ENCODING_ENCRYPTED
protected static int
ENCODING_PROTO_MESSAGE_FIELD
protected static int
ENCODING_PROTO_TYPE_MASK
protected boolean
encryptWhenSerializing
protected RecordSerializer<M>
inner
protected static int
MAX_COMPRESSION_VERSION
protected static int
MIN_COMPRESSION_VERSION
-
Constructor Summary
Constructors Modifier Constructor Description protected
TransformedRecordSerializer(RecordSerializer<M> inner, boolean compressWhenSerializing, int compressionLevel, boolean encryptWhenSerializing)
-
Method Summary
-
-
-
Field Detail
-
ENCODING_ENCRYPTED
protected static final int ENCODING_ENCRYPTED
- See Also:
- Constant Field Values
-
ENCODING_CLEAR
protected static final int ENCODING_CLEAR
- See Also:
- Constant Field Values
-
ENCODING_COMPRESSED
protected static final int ENCODING_COMPRESSED
- See Also:
- Constant Field Values
-
ENCODING_PROTO_MESSAGE_FIELD
protected static final int ENCODING_PROTO_MESSAGE_FIELD
- See Also:
- Constant Field Values
-
ENCODING_PROTO_TYPE_MASK
protected static final int ENCODING_PROTO_TYPE_MASK
- See Also:
- Constant Field Values
-
DEFAULT_COMPRESSION_LEVEL
protected static final int DEFAULT_COMPRESSION_LEVEL
- See Also:
- Constant Field Values
-
MIN_COMPRESSION_VERSION
protected static final int MIN_COMPRESSION_VERSION
- See Also:
- Constant Field Values
-
MAX_COMPRESSION_VERSION
protected static final int MAX_COMPRESSION_VERSION
- See Also:
- Constant Field Values
-
inner
@Nonnull protected final RecordSerializer<M extends Message> inner
-
compressWhenSerializing
protected final boolean compressWhenSerializing
-
compressionLevel
protected final int compressionLevel
-
encryptWhenSerializing
protected final boolean encryptWhenSerializing
-
-
Constructor Detail
-
TransformedRecordSerializer
protected TransformedRecordSerializer(@Nonnull RecordSerializer<M> inner, boolean compressWhenSerializing, int compressionLevel, boolean encryptWhenSerializing)
-
-
Method Detail
-
compress
protected void compress(@Nonnull TransformedRecordSerializer.TransformState state, @Nullable StoreTimer timer)
-
encrypt
protected void encrypt(@Nonnull TransformedRecordSerializer.TransformState state, @Nullable StoreTimer timer) throws GeneralSecurityException
- Throws:
GeneralSecurityException
-
serialize
@Nonnull public byte[] serialize(@Nonnull RecordMetaData metaData, @Nonnull RecordType recordType, @Nonnull M record, @Nullable StoreTimer timer)
Description copied from interface:RecordSerializer
Convert a Protobuf record to bytes. While Protobuf messages provide their ownMessageLite.toByteArray()
method for serialization, record stores may elect to first wrap the raw Protobuf record in another wrapping type or perform additional transformations like encrypt or compress records. Implementors of this interface can control how exactly which transformations are done, with the main constraint being that whatever operation is done should be reversible by calling thedeserialize()
method on those bytes. Implementors should also be careful as they evolve this method that thedeserialize
method is still able to read older data unless they are certain that any record store that used the older implementation has since been cleared out or migrated to a newer format.- Specified by:
serialize
in interfaceRecordSerializer<M extends Message>
- Parameters:
metaData
- the store's meta-datarecordType
- the record type of the messagerecord
- the Protobuf record to serializetimer
- a timer used to instrument serialization- Returns:
- the serialized record
-
decompress
protected void decompress(@Nonnull TransformedRecordSerializer.TransformState state, @Nullable StoreTimer timer) throws DataFormatException
- Throws:
DataFormatException
-
decrypt
protected void decrypt(@Nonnull TransformedRecordSerializer.TransformState state, @Nullable StoreTimer timer) throws GeneralSecurityException
- Throws:
GeneralSecurityException
-
deserialize
@Nonnull public M deserialize(@Nonnull RecordMetaData metaData, @Nonnull Tuple primaryKey, @Nonnull byte[] serialized, @Nullable StoreTimer timer)
Description copied from interface:RecordSerializer
Convert a byte array to a Protobuf record. This should be the inverse of theserialize()
method.- Specified by:
deserialize
in interfaceRecordSerializer<M extends Message>
- Parameters:
metaData
- the store's meta-dataprimaryKey
- the primary key of the recordserialized
- the serialized bytestimer
- a timer used to instrument deserialization- Returns:
- the deserialized record
-
widen
@Nonnull public RecordSerializer<Message> widen()
Description copied from interface:RecordSerializer
Convert this typed record serializer to an untyped one.- If this serializer wraps another serializer, for example, because it does compressions, then
widen
that serializer and return the result wrapped equivalently. - If this serializer parses messages in a type-sensitive way, return the closest general purpose way.
- If this serializer is inherently bound to its type parameter, throw an exception.
If a
FDBTypedRecordStore
is created without callingFDBTypedRecordStore.Builder.setUntypedSerializer(com.apple.foundationdb.record.provider.common.RecordSerializer<com.google.protobuf.Message>)
, then in order to make the untyped record store that is paired with the typed store (used, for instance, to rebuild indexes), the given typed serializer will be widened by calling this method.- Specified by:
widen
in interfaceRecordSerializer<M extends Message>
- Returns:
- a new serializer that works the same way but handles all record types.
- If this serializer wraps another serializer, for example, because it does compressions, then
-
newDefaultBuilder
public static TransformedRecordSerializer.Builder<Message> newDefaultBuilder()
Creates a newTransformedRecordSerializer.Builder
instance that is backed by the default serializer forMessage
s, namely aDynamicMessageRecordSerializer
. Methods on the returnedBuilder
instance can be used to specify which transformations to apply after using the default serializer.- Returns:
Builder
instance backed by aDynamicMessageRecordSerializer
-
newBuilder
public static <M extends Message> TransformedRecordSerializer.Builder<M> newBuilder(@Nonnull RecordSerializer<M> inner)
Creates a newTransformedRecordSerializer.Builder
instance around the given serializer. Methods on theBuilder
instance can be used to specify which transformations after using the provided serializer.- Type Parameters:
M
- type ofMessage
that underlying records will use- Parameters:
inner
-RecordSerializer
to use before/after applying transformations- Returns:
Builder
instance that can be used to specify transformations
-
-