com.apple.foundationdb.record.metadata

Class MetaDataEvolutionValidator

java.lang.Object

com.apple.foundationdb.record.metadata.MetaDataEvolutionValidator

@API(value=EXPERIMENTAL)
public class MetaDataEvolutionValidator
extends Object

This class can be used to validate that the new meta-data for a record store is responsibly evolved from its existing meta-data. In particular, this validates that:

• No record types are dropped.
• None of the fields of any existing record type are dropped.
• None of the fields change type except in ways that preserve their serialized form in both Protobuf fields and as Tuple elements.
• None of the fields change name (which is required as key expressions reference fields by name).
• None of the fields change their label (e.g., switch from optional to required.
• The new meta-data continues to split long records if the old meta-data did.
• New record types include the version in which they were introduced.
• The added and last modified versions of Indexes are consistent with the two meta-data versions.
• Any index that is removed is replaced with a FormerIndex.
• The added and removed versions of FormerIndexes are consistent with the two meta-data versions.
• Any changes to existing indexes do not change the on-disk format.

This is a relatively strict set of requirements that aims to allow anything written by a record store built with the old meta-data to be readable from record stores built with the new meta-data. Some of the validation rules, however, are stricter than necessary and are instead intended to prevent the user from accidentally changing the meta-data in a way that will require significant work. For example, it is assumed that any change that requires that an existing index be rebuilt is a mistake, but there are legitimate use cases for which this is actually the desired behavior. The user can disable those checks by creating a new validator and allowing laxer standards. For example, one can create a validator that allows format-incompatible changes to existing indexes by calling:

     MetaDataEvolutionValidator.newBuilder()
         .setAllowIndexRebuilds(true)
         .build();
 

See the Schema Evolution Guidelines in our documentation for more details on what changes may be safely made to the meta-data and why.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	MetaDataEvolutionValidator.Builder A builder class for the MetaDataEvolutionValidator.

Method Summary

Modifier and Type	Method and Description
boolean	allowsIndexRebuilds() Whether this validator allows new meta-data that require existing indexes to be rebuilt.
boolean	allowsMissingFormerIndexNames() Whether this validator allows former indexes to drop the name of the index they replace.
boolean	allowsNoSinceVersion() Whether this validator allows new record types to not declare a "since version".
boolean	allowsNoVersionChange() Whether this validator allows the evolved meta-data to have the same version as the old meta-data.
boolean	allowsOlderFormerIndexAddedVersions() Whether this validator allows former indexes to set their added version to something that is older than necessary.
boolean	allowsUnsplitToSplit() Whether this validator allows the meta-data to begin to split long records.
MetaDataEvolutionValidator.Builder	asBuilder() Convert this validator into a builder.
boolean	disallowsTypeRenames() Whether this validator disallows record types from being renamed.
static MetaDataEvolutionValidator	getDefaultInstance() Get the default validator.
IndexMaintainerRegistry	getIndexMaintainerRegistry() Get the registry of index maintainers used to validate indexes.
static MetaDataEvolutionValidator.Builder	newBuilder() Create a new builder.
void	validate(RecordMetaData oldMetaData, RecordMetaData newMetaData) Validate that the new meta-data has been safely evolved from an older version.
void	validateUnion(Descriptors.Descriptor oldUnionDescriptor, Descriptors.Descriptor newUnionDescriptor) Validate that the record types have all been evolved in a legal way.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

validate

public void validate(@Nonnull
                     RecordMetaData oldMetaData,
                     @Nonnull
                     RecordMetaData newMetaData)

Validate that the new meta-data has been safely evolved from an older version. This makes sure that existing records saved within the record store can still be read using the new meta-data and it verifies that indexes are added and removed in a safe way.

Parameters:

oldMetaData - the current meta-data for one or more record stores

newMetaData - the new meta-data for the same record stores

validateUnion

public void validateUnion(@Nonnull
                          Descriptors.Descriptor oldUnionDescriptor,
                          @Nonnull
                          Descriptors.Descriptor newUnionDescriptor)

Validate that the record types have all been evolved in a legal way. In particular, this makes sure that each record type defined in the union descriptor is in the new union descriptor in the correct place. It will then verify that each message type has been updated in a legal way, i.e., that it only includes new fields.

Parameters:

oldUnionDescriptor - the union descriptor for the existing meta-data for some record store

newUnionDescriptor - the new proposed meta-data

getIndexMaintainerRegistry

@Nonnull
public IndexMaintainerRegistry getIndexMaintainerRegistry()

Get the registry of index maintainers used to validate indexes. This registry should generally be the same registry as the registry that will be used by any record stores that may use the meta-data. By default, this uses the default IndexMaintainerRegistryImpl instance.

Returns:

the index maintainer registry used to validate indexes

See Also:

FDBRecordStoreBase.BaseBuilder.setIndexMaintainerRegistry(IndexMaintainerRegistry)

allowsNoVersionChange

public boolean allowsNoVersionChange()

Whether this validator allows the evolved meta-data to have the same version as the old meta-data. By default, this is false as the assumption is that this validator is only being called after the meta-data has been updated in some way, and it is good practice to update the meta-data version in that instance.

Returns:

whether this validator allows the evolved meta-data to have the same version as the old meta-data

allowsNoSinceVersion

public boolean allowsNoSinceVersion()

Whether this validator allows new record types to not declare a "since version". It is good practice to set the since version on newer record types as that can be used to prevent unnecessary index builds from occurring on indexes defined on the new types. However, failing to set this version is safe insofar as none of the existing data is corrupted and the extra work, while unfortunate, is safe to perform. By default, this option is set to false.

Returns:

whether this validator allows new record types to not declare a since version

allowsIndexRebuilds

public boolean allowsIndexRebuilds()

Whether this validator allows new meta-data that require existing indexes to be rebuilt. This can happen if an index is modified and its last modified version is updated. By default, this is false as index rebuilds are expensive and the index will not be queryable during the rebuild. The best practice if an index must be updated is to create a new index with the change and then remove the index once the new index has been built. This allows the old index to serve queries until such time as the new index is ready.

Returns:

whether this validator allows existing indexes to be modified in a ways that requires rebuilding

allowsMissingFormerIndexNames

public boolean allowsMissingFormerIndexNames()

Whether this validator allows former indexes to drop the name of the index they replace. This is information is only used for book-keeping, so it is safe to create a FormerIndex to replace an existing Index where FormerIndex.getFormerName() returns null. However, this can be a sign that some amount of information is being lost in the upgrade process, so the default value for this option is false, but users wanting more lenient validation can set it to true.

Returns:

whether this validator allows former indexes to drop the associated name of the index they replace

allowsOlderFormerIndexAddedVersions

public boolean allowsOlderFormerIndexAddedVersions()

Whether this validator allows former indexes to set their added version to something that is older than necessary. The added version of a FormerIndex is used as an optimization at meta-data upgrade time to decide not to clear the data for a former index if the index was both added and dropped since the meta-data version was last updated. As a result, it is safe to include older versions than the actual added version of the associated Index within a FormerIndex, but it might result in extra work and could also be a sign of another problem. As a result, the default value of this option is false, but users wanting more lenient validation can set it to true.

Returns:

whether this validator allows former indexes to have older added versions than necessary

allowsUnsplitToSplit

public boolean allowsUnsplitToSplit()

Whether this validator allows the meta-data to begin to split long records. For record stores that were first created with format version SAVE_UNSPLIT_WITH_SUFFIX_FORMAT_VERSION or newer, this change is safe as the data layout is the same regardless of the value of RecordMetaData.isSplitLongRecords(). However, older stores used a different format for if records were unsplit, and accidentally upgrading the meta-data to split long records is potentially catastrophic as every record in the store will now appear to be written under the wrong key. The default value of this option is thus false, but as the new format version has been the default since prior to version 2.0, most users should be able set this to true.

Returns:

whether this validator allows the meta-data to begin to split long records.

disallowsTypeRenames

public boolean disallowsTypeRenames()

Whether this validator disallows record types from being renamed. The record type name is not persisted to the database with any record, so renaming a record type is generally possible as long as one also updates the record type's name in the any referencing index. Note, however, that renaming a record type also requires that the user modify any existing queries that are restricted to that record type. As a result, the user may elect to disallow renaming record types to avoid needing to audit and update any references to the changed record type name.

Returns:

whether this validator disallows record types from being renamed

asBuilder

@Nonnull
public MetaDataEvolutionValidator.Builder asBuilder()

Convert this validator into a builder. This will copy over any options that have been set into the builder.

Returns:

a new MetaDataEvoluationValidator.Builder with the same options as this validator

newBuilder

@Nonnull
public static MetaDataEvolutionValidator.Builder newBuilder()

Create a new builder. This will have all of its options set to their default values.

Returns:

a new MetaDataEvoluationValidator.Builder with all options set to their defaults

getDefaultInstance

@Nonnull
public static MetaDataEvolutionValidator getDefaultInstance()

Get the default validator. This has all options set to their default values.

Returns:

the default validator