Package org.apache.parquet.crypto

Interface EncryptionPropertiesFactory

  • All Known Implementing Classes:
    PropertiesDrivenCryptoFactory
    public interface EncryptionPropertiesFactory
    EncryptionPropertiesFactory interface enables transparent activation of Parquet encryption. Its customized implementations produce encryption properties for each Parquet file, using the input information available in Parquet file writers: file path, file extended schema - and also Hadoop configuration properties that can pass custom parameters required by a crypto factory. A factory implementation can use or ignore any of these inputs. The example usage could be as below. 1. Write a class to implement EncryptionPropertiesFactory. 2. Set configuration of "parquet.crypto.factory.class" with the fully qualified name of this class. For example, we can set the configuration in SparkSession as below. SparkSession spark = SparkSession .config("parquet.crypto.factory.class", "xxx.xxx.EncryptionPropertiesClassLoaderImpl") The implementation of this interface will be instantiated by loadFactory(Configuration).

    • Field Detail

      • LOG

        static final org.slf4j.Logger LOG

    • Method Detail

      • loadFactory

        static EncryptionPropertiesFactory loadFactory​(org.apache.hadoop.conf.Configuration conf)
        Load EncryptionPropertiesFactory class specified by CRYPTO_FACTORY_CLASS_PROPERTY_NAME as the path in the configuration
        Parameters:
        conf - Configuration where user specifies the class path
        Returns:
        object with class EncryptionPropertiesFactory if user specified the class path and invoking of the class succeeds. Null if user doesn't specify the class path (no encryption then).
        Throws:
        BadConfigurationException - if the instantiation of the configured class fails

      • getFileEncryptionProperties

        FileEncryptionProperties getFileEncryptionProperties​(org.apache.hadoop.conf.Configuration fileHadoopConfig,
                                                     org.apache.hadoop.fs.Path tempFilePath,
                                                     WriteSupport.WriteContext fileWriteContext)
                                              throws ParquetCryptoRuntimeException
        Get FileEncryptionProperties object which is created by the implementation of this interface. Please see the unit test SampleEncryptionPropertiesFactory for example
        Parameters:
        fileHadoopConfig - Configuration that is used to pass the needed information, e.g. KMS uri
        tempFilePath - File path of the parquet file being written. Can be used for AAD prefix creation, key material management, etc. Implementations must not presume the path is permanent, as the file can be moved or renamed later
        fileWriteContext - WriteContext to provide information like schema to build the FileEncryptionProperties
        Returns:
        object with class of FileEncryptionProperties. Null return value means the file should not be encrypted.
        Throws:
        ParquetCryptoRuntimeException - if there is an exception while creating the object