Package com.pulumi.aws.dms.outputs

Class EndpointS3Settings

  • public final class EndpointS3Settings
extends java.lang.Object

    • Method Detail

      • addColumnName

        public java.util.Optional<java.lang.Boolean> addColumnName()
        Returns:
        Whether to add column name information to the .csv output file. Default is `false`.

      • bucketFolder

        public java.util.Optional<java.lang.String> bucketFolder()
        Returns:
        Custom S3 Bucket Object prefix for intermediate storage.

      • bucketName

        public java.util.Optional<java.lang.String> bucketName()
        Returns:
        Custom S3 Bucket name for intermediate storage.

      • cannedAclForObjects

        public java.util.Optional<java.lang.String> cannedAclForObjects()
        Returns:
        Predefined (canned) access control list for objects created in an S3 bucket. Valid values include `none`, `private`, `public-read`, `public-read-write`, `authenticated-read`, `aws-exec-read`, `bucket-owner-read`, and `bucket-owner-full-control`. Default is `none`.

      • cdcInsertsAndUpdates

        public java.util.Optional<java.lang.Boolean> cdcInsertsAndUpdates()
        Returns:
        Whether to write insert and update operations to .csv or .parquet output files. Default is `false`.

      • cdcInsertsOnly

        public java.util.Optional<java.lang.Boolean> cdcInsertsOnly()
        Returns:
        Whether to write insert operations to .csv or .parquet output files. Default is `false`.

      • cdcMaxBatchInterval

        public java.util.Optional<java.lang.Integer> cdcMaxBatchInterval()
        Returns:
        Maximum length of the interval, defined in seconds, after which to output a file to Amazon S3. Default is `60`.

      • cdcMinFileSize

        public java.util.Optional<java.lang.Integer> cdcMinFileSize()
        Returns:
        Minimum file size condition as defined in kilobytes to output a file to Amazon S3. Default is `32000`. **NOTE:** Previously, this setting was measured in megabytes but now represents kilobytes. Update configurations accordingly.

      • cdcPath

        public java.util.Optional<java.lang.String> cdcPath()
        Returns:
        Folder path of CDC files. For an S3 source, this setting is required if a task captures change data; otherwise, it's optional. If `cdc_path` is set, AWS DMS reads CDC files from this path and replicates the data changes to the target endpoint. Supported in AWS DMS versions 3.4.2 and later.

      • compressionType

        public java.util.Optional<java.lang.String> compressionType()
        Returns:
        Set to compress target files. Default is `NONE`. Valid values are `GZIP` and `NONE`.

      • csvDelimiter

        public java.util.Optional<java.lang.String> csvDelimiter()
        Returns:
        Delimiter used to separate columns in the source files. Default is `,`.

      • csvNoSupValue

        public java.util.Optional<java.lang.String> csvNoSupValue()
        Returns:
        String to use for all columns not included in the supplemental log.

      • csvNullValue

        public java.util.Optional<java.lang.String> csvNullValue()
        Returns:
        String to as null when writing to the target.

      • csvRowDelimiter

        public java.util.Optional<java.lang.String> csvRowDelimiter()
        Returns:
        Delimiter used to separate rows in the source files. Default is `\n`.

      • dataFormat

        public java.util.Optional<java.lang.String> dataFormat()
        Returns:
        Output format for the files that AWS DMS uses to create S3 objects. Valid values are `csv` and `parquet`. Default is `csv`.

      • dataPageSize

        public java.util.Optional<java.lang.Integer> dataPageSize()
        Returns:
        Size of one data page in bytes. Default is `1048576` (1 MiB).

      • datePartitionDelimiter

        public java.util.Optional<java.lang.String> datePartitionDelimiter()
        Returns:
        Date separating delimiter to use during folder partitioning. Valid values are `SLASH`, `UNDERSCORE`, `DASH`, and `NONE`. Default is `SLASH`.

      • datePartitionEnabled

        public java.util.Optional<java.lang.Boolean> datePartitionEnabled()
        Returns:
        Partition S3 bucket folders based on transaction commit dates. Default is `false`.

      • datePartitionSequence

        public java.util.Optional<java.lang.String> datePartitionSequence()
        Returns:
        Date format to use during folder partitioning. Use this parameter when `date_partition_enabled` is set to true. Valid values are `YYYYMMDD`, `YYYYMMDDHH`, `YYYYMM`, `MMYYYYDD`, and `DDMMYYYY`. Default is `YYYYMMDD`.

      • dictPageSizeLimit

        public java.util.Optional<java.lang.Integer> dictPageSizeLimit()
        Returns:
        Maximum size in bytes of an encoded dictionary page of a column. Default is `1048576` (1 MiB).

      • enableStatistics

        public java.util.Optional<java.lang.Boolean> enableStatistics()
        Returns:
        Whether to enable statistics for Parquet pages and row groups. Default is `true`.

      • encodingType

        public java.util.Optional<java.lang.String> encodingType()
        Returns:
        Type of encoding to use. Value values are `rle_dictionary`, `plain`, and `plain_dictionary`. Default is `rle_dictionary`.

      • encryptionMode

        public java.util.Optional<java.lang.String> encryptionMode()
        Returns:
        The server-side encryption mode that you want to encrypt your intermediate .csv object files copied to S3. Defaults to `SSE_S3`. Valid values are `SSE_S3` and `SSE_KMS`.

      • externalTableDefinition

        public java.util.Optional<java.lang.String> externalTableDefinition()
        Returns:
        JSON document that describes how AWS DMS should interpret the data.

      • glueCatalogGeneration

        public java.util.Optional<java.lang.Boolean> glueCatalogGeneration()
        Returns:
        Whether to integrate AWS Glue Data Catalog with an Amazon S3 target. See [Using AWS Glue Data Catalog with an Amazon S3 target for AWS DMS](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Target.S3.html#CHAP_Target.S3.GlueCatalog) for more information. Default is `false`.

      • ignoreHeaderRows

        public java.util.Optional<java.lang.Integer> ignoreHeaderRows()
        Returns:
        When this value is set to `1`, DMS ignores the first row header in a .csv file. Default is `0`.

      • includeOpForFullLoad

        public java.util.Optional<java.lang.Boolean> includeOpForFullLoad()
        Returns:
        Whether to enable a full load to write INSERT operations to the .csv output files only to indicate how the rows were added to the source database. Default is `false`.

      • maxFileSize

        public java.util.Optional<java.lang.Integer> maxFileSize()
        Returns:
        Specifies the maximum size (in KB) of any .csv file used to transfer data to PostgreSQL. Default is `32,768 KB`.

      • parquetTimestampInMillisecond

        public java.util.Optional<java.lang.Boolean> parquetTimestampInMillisecond()
        Returns:
        Specifies the precision of any TIMESTAMP column values written to an S3 object file in .parquet format. Default is `false`.

      • parquetVersion

        public java.util.Optional<java.lang.String> parquetVersion()
        Returns:
        Version of the .parquet file format. Default is `parquet-1-0`. Valid values are `parquet-1-0` and `parquet-2-0`.

      • preserveTransactions

        public java.util.Optional<java.lang.Boolean> preserveTransactions()
        Returns:
        Whether DMS saves the transaction order for a CDC load on the S3 target specified by `cdc_path`. Default is `false`.

      • rfc4180

        public java.util.Optional<java.lang.Boolean> rfc4180()
        Returns:
        For an S3 source, whether each leading double quotation mark has to be followed by an ending double quotation mark. Default is `true`.

      • rowGroupLength

        public java.util.Optional<java.lang.Integer> rowGroupLength()
        Returns:
        Number of rows in a row group. Default is `10000`.

      • serverSideEncryptionKmsKeyId

        public java.util.Optional<java.lang.String> serverSideEncryptionKmsKeyId()
        Returns:
        ARN or Id of KMS Key to use when `encryption_mode` is `SSE_KMS`.

      • serviceAccessRoleArn

        public java.util.Optional<java.lang.String> serviceAccessRoleArn()
        Returns:
        ARN of the IAM Role with permissions to write to the OpenSearch cluster.

      • timestampColumnName

        public java.util.Optional<java.lang.String> timestampColumnName()
        Returns:
        Column to add with timestamp information to the endpoint data for an Amazon S3 target.

      • useCsvNoSupValue

        public java.util.Optional<java.lang.Boolean> useCsvNoSupValue()
        Returns:
        Whether to use `csv_no_sup_value` for columns not included in the supplemental log.

      • useTaskStartTimeForFullLoadTimestamp

        public java.util.Optional<java.lang.Boolean> useTaskStartTimeForFullLoadTimestamp()
        Returns:
        When set to true, uses the task start time as the timestamp column value instead of the time data is written to target. For full load, when set to true, each row of the timestamp column contains the task start time. For CDC loads, each row of the timestamp column contains the transaction commit time. When set to false, the full load timestamp in the timestamp column increments with the time data arrives at the target. Default is `false`.