public static interface OrcSerDe.Builder extends SdkPojo, CopyableBuilder<OrcSerDe.Builder,OrcSerDe>
Modifier and Type | Method and Description |
---|---|
OrcSerDe.Builder |
blockSizeBytes(Integer blockSizeBytes)
The Hadoop Distributed File System (HDFS) block size.
|
OrcSerDe.Builder |
bloomFilterColumns(Collection<String> bloomFilterColumns)
The column names for which you want Kinesis Data Firehose to create bloom filters.
|
OrcSerDe.Builder |
bloomFilterColumns(String... bloomFilterColumns)
The column names for which you want Kinesis Data Firehose to create bloom filters.
|
OrcSerDe.Builder |
bloomFilterFalsePositiveProbability(Double bloomFilterFalsePositiveProbability)
The Bloom filter false positive probability (FPP).
|
OrcSerDe.Builder |
compression(OrcCompression compression)
The compression code to use over data blocks.
|
OrcSerDe.Builder |
compression(String compression)
The compression code to use over data blocks.
|
OrcSerDe.Builder |
dictionaryKeyThreshold(Double dictionaryKeyThreshold)
Represents the fraction of the total number of non-null rows.
|
OrcSerDe.Builder |
enablePadding(Boolean enablePadding)
Set this to
true to indicate that you want stripes to be padded to the HDFS block boundaries. |
OrcSerDe.Builder |
formatVersion(OrcFormatVersion formatVersion)
The version of the file to write.
|
OrcSerDe.Builder |
formatVersion(String formatVersion)
The version of the file to write.
|
OrcSerDe.Builder |
paddingTolerance(Double paddingTolerance)
A number between 0 and 1 that defines the tolerance for block padding as a decimal fraction of stripe size.
|
OrcSerDe.Builder |
rowIndexStride(Integer rowIndexStride)
The number of rows between index entries.
|
OrcSerDe.Builder |
stripeSizeBytes(Integer stripeSizeBytes)
The number of bytes in each stripe.
|
equalsBySdkFields, sdkFields
copy
applyMutation, build
OrcSerDe.Builder stripeSizeBytes(Integer stripeSizeBytes)
The number of bytes in each stripe. The default is 64 MiB and the minimum is 8 MiB.
stripeSizeBytes
- The number of bytes in each stripe. The default is 64 MiB and the minimum is 8 MiB.OrcSerDe.Builder blockSizeBytes(Integer blockSizeBytes)
The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.
blockSizeBytes
- The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data
from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data
Firehose uses this value for padding calculations.OrcSerDe.Builder rowIndexStride(Integer rowIndexStride)
The number of rows between index entries. The default is 10,000 and the minimum is 1,000.
rowIndexStride
- The number of rows between index entries. The default is 10,000 and the minimum is 1,000.OrcSerDe.Builder enablePadding(Boolean enablePadding)
Set this to true
to indicate that you want stripes to be padded to the HDFS block boundaries.
This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is
false
.
enablePadding
- Set this to true
to indicate that you want stripes to be padded to the HDFS block
boundaries. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The
default is false
.OrcSerDe.Builder paddingTolerance(Double paddingTolerance)
A number between 0 and 1 that defines the tolerance for block padding as a decimal fraction of stripe size. The default value is 0.05, which means 5 percent of stripe size.
For the default values of 64 MiB ORC stripes and 256 MiB HDFS blocks, the default block padding tolerance of 5 percent reserves a maximum of 3.2 MiB for padding within the 256 MiB block. In such a case, if the available size within the block is more than 3.2 MiB, a new, smaller stripe is inserted to fit within that space. This ensures that no stripe crosses block boundaries and causes remote reads within a node-local task.
Kinesis Data Firehose ignores this parameter when OrcSerDe$EnablePadding is false
.
paddingTolerance
- A number between 0 and 1 that defines the tolerance for block padding as a decimal fraction of stripe
size. The default value is 0.05, which means 5 percent of stripe size.
For the default values of 64 MiB ORC stripes and 256 MiB HDFS blocks, the default block padding tolerance of 5 percent reserves a maximum of 3.2 MiB for padding within the 256 MiB block. In such a case, if the available size within the block is more than 3.2 MiB, a new, smaller stripe is inserted to fit within that space. This ensures that no stripe crosses block boundaries and causes remote reads within a node-local task.
Kinesis Data Firehose ignores this parameter when OrcSerDe$EnablePadding is false
.
OrcSerDe.Builder compression(String compression)
The compression code to use over data blocks. The default is SNAPPY
.
compression
- The compression code to use over data blocks. The default is SNAPPY
.OrcCompression
,
OrcCompression
OrcSerDe.Builder compression(OrcCompression compression)
The compression code to use over data blocks. The default is SNAPPY
.
compression
- The compression code to use over data blocks. The default is SNAPPY
.OrcCompression
,
OrcCompression
OrcSerDe.Builder bloomFilterColumns(Collection<String> bloomFilterColumns)
The column names for which you want Kinesis Data Firehose to create bloom filters. The default is
null
.
bloomFilterColumns
- The column names for which you want Kinesis Data Firehose to create bloom filters. The default is
null
.OrcSerDe.Builder bloomFilterColumns(String... bloomFilterColumns)
The column names for which you want Kinesis Data Firehose to create bloom filters. The default is
null
.
bloomFilterColumns
- The column names for which you want Kinesis Data Firehose to create bloom filters. The default is
null
.OrcSerDe.Builder bloomFilterFalsePositiveProbability(Double bloomFilterFalsePositiveProbability)
The Bloom filter false positive probability (FPP). The lower the FPP, the bigger the Bloom filter. The default value is 0.05, the minimum is 0, and the maximum is 1.
bloomFilterFalsePositiveProbability
- The Bloom filter false positive probability (FPP). The lower the FPP, the bigger the Bloom filter. The
default value is 0.05, the minimum is 0, and the maximum is 1.OrcSerDe.Builder dictionaryKeyThreshold(Double dictionaryKeyThreshold)
Represents the fraction of the total number of non-null rows. To turn off dictionary encoding, set this fraction to a number that is less than the number of distinct keys in a dictionary. To always use dictionary encoding, set this threshold to 1.
dictionaryKeyThreshold
- Represents the fraction of the total number of non-null rows. To turn off dictionary encoding, set
this fraction to a number that is less than the number of distinct keys in a dictionary. To always use
dictionary encoding, set this threshold to 1.OrcSerDe.Builder formatVersion(String formatVersion)
The version of the file to write. The possible values are V0_11
and V0_12
. The
default is V0_12
.
formatVersion
- The version of the file to write. The possible values are V0_11
and V0_12
.
The default is V0_12
.OrcFormatVersion
,
OrcFormatVersion
OrcSerDe.Builder formatVersion(OrcFormatVersion formatVersion)
The version of the file to write. The possible values are V0_11
and V0_12
. The
default is V0_12
.
formatVersion
- The version of the file to write. The possible values are V0_11
and V0_12
.
The default is V0_12
.OrcFormatVersion
,
OrcFormatVersion
Copyright © 2020. All rights reserved.