org.apache.parquet.column

Class ParquetProperties.Builder

java.lang.Object

org.apache.parquet.column.ParquetProperties.Builder

Enclosing class:

ParquetProperties

public static class ParquetProperties.Builder
extends Object

Method Summary

Modifier and Type	Method and Description
ParquetProperties	build()
ParquetProperties.Builder	estimateRowCountForPageSizeCheck(boolean estimateNextSizeCheck)
ParquetProperties.Builder	withAdaptiveBloomFilterEnabled(boolean enabled) Whether to use adaptive bloom filter to automatically adjust the bloom filter size according to `parquet.bloom.filter.max.bytes`.
ParquetProperties.Builder	withAllocator(ByteBufferAllocator allocator)
ParquetProperties.Builder	withBloomFilterCandidatesNumber(String columnPath, int number) When `AdaptiveBloomFilter` is enabled, set how many bloom filter candidates to use.
ParquetProperties.Builder	withBloomFilterEnabled(boolean enabled) Enable or disable the bloom filter for the columns not specified by withBloomFilterEnabled(String, boolean).
ParquetProperties.Builder	withBloomFilterEnabled(String columnPath, boolean enabled) Enable or disable the bloom filter for the specified column.
ParquetProperties.Builder	withBloomFilterFPP(String columnPath, double fpp)
ParquetProperties.Builder	withBloomFilterNDV(String columnPath, long ndv) Set Bloom filter NDV (number of distinct values) for the specified column.
ParquetProperties.Builder	withByteStreamSplitEncoding(boolean enable) Enable or disable BYTE_STREAM_SPLIT encoding for FLOAT and DOUBLE columns.
ParquetProperties.Builder	withByteStreamSplitEncoding(String columnPath, boolean enable) Enable or disable BYTE_STREAM_SPLIT encoding for specified columns.
ParquetProperties.Builder	withColumnIndexTruncateLength(int length)
ParquetProperties.Builder	withDictionaryEncoding(boolean enableDictionary) Enable or disable dictionary encoding.
ParquetProperties.Builder	withDictionaryEncoding(String columnPath, boolean enableDictionary) Enable or disable dictionary encoding for the specified column.
ParquetProperties.Builder	withDictionaryPageSize(int dictionaryPageSize) Set the Parquet format dictionary page size.
ParquetProperties.Builder	withExtendedByteStreamSplitEncoding(boolean enable) Enable or disable BYTE_STREAM_SPLIT encoding for FLOAT, DOUBLE, INT32, INT64 and FIXED_LEN_BYTE_ARRAY columns.
ParquetProperties.Builder	withExtraMetaData(Map<String,String> extraMetaData)
ParquetProperties.Builder	withMaxBloomFilterBytes(int maxBloomFilterBytes) Set max Bloom filter bytes for related columns.
ParquetProperties.Builder	withMaxRowCountForPageSizeCheck(int max)
ParquetProperties.Builder	withMinRowCountForPageSizeCheck(int min)
ParquetProperties.Builder	withPageRowCountLimit(int rowCount)
ParquetProperties.Builder	withPageSize(int pageSize) Set the Parquet format page size.
ParquetProperties.Builder	withPageValueCountThreshold(int value)
ParquetProperties.Builder	withPageWriteChecksumEnabled(boolean val)
ParquetProperties.Builder	withStatisticsTruncateLength(int length)
ParquetProperties.Builder	withValuesWriterFactory(ValuesWriterFactory factory)
ParquetProperties.Builder	withWriterVersion(ParquetProperties.WriterVersion version) Set the format version.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

withPageSize

public ParquetProperties.Builder withPageSize(int pageSize)

Set the Parquet format page size.

Parameters:

pageSize - an integer size in bytes

Returns:

this builder for method chaining.

withDictionaryEncoding

public ParquetProperties.Builder withDictionaryEncoding(boolean enableDictionary)

Enable or disable dictionary encoding.

Parameters:

enableDictionary - whether dictionary encoding should be enabled

Returns:

this builder for method chaining.

withDictionaryEncoding

public ParquetProperties.Builder withDictionaryEncoding(String columnPath,
                                                        boolean enableDictionary)

Enable or disable dictionary encoding for the specified column.

Parameters:

columnPath - the path of the column (dot-string)

enableDictionary - whether dictionary encoding should be enabled

Returns:

this builder for method chaining.

withByteStreamSplitEncoding

public ParquetProperties.Builder withByteStreamSplitEncoding(boolean enable)

Enable or disable BYTE_STREAM_SPLIT encoding for FLOAT and DOUBLE columns.

Parameters:

enable - whether BYTE_STREAM_SPLIT encoding should be enabled

Returns:

this builder for method chaining.

withByteStreamSplitEncoding

public ParquetProperties.Builder withByteStreamSplitEncoding(String columnPath,
                                                             boolean enable)

Enable or disable BYTE_STREAM_SPLIT encoding for specified columns.

Parameters:

columnPath - the path of the column (dot-string)

enable - whether BYTE_STREAM_SPLIT encoding should be enabled

Returns:

this builder for method chaining.

withExtendedByteStreamSplitEncoding

public ParquetProperties.Builder withExtendedByteStreamSplitEncoding(boolean enable)

Enable or disable BYTE_STREAM_SPLIT encoding for FLOAT, DOUBLE, INT32, INT64 and FIXED_LEN_BYTE_ARRAY columns.

Parameters:

enable - whether BYTE_STREAM_SPLIT encoding should be enabled

Returns:

this builder for method chaining.

withDictionaryPageSize

public ParquetProperties.Builder withDictionaryPageSize(int dictionaryPageSize)

Set the Parquet format dictionary page size.

Parameters:

dictionaryPageSize - an integer size in bytes

Returns:

this builder for method chaining.

withWriterVersion

public ParquetProperties.Builder withWriterVersion(ParquetProperties.WriterVersion version)

Set the format version.

Parameters:

version - a WriterVersion

Returns:

this builder for method chaining.

withMinRowCountForPageSizeCheck

public ParquetProperties.Builder withMinRowCountForPageSizeCheck(int min)

withMaxRowCountForPageSizeCheck

public ParquetProperties.Builder withMaxRowCountForPageSizeCheck(int max)

withPageValueCountThreshold

public ParquetProperties.Builder withPageValueCountThreshold(int value)

estimateRowCountForPageSizeCheck

public ParquetProperties.Builder estimateRowCountForPageSizeCheck(boolean estimateNextSizeCheck)

withAllocator

public ParquetProperties.Builder withAllocator(ByteBufferAllocator allocator)

withValuesWriterFactory

public ParquetProperties.Builder withValuesWriterFactory(ValuesWriterFactory factory)

withColumnIndexTruncateLength

public ParquetProperties.Builder withColumnIndexTruncateLength(int length)

withStatisticsTruncateLength

public ParquetProperties.Builder withStatisticsTruncateLength(int length)

withMaxBloomFilterBytes

public ParquetProperties.Builder withMaxBloomFilterBytes(int maxBloomFilterBytes)

Set max Bloom filter bytes for related columns.

Parameters:

maxBloomFilterBytes - the max bytes of a Bloom filter bitset for a column.

Returns:

this builder for method chaining

withBloomFilterNDV

public ParquetProperties.Builder withBloomFilterNDV(String columnPath,
                                                    long ndv)

Set Bloom filter NDV (number of distinct values) for the specified column. If set for a column then the writing of the bloom filter for that column will be automatically enabled (see withBloomFilterEnabled(String, boolean)).

Parameters:

columnPath - the path of the column (dot-string)

ndv - the NDV of the column

Returns:

this builder for method chaining

withBloomFilterFPP

public ParquetProperties.Builder withBloomFilterFPP(String columnPath,
                                                    double fpp)

withBloomFilterEnabled

public ParquetProperties.Builder withBloomFilterEnabled(boolean enabled)

Enable or disable the bloom filter for the columns not specified by withBloomFilterEnabled(String, boolean).

Parameters:

enabled - whether bloom filter shall be enabled for all columns

Returns:

this builder for method chaining

withAdaptiveBloomFilterEnabled

public ParquetProperties.Builder withAdaptiveBloomFilterEnabled(boolean enabled)

Whether to use adaptive bloom filter to automatically adjust the bloom filter size according to `parquet.bloom.filter.max.bytes`. If NDV (number of distinct values) for a specified column is set, it will be ignored

Parameters:

enabled - whether to use adaptive bloom filter

withBloomFilterCandidatesNumber

public ParquetProperties.Builder withBloomFilterCandidatesNumber(String columnPath,
                                                                 int number)

When `AdaptiveBloomFilter` is enabled, set how many bloom filter candidates to use.

Parameters:

columnPath - the path of the column (dot-string)

number - the number of candidates

withBloomFilterEnabled

public ParquetProperties.Builder withBloomFilterEnabled(String columnPath,
                                                        boolean enabled)

Enable or disable the bloom filter for the specified column. One may either disable bloom filters for all columns by invoking withBloomFilterEnabled(boolean) with a false value and then enable the bloom filters for the required columns one-by-one by invoking this method or vice versa.

Parameters:

columnPath - the path of the column (dot-string)

enabled - whether bloom filter shall be enabled

Returns:

this builder for method chaining

withPageRowCountLimit

public ParquetProperties.Builder withPageRowCountLimit(int rowCount)

withPageWriteChecksumEnabled

public ParquetProperties.Builder withPageWriteChecksumEnabled(boolean val)

withExtraMetaData

public ParquetProperties.Builder withExtraMetaData(Map<String,String> extraMetaData)

build

public ParquetProperties build()