org.apache.parquet.avro

Class AvroParquetWriter<T>

java.lang.Object

org.apache.parquet.hadoop.ParquetWriter<T>

org.apache.parquet.avro.AvroParquetWriter<T>

All Implemented Interfaces:

Closeable, AutoCloseable

public class AvroParquetWriter<T>
extends ParquetWriter<T>

Write Avro records to a Parquet file.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	AvroParquetWriter.Builder<T>

Field Summary

Fields inherited from class org.apache.parquet.hadoop.ParquetWriter

DEFAULT_BLOCK_SIZE, DEFAULT_COMPRESSION_CODEC_NAME, DEFAULT_IS_DICTIONARY_ENABLED, DEFAULT_IS_VALIDATING_ENABLED, DEFAULT_PAGE_SIZE, DEFAULT_WRITER_VERSION, MAX_PADDING_SIZE_DEFAULT

Constructor Summary

Constructors

Constructor and Description
AvroParquetWriter(org.apache.hadoop.fs.Path file, org.apache.avro.Schema avroSchema) Deprecated.
AvroParquetWriter(org.apache.hadoop.fs.Path file, org.apache.avro.Schema avroSchema, CompressionCodecName compressionCodecName, int blockSize, int pageSize) Deprecated.
AvroParquetWriter(org.apache.hadoop.fs.Path file, org.apache.avro.Schema avroSchema, CompressionCodecName compressionCodecName, int blockSize, int pageSize, boolean enableDictionary) Deprecated.
AvroParquetWriter(org.apache.hadoop.fs.Path file, org.apache.avro.Schema avroSchema, CompressionCodecName compressionCodecName, int blockSize, int pageSize, boolean enableDictionary, org.apache.hadoop.conf.Configuration conf) Deprecated.

Method Summary

Methods

Modifier and Type	Method and Description
static <T> AvroParquetWriter.Builder<T>	builder(org.apache.hadoop.fs.Path file)

Methods inherited from class org.apache.parquet.hadoop.ParquetWriter

close, getDataSize, write

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AvroParquetWriter

@Deprecated
public AvroParquetWriter(org.apache.hadoop.fs.Path file,
                            org.apache.avro.Schema avroSchema,
                            CompressionCodecName compressionCodecName,
                            int blockSize,
                            int pageSize)
                  throws IOException

Deprecated.

Create a new AvroParquetWriter.

Parameters:

file -

avroSchema -

compressionCodecName -

blockSize -

pageSize -

Throws:

IOException

AvroParquetWriter

@Deprecated
public AvroParquetWriter(org.apache.hadoop.fs.Path file,
                            org.apache.avro.Schema avroSchema,
                            CompressionCodecName compressionCodecName,
                            int blockSize,
                            int pageSize,
                            boolean enableDictionary)
                  throws IOException

Deprecated.

Create a new AvroParquetWriter.

Parameters:

file - The file name to write to.

avroSchema - The schema to write with.

compressionCodecName - Compression code to use, or CompressionCodecName.UNCOMPRESSED

blockSize - the block size threshold.

pageSize - See parquet write up. Blocks are subdivided into pages for alignment and other purposes.

enableDictionary - Whether to use a dictionary to compress columns.

Throws:

IOException

AvroParquetWriter

@Deprecated
public AvroParquetWriter(org.apache.hadoop.fs.Path file,
                            org.apache.avro.Schema avroSchema)
                  throws IOException

Deprecated.

Create a new AvroParquetWriter. The default block size is 50 MB.The default page size is 1 MB. Default compression is no compression. (Inherited from ParquetWriter)

Parameters:

file - The file name to write to.

avroSchema - The schema to write with.

Throws:

IOException

AvroParquetWriter

@Deprecated
public AvroParquetWriter(org.apache.hadoop.fs.Path file,
                            org.apache.avro.Schema avroSchema,
                            CompressionCodecName compressionCodecName,
                            int blockSize,
                            int pageSize,
                            boolean enableDictionary,
                            org.apache.hadoop.conf.Configuration conf)
                  throws IOException

Deprecated.

Create a new AvroParquetWriter.

Parameters:

file - The file name to write to.

avroSchema - The schema to write with.

compressionCodecName - Compression code to use, or CompressionCodecName.UNCOMPRESSED

blockSize - the block size threshold.

pageSize - See parquet write up. Blocks are subdivided into pages for alignment and other purposes.

enableDictionary - Whether to use a dictionary to compress columns.

conf - The Configuration to use.

Throws:

IOException

Method Detail

builder

public static <T> AvroParquetWriter.Builder<T> builder(org.apache.hadoop.fs.Path file)