org.apache.parquet.avro

Class AvroParquetWriter<T>

java.lang.Object

org.apache.parquet.hadoop.ParquetWriter<T>

org.apache.parquet.avro.AvroParquetWriter<T>

All Implemented Interfaces:

Closeable, AutoCloseable

public class AvroParquetWriter<T>
extends ParquetWriter<T>

Write Avro records to a Parquet file.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	AvroParquetWriter.Builder<T>

Field Summary

Fields inherited from class org.apache.parquet.hadoop.ParquetWriter

DEFAULT_BLOCK_SIZE, DEFAULT_COMPRESSION_CODEC_NAME, DEFAULT_IS_DICTIONARY_ENABLED, DEFAULT_IS_VALIDATING_ENABLED, DEFAULT_PAGE_SIZE, DEFAULT_WRITER_VERSION, MAX_PADDING_SIZE_DEFAULT, OBJECT_MODEL_NAME_PROP

Constructor Summary

Constructors

Constructor and Description
AvroParquetWriter(org.apache.hadoop.fs.Path file, org.apache.avro.Schema avroSchema) Deprecated.
AvroParquetWriter(org.apache.hadoop.fs.Path file, org.apache.avro.Schema avroSchema, CompressionCodecName compressionCodecName, int blockSize, int pageSize) Deprecated.
AvroParquetWriter(org.apache.hadoop.fs.Path file, org.apache.avro.Schema avroSchema, CompressionCodecName compressionCodecName, int blockSize, int pageSize, boolean enableDictionary) Deprecated.
AvroParquetWriter(org.apache.hadoop.fs.Path file, org.apache.avro.Schema avroSchema, CompressionCodecName compressionCodecName, int blockSize, int pageSize, boolean enableDictionary, org.apache.hadoop.conf.Configuration conf) Deprecated.

Method Summary

Modifier and Type	Method and Description
static <T> AvroParquetWriter.Builder<T>	builder(OutputFile file)
static <T> AvroParquetWriter.Builder<T>	builder(org.apache.hadoop.fs.Path file)

Methods inherited from class org.apache.parquet.hadoop.ParquetWriter

close, getDataSize, getFooter, write

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AvroParquetWriter

@Deprecated
public AvroParquetWriter(org.apache.hadoop.fs.Path file,
                                     org.apache.avro.Schema avroSchema,
                                     CompressionCodecName compressionCodecName,
                                     int blockSize,
                                     int pageSize)
                              throws IOException

Deprecated.

Create a new AvroParquetWriter.

Parameters:

file - a file path

avroSchema - a schema for the write

compressionCodecName - compression codec

blockSize - target block size

pageSize - target page size

Throws:

IOException - if there is an error while writing

AvroParquetWriter

@Deprecated
public AvroParquetWriter(org.apache.hadoop.fs.Path file,
                                     org.apache.avro.Schema avroSchema,
                                     CompressionCodecName compressionCodecName,
                                     int blockSize,
                                     int pageSize,
                                     boolean enableDictionary)
                              throws IOException

Deprecated.

Create a new AvroParquetWriter.

Parameters:

file - The file name to write to.

avroSchema - The schema to write with.

compressionCodecName - Compression code to use, or CompressionCodecName.UNCOMPRESSED

blockSize - the block size threshold.

pageSize - See parquet write up. Blocks are subdivided into pages for alignment and other purposes.

enableDictionary - Whether to use a dictionary to compress columns.

Throws:

IOException - if there is an error while writing

AvroParquetWriter

@Deprecated
public AvroParquetWriter(org.apache.hadoop.fs.Path file,
                                     org.apache.avro.Schema avroSchema)
                              throws IOException

Deprecated.

Create a new AvroParquetWriter. The default block size is 50 MB.The default page size is 1 MB. Default compression is no compression. (Inherited from ParquetWriter)

Parameters:

file - The file name to write to.

avroSchema - The schema to write with.

Throws:

IOException - if there is an error while writing

AvroParquetWriter

@Deprecated
public AvroParquetWriter(org.apache.hadoop.fs.Path file,
                                     org.apache.avro.Schema avroSchema,
                                     CompressionCodecName compressionCodecName,
                                     int blockSize,
                                     int pageSize,
                                     boolean enableDictionary,
                                     org.apache.hadoop.conf.Configuration conf)
                              throws IOException

Deprecated.

Create a new AvroParquetWriter.

Parameters:

file - The file name to write to.

avroSchema - The schema to write with.

compressionCodecName - Compression code to use, or CompressionCodecName.UNCOMPRESSED

blockSize - the block size threshold.

pageSize - See parquet write up. Blocks are subdivided into pages for alignment and other purposes.

enableDictionary - Whether to use a dictionary to compress columns.

conf - The Configuration to use.

Throws:

IOException - if there is an error while writing

Method Detail

builder

public static <T> AvroParquetWriter.Builder<T> builder(org.apache.hadoop.fs.Path file)

builder

public static <T> AvroParquetWriter.Builder<T> builder(OutputFile file)