Package org.apache.parquet.avro
Class AvroParquetWriter<T>
- java.lang.Object
-
- org.apache.parquet.hadoop.ParquetWriter<T>
-
- org.apache.parquet.avro.AvroParquetWriter<T>
-
- All Implemented Interfaces:
Closeable
,AutoCloseable
public class AvroParquetWriter<T> extends ParquetWriter<T>
Write Avro records to a Parquet file.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
AvroParquetWriter.Builder<T>
-
Field Summary
-
Fields inherited from class org.apache.parquet.hadoop.ParquetWriter
DEFAULT_BLOCK_SIZE, DEFAULT_COMPRESSION_CODEC_NAME, DEFAULT_IS_DICTIONARY_ENABLED, DEFAULT_IS_VALIDATING_ENABLED, DEFAULT_PAGE_SIZE, DEFAULT_WRITER_VERSION, MAX_PADDING_SIZE_DEFAULT, OBJECT_MODEL_NAME_PROP
-
-
Constructor Summary
Constructors Constructor Description AvroParquetWriter(org.apache.hadoop.fs.Path file, org.apache.avro.Schema avroSchema)
Deprecated.AvroParquetWriter(org.apache.hadoop.fs.Path file, org.apache.avro.Schema avroSchema, CompressionCodecName compressionCodecName, int blockSize, int pageSize)
Deprecated.AvroParquetWriter(org.apache.hadoop.fs.Path file, org.apache.avro.Schema avroSchema, CompressionCodecName compressionCodecName, int blockSize, int pageSize, boolean enableDictionary)
Deprecated.AvroParquetWriter(org.apache.hadoop.fs.Path file, org.apache.avro.Schema avroSchema, CompressionCodecName compressionCodecName, int blockSize, int pageSize, boolean enableDictionary, org.apache.hadoop.conf.Configuration conf)
Deprecated.
-
Method Summary
All Methods Static Methods Concrete Methods Deprecated Methods Modifier and Type Method Description static <T> AvroParquetWriter.Builder<T>
builder(org.apache.hadoop.fs.Path file)
Deprecated.will be removed in 2.0.0; usebuilder(OutputFile)
instead.static <T> AvroParquetWriter.Builder<T>
builder(OutputFile file)
-
Methods inherited from class org.apache.parquet.hadoop.ParquetWriter
close, getDataSize, getFooter, write
-
-
-
-
Constructor Detail
-
AvroParquetWriter
@Deprecated public AvroParquetWriter(org.apache.hadoop.fs.Path file, org.apache.avro.Schema avroSchema, CompressionCodecName compressionCodecName, int blockSize, int pageSize) throws IOException
Deprecated.Create a newAvroParquetWriter
.- Parameters:
file
- a file pathavroSchema
- a schema for the writecompressionCodecName
- compression codecblockSize
- target block sizepageSize
- target page size- Throws:
IOException
- if there is an error while writing
-
AvroParquetWriter
@Deprecated public AvroParquetWriter(org.apache.hadoop.fs.Path file, org.apache.avro.Schema avroSchema, CompressionCodecName compressionCodecName, int blockSize, int pageSize, boolean enableDictionary) throws IOException
Deprecated.Create a newAvroParquetWriter
.- Parameters:
file
- The file name to write to.avroSchema
- The schema to write with.compressionCodecName
- Compression code to use, or CompressionCodecName.UNCOMPRESSEDblockSize
- the block size threshold.pageSize
- See parquet write up. Blocks are subdivided into pages for alignment and other purposes.enableDictionary
- Whether to use a dictionary to compress columns.- Throws:
IOException
- if there is an error while writing
-
AvroParquetWriter
@Deprecated public AvroParquetWriter(org.apache.hadoop.fs.Path file, org.apache.avro.Schema avroSchema) throws IOException
Deprecated.Create a newAvroParquetWriter
. The default block size is 50 MB.The default page size is 1 MB. Default compression is no compression. (Inherited fromParquetWriter
)- Parameters:
file
- The file name to write to.avroSchema
- The schema to write with.- Throws:
IOException
- if there is an error while writing
-
AvroParquetWriter
@Deprecated public AvroParquetWriter(org.apache.hadoop.fs.Path file, org.apache.avro.Schema avroSchema, CompressionCodecName compressionCodecName, int blockSize, int pageSize, boolean enableDictionary, org.apache.hadoop.conf.Configuration conf) throws IOException
Deprecated.Create a newAvroParquetWriter
.- Parameters:
file
- The file name to write to.avroSchema
- The schema to write with.compressionCodecName
- Compression code to use, or CompressionCodecName.UNCOMPRESSEDblockSize
- the block size threshold.pageSize
- See parquet write up. Blocks are subdivided into pages for alignment and other purposes.enableDictionary
- Whether to use a dictionary to compress columns.conf
- The Configuration to use.- Throws:
IOException
- if there is an error while writing
-
-
Method Detail
-
builder
@Deprecated public static <T> AvroParquetWriter.Builder<T> builder(org.apache.hadoop.fs.Path file)
Deprecated.will be removed in 2.0.0; usebuilder(OutputFile)
instead.- Type Parameters:
T
- the Java type of records to read from the file- Parameters:
file
- a file path- Returns:
- an Avro reader builder
-
builder
public static <T> AvroParquetWriter.Builder<T> builder(OutputFile file)
-
-