public class AvroParquetOutputFormat<T> extends ParquetOutputFormat<T>
OutputFormat
for Parquet files.ParquetOutputFormat.JobSummaryLevel
BLOCK_SIZE, COMPRESSION, DICTIONARY_PAGE_SIZE, ENABLE_DICTIONARY, ENABLE_JOB_SUMMARY, ESTIMATE_PAGE_SIZE_CHECK, JOB_SUMMARY_LEVEL, MAX_PADDING_BYTES, MAX_ROW_COUNT_FOR_PAGE_SIZE_CHECK, MEMORY_POOL_RATIO, MIN_MEMORY_ALLOCATION, MIN_ROW_COUNT_FOR_PAGE_SIZE_CHECK, PAGE_SIZE, VALIDATION, WRITE_SUPPORT_CLASS, WRITER_VERSION
Constructor and Description |
---|
AvroParquetOutputFormat() |
Modifier and Type | Method and Description |
---|---|
static void |
setAvroDataSupplier(org.apache.hadoop.mapreduce.Job job,
Class<? extends AvroDataSupplier> supplierClass)
Sets the
AvroDataSupplier class that will be used. |
static void |
setSchema(org.apache.hadoop.mapreduce.Job job,
org.apache.avro.Schema schema)
Set the Avro schema to use for writing.
|
getBlockSize, getBlockSize, getCompression, getCompression, getDictionaryPageSize, getDictionaryPageSize, getEnableDictionary, getEnableDictionary, getEstimatePageSizeCheck, getJobSummaryLevel, getLongBlockSize, getMaxRowCountForPageSizeCheck, getMemoryManager, getMinRowCountForPageSizeCheck, getOutputCommitter, getPageSize, getPageSize, getRecordWriter, getRecordWriter, getRecordWriter, getValidation, getValidation, getWriterVersion, getWriteSupport, getWriteSupportClass, isCompressionSet, isCompressionSet, setBlockSize, setCompression, setDictionaryPageSize, setEnableDictionary, setMaxPaddingSize, setMaxPaddingSize, setPageSize, setValidation, setValidation, setWriteSupportClass, setWriteSupportClass
checkOutputSpecs, getCompressOutput, getDefaultWorkFile, getOutputCompressorClass, getOutputName, getOutputPath, getPathForWorkFile, getUniqueFile, getWorkOutputPath, setCompressOutput, setOutputCompressorClass, setOutputName, setOutputPath
public static void setSchema(org.apache.hadoop.mapreduce.Job job, org.apache.avro.Schema schema)
job
- a jobschema
- a schema for the data that will be writtenAvroParquetInputFormat.setAvroReadSchema(org.apache.hadoop.mapreduce.Job, org.apache.avro.Schema)
public static void setAvroDataSupplier(org.apache.hadoop.mapreduce.Job job, Class<? extends AvroDataSupplier> supplierClass)
AvroDataSupplier
class that will be used. The data
supplier provides instances of GenericData
that are used to deconstruct records.job
- a Job
to configuresupplierClass
- a supplier classCopyright © 2019 The Apache Software Foundation. All rights reserved.