Class ExampleOutputFormat
- java.lang.Object
-
- org.apache.hadoop.mapreduce.OutputFormat<K,V>
-
- org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<Void,T>
-
- org.apache.parquet.hadoop.ParquetOutputFormat<Group>
-
- org.apache.parquet.hadoop.example.ExampleOutputFormat
-
public class ExampleOutputFormat extends ParquetOutputFormat<Group>
An example output format must be provided the schema up front
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.parquet.hadoop.ParquetOutputFormat
ParquetOutputFormat.JobSummaryLevel
-
-
Field Summary
-
Fields inherited from class org.apache.parquet.hadoop.ParquetOutputFormat
BLOCK_SIZE, BLOOM_FILTER_ENABLED, BLOOM_FILTER_EXPECTED_NDV, BLOOM_FILTER_MAX_BYTES, COLUMN_INDEX_TRUNCATE_LENGTH, COMPRESSION, DICTIONARY_PAGE_SIZE, ENABLE_DICTIONARY, ENABLE_JOB_SUMMARY, ESTIMATE_PAGE_SIZE_CHECK, JOB_SUMMARY_LEVEL, MAX_PADDING_BYTES, MAX_ROW_COUNT_FOR_PAGE_SIZE_CHECK, MEMORY_POOL_RATIO, MIN_MEMORY_ALLOCATION, MIN_ROW_COUNT_FOR_PAGE_SIZE_CHECK, PAGE_ROW_COUNT_LIMIT, PAGE_SIZE, PAGE_WRITE_CHECKSUM_ENABLED, STATISTICS_TRUNCATE_LENGTH, VALIDATION, WRITE_SUPPORT_CLASS, WRITER_VERSION
-
-
Constructor Summary
Constructors Constructor Description ExampleOutputFormat()
-
Method Summary
All Methods Static Methods Concrete Methods Modifier and Type Method Description static MessageType
getSchema(org.apache.hadoop.mapreduce.Job job)
retrieve the schema from the confstatic void
setSchema(org.apache.hadoop.mapreduce.Job job, MessageType schema)
set the schema being written to the job conf-
Methods inherited from class org.apache.parquet.hadoop.ParquetOutputFormat
createEncryptionProperties, getBlockSize, getBlockSize, getBloomFilterEnabled, getBloomFilterMaxBytes, getCompression, getCompression, getDictionaryPageSize, getDictionaryPageSize, getEnableDictionary, getEnableDictionary, getEstimatePageSizeCheck, getJobSummaryLevel, getLongBlockSize, getMaxRowCountForPageSizeCheck, getMemoryManager, getMinRowCountForPageSizeCheck, getOutputCommitter, getPageSize, getPageSize, getPageWriteChecksumEnabled, getRecordWriter, getRecordWriter, getRecordWriter, getRecordWriter, getRecordWriter, getRecordWriter, getValidation, getValidation, getWriterVersion, getWriteSupport, getWriteSupportClass, isCompressionSet, isCompressionSet, setBlockSize, setColumnIndexTruncateLength, setColumnIndexTruncateLength, setCompression, setDictionaryPageSize, setEnableDictionary, setMaxPaddingSize, setMaxPaddingSize, setPageRowCountLimit, setPageRowCountLimit, setPageSize, setPageWriteChecksumEnabled, setPageWriteChecksumEnabled, setStatisticsTruncateLength, setValidation, setValidation, setWriteSupportClass, setWriteSupportClass
-
Methods inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat
checkOutputSpecs, getCompressOutput, getDefaultWorkFile, getOutputCompressorClass, getOutputName, getOutputPath, getPathForWorkFile, getUniqueFile, getWorkOutputPath, setCompressOutput, setOutputCompressorClass, setOutputName, setOutputPath
-
-
-
-
Method Detail
-
setSchema
public static void setSchema(org.apache.hadoop.mapreduce.Job job, MessageType schema)
set the schema being written to the job conf- Parameters:
job
- a jobschema
- the schema of the data
-
getSchema
public static MessageType getSchema(org.apache.hadoop.mapreduce.Job job)
retrieve the schema from the conf- Parameters:
job
- a job- Returns:
- the schema
-
-