Package org.apache.parquet.hadoop

Class ParquetOutputCommitter

  • public class ParquetOutputCommitter
extends org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter

    • Field Summary

      • Fields inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter

        FILEOUTPUTCOMMITTER_ALGORITHM_VERSION, FILEOUTPUTCOMMITTER_ALGORITHM_VERSION_DEFAULT, FILEOUTPUTCOMMITTER_CLEANUP_FAILURES_IGNORED, FILEOUTPUTCOMMITTER_CLEANUP_FAILURES_IGNORED_DEFAULT, FILEOUTPUTCOMMITTER_CLEANUP_SKIPPED, FILEOUTPUTCOMMITTER_CLEANUP_SKIPPED_DEFAULT, FILEOUTPUTCOMMITTER_FAILURE_ATTEMPTS, FILEOUTPUTCOMMITTER_FAILURE_ATTEMPTS_DEFAULT, FILEOUTPUTCOMMITTER_TASK_CLEANUP_ENABLED, FILEOUTPUTCOMMITTER_TASK_CLEANUP_ENABLED_DEFAULT, PENDING_DIR_NAME, SUCCEEDED_FILE_NAME, SUCCESSFUL_JOB_OUTPUT_DIR_MARKER, TEMP_DIR_NAME

    • Constructor Summary

      Constructors 
      Constructor Description
      ParquetOutputCommitter​(org.apache.hadoop.fs.Path outputPath, org.apache.hadoop.mapreduce.TaskAttemptContext context)  

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void commitJob​(org.apache.hadoop.mapreduce.JobContext jobContext)  
      static void writeMetaDataFile​(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path outputPath)  

      • Methods inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter

        abortJob, abortTask, abortTask, cleanupJob, commitJobInternal, commitTask, commitTask, getCommittedTaskPath, getCommittedTaskPath, getCommittedTaskPath, getJobAttemptPath, getJobAttemptPath, getJobAttemptPath, getTaskAttemptPath, getTaskAttemptPath, getWorkPath, isCommitJobRepeatable, isRecoverySupported, needsTaskCommit, needsTaskCommit, recoverTask, setupJob, setupTask

      • Methods inherited from class org.apache.hadoop.mapreduce.OutputCommitter

        isRecoverySupported

    • Constructor Detail

      • ParquetOutputCommitter

        public ParquetOutputCommitter​(org.apache.hadoop.fs.Path outputPath,
                              org.apache.hadoop.mapreduce.TaskAttemptContext context)
                       throws IOException
        Throws:
        IOException

    • Method Detail

      • commitJob

        public void commitJob​(org.apache.hadoop.mapreduce.JobContext jobContext)
               throws IOException
        Overrides:
        commitJob in class org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
        Throws:
        IOException

      • writeMetaDataFile

        public static void writeMetaDataFile​(org.apache.hadoop.conf.Configuration configuration,
                                     org.apache.hadoop.fs.Path outputPath)