Package org.apache.parquet.hadoop
Class CodecFactory
- java.lang.Object
-
- org.apache.parquet.hadoop.CodecFactory
-
- All Implemented Interfaces:
org.apache.parquet.compression.CompressionCodecFactory
public class CodecFactory extends Object implements org.apache.parquet.compression.CompressionCodecFactory
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
CodecFactory.BytesCompressor
Deprecated.will be removed in 2.0.0; use CompressionCodecFactory.BytesInputCompressor instead.static class
CodecFactory.BytesDecompressor
Deprecated.will be removed in 2.0.0; use CompressionCodecFactory.BytesInputDecompressor instead.
-
Field Summary
Fields Modifier and Type Field Description protected static Map<String,org.apache.hadoop.io.compress.CompressionCodec>
CODEC_BY_NAME
protected org.apache.hadoop.conf.Configuration
configuration
protected int
pageSize
-
Constructor Summary
Constructors Constructor Description CodecFactory(org.apache.hadoop.conf.Configuration configuration, int pageSize)
Create a new codec factory.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description protected CodecFactory.BytesCompressor
createCompressor(org.apache.parquet.hadoop.metadata.CompressionCodecName codecName)
protected CodecFactory.BytesDecompressor
createDecompressor(org.apache.parquet.hadoop.metadata.CompressionCodecName codecName)
static CodecFactory
createDirectCodecFactory(org.apache.hadoop.conf.Configuration config, org.apache.parquet.bytes.ByteBufferAllocator allocator, int pageSize)
Create a codec factory that will provide compressors and decompressors that will work natively with ByteBuffers backed by direct memory.protected org.apache.hadoop.io.compress.CompressionCodec
getCodec(org.apache.parquet.hadoop.metadata.CompressionCodecName codecName)
CodecFactory.BytesCompressor
getCompressor(org.apache.parquet.hadoop.metadata.CompressionCodecName codecName)
CodecFactory.BytesDecompressor
getDecompressor(org.apache.parquet.hadoop.metadata.CompressionCodecName codecName)
void
release()
-
-
-
Constructor Detail
-
CodecFactory
public CodecFactory(org.apache.hadoop.conf.Configuration configuration, int pageSize)
Create a new codec factory.- Parameters:
configuration
- used to pass compression codec configuration informationpageSize
- the expected page size, does not set a hard limit, currently just used to set the initial size of the output stream used when compressing a buffer. If this factory is only used to construct decompressors this parameter has no impact on the function of the factory
-
-
Method Detail
-
createDirectCodecFactory
public static CodecFactory createDirectCodecFactory(org.apache.hadoop.conf.Configuration config, org.apache.parquet.bytes.ByteBufferAllocator allocator, int pageSize)
Create a codec factory that will provide compressors and decompressors that will work natively with ByteBuffers backed by direct memory.- Parameters:
config
- configuration options for different compression codecsallocator
- an allocator for creating result buffers during compression and decompression, must provide buffers backed by Direct memory and return true for the isDirect() method on the ByteBufferAllocator interfacepageSize
- the default page size. This does not set a hard limit on the size of buffers that can be compressed, but performance may be improved by setting it close to the expected size of buffers (in the case of parquet, pages) that will be compressed. This setting is unused in the case of decompressing data, as parquet always records the uncompressed size of a buffer. If this CodecFactory is only going to be used for decompressors, this parameter will not impact the function of the factory.- Returns:
- a configured direct codec factory
-
getCompressor
public CodecFactory.BytesCompressor getCompressor(org.apache.parquet.hadoop.metadata.CompressionCodecName codecName)
- Specified by:
getCompressor
in interfaceorg.apache.parquet.compression.CompressionCodecFactory
-
getDecompressor
public CodecFactory.BytesDecompressor getDecompressor(org.apache.parquet.hadoop.metadata.CompressionCodecName codecName)
- Specified by:
getDecompressor
in interfaceorg.apache.parquet.compression.CompressionCodecFactory
-
createCompressor
protected CodecFactory.BytesCompressor createCompressor(org.apache.parquet.hadoop.metadata.CompressionCodecName codecName)
-
createDecompressor
protected CodecFactory.BytesDecompressor createDecompressor(org.apache.parquet.hadoop.metadata.CompressionCodecName codecName)
-
getCodec
protected org.apache.hadoop.io.compress.CompressionCodec getCodec(org.apache.parquet.hadoop.metadata.CompressionCodecName codecName)
- Parameters:
codecName
- the requested codec- Returns:
- the corresponding hadoop codec. null if UNCOMPRESSED
-
release
public void release()
- Specified by:
release
in interfaceorg.apache.parquet.compression.CompressionCodecFactory
-
-