Package org.apache.parquet.hadoop.codec
Class NonBlockedCompressor
- java.lang.Object
-
- org.apache.parquet.hadoop.codec.NonBlockedCompressor
-
- All Implemented Interfaces:
org.apache.hadoop.io.compress.Compressor
- Direct Known Subclasses:
Lz4RawCompressor
,SnappyCompressor
public abstract class NonBlockedCompressor extends Object implements org.apache.hadoop.io.compress.Compressor
This class is a wrapper around the underlying compressor. It always consumes the entire input in setInput and compresses it as one compressed block.
-
-
Constructor Summary
Constructors Constructor Description NonBlockedCompressor()
-
Method Summary
All Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description int
compress(byte[] buffer, int off, int len)
Fills specified buffer with compressed data.protected abstract int
compress(ByteBuffer uncompressed, ByteBuffer compressed)
Compress the content in the given input buffer.void
end()
void
finish()
boolean
finished()
long
getBytesRead()
long
getBytesWritten()
protected abstract int
maxCompressedLength(int byteSize)
Get the maximum byte size needed for compressing data of the given byte size.boolean
needsInput()
void
reinit(org.apache.hadoop.conf.Configuration c)
void
reset()
void
setDictionary(byte[] dictionary, int off, int len)
void
setInput(byte[] buffer, int off, int len)
-
-
-
Method Detail
-
compress
public int compress(byte[] buffer, int off, int len) throws IOException
Fills specified buffer with compressed data. Returns actual number of bytes of compressed data. A return value of 0 indicates that needsInput() should be called in order to determine if more input data is required.- Specified by:
compress
in interfaceorg.apache.hadoop.io.compress.Compressor
- Parameters:
buffer
- Buffer for the compressed dataoff
- Start offset of the datalen
- Size of the buffer- Returns:
- The actual number of bytes of compressed data.
- Throws:
IOException
-
setInput
public void setInput(byte[] buffer, int off, int len)
- Specified by:
setInput
in interfaceorg.apache.hadoop.io.compress.Compressor
-
end
public void end()
- Specified by:
end
in interfaceorg.apache.hadoop.io.compress.Compressor
-
finish
public void finish()
- Specified by:
finish
in interfaceorg.apache.hadoop.io.compress.Compressor
-
finished
public boolean finished()
- Specified by:
finished
in interfaceorg.apache.hadoop.io.compress.Compressor
-
getBytesRead
public long getBytesRead()
- Specified by:
getBytesRead
in interfaceorg.apache.hadoop.io.compress.Compressor
-
getBytesWritten
public long getBytesWritten()
- Specified by:
getBytesWritten
in interfaceorg.apache.hadoop.io.compress.Compressor
-
needsInput
public boolean needsInput()
- Specified by:
needsInput
in interfaceorg.apache.hadoop.io.compress.Compressor
-
reinit
public void reinit(org.apache.hadoop.conf.Configuration c)
- Specified by:
reinit
in interfaceorg.apache.hadoop.io.compress.Compressor
-
reset
public void reset()
- Specified by:
reset
in interfaceorg.apache.hadoop.io.compress.Compressor
-
setDictionary
public void setDictionary(byte[] dictionary, int off, int len)
- Specified by:
setDictionary
in interfaceorg.apache.hadoop.io.compress.Compressor
-
maxCompressedLength
protected abstract int maxCompressedLength(int byteSize)
Get the maximum byte size needed for compressing data of the given byte size.- Parameters:
byteSize
- byte size of the data to compress- Returns:
- maximum byte size of the compressed data
-
compress
protected abstract int compress(ByteBuffer uncompressed, ByteBuffer compressed) throws IOException
Compress the content in the given input buffer. After the compression, you can retrieve the compressed data from the output buffer [pos() ... limit()) (compressed data size = limit() - pos() = remaining())- Parameters:
uncompressed
- buffer[pos() ... limit()) containing the input datacompressed
- output of the compressed data. Uses range [pos()..].- Returns:
- byte size of the compressed data.
- Throws:
IOException
-
-