org.apache.hadoop.hbase.regionserver.compactions

Class Compactor<T extends CellSink>

java.lang.Object

org.apache.hadoop.hbase.regionserver.compactions.Compactor<T>

Direct Known Subclasses:

AbstractMultiOutputCompactor, DefaultCompactor

@InterfaceAudience.Private
public abstract class Compactor<T extends CellSink>
extends Object

A compactor is a compaction algorithm associated a given policy. Base class also contains reusable parts for implementing compactors (what is common and what isn't is evolving).

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static interface	Compactor.CellSinkFactory<S>
protected static class	Compactor.FileDetails The sole reason this class exists is that java has no ref/out/pointer parameters.
protected static interface	Compactor.InternalScannerFactory

Field Summary

Fields

Modifier and Type	Field and Description
protected static long	COMPACTION_PROGRESS_LOG_INTERVAL
protected Compression.Algorithm	compactionCompression
protected int	compactionKVMax
protected org.apache.hadoop.conf.Configuration	conf
protected Compactor.InternalScannerFactory	defaultScannerFactory
protected int	keepSeqIdPeriod specify how many days to keep MVCC values during major compaction
protected static String	MAJOR_COMPACTION_DROP_CACHE
protected static String	MINOR_COMPACTION_DROP_CACHE
protected CompactionProgress	progress
protected HStore	store

Method Summary

Modifier and Type	Method and Description
protected abstract void	abortWriter(T writer)
protected abstract List<org.apache.hadoop.fs.Path>	commitWriter(T writer, Compactor.FileDetails fd, CompactionRequestImpl request)
protected List<org.apache.hadoop.fs.Path>	compact(CompactionRequestImpl request, Compactor.InternalScannerFactory scannerFactory, Compactor.CellSinkFactory<T> sinkFactory, ThroughputController throughputController, User user)
protected InternalScanner	createScanner(HStore store, ScanInfo scanInfo, List<StoreFileScanner> scanners, long smallestReadPoint, long earliestPutTs, byte[] dropDeletesFromRow, byte[] dropDeletesToRow)
protected InternalScanner	createScanner(HStore store, ScanInfo scanInfo, List<StoreFileScanner> scanners, ScanType scanType, long smallestReadPoint, long earliestPutTs)
protected StoreFileWriter	createTmpWriter(Compactor.FileDetails fd, boolean shouldDropBehind) Creates a writer for a new file in a temporary directory.
CompactionProgress	getProgress()
protected boolean	performCompaction(Compactor.FileDetails fd, InternalScanner scanner, CellSink writer, long smallestReadPoint, boolean cleanSeqId, ThroughputController throughputController, boolean major, int numofFilesToCompact) Performs the compaction.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

COMPACTION_PROGRESS_LOG_INTERVAL

protected static final long COMPACTION_PROGRESS_LOG_INTERVAL

See Also:

Constant Field Values

progress

protected volatile CompactionProgress progress

conf

protected final org.apache.hadoop.conf.Configuration conf

store

protected final HStore store

compactionKVMax

protected final int compactionKVMax

compactionCompression

protected final Compression.Algorithm compactionCompression

keepSeqIdPeriod

protected int keepSeqIdPeriod

specify how many days to keep MVCC values during major compaction

MAJOR_COMPACTION_DROP_CACHE

protected static final String MAJOR_COMPACTION_DROP_CACHE

See Also:

Constant Field Values

MINOR_COMPACTION_DROP_CACHE

protected static final String MINOR_COMPACTION_DROP_CACHE

See Also:

Constant Field Values

defaultScannerFactory

protected final Compactor.InternalScannerFactory defaultScannerFactory

Method Detail

getProgress

public CompactionProgress getProgress()

createTmpWriter

protected final StoreFileWriter createTmpWriter(Compactor.FileDetails fd,
                                                boolean shouldDropBehind)
                                         throws IOException

Creates a writer for a new file in a temporary directory.

Parameters:

fd - The file details.

Returns:

Writer for a new StoreFile in the tmp dir.

Throws:

IOException - if creation failed

compact

protected final List<org.apache.hadoop.fs.Path> compact(CompactionRequestImpl request,
                                                        Compactor.InternalScannerFactory scannerFactory,
                                                        Compactor.CellSinkFactory<T> sinkFactory,
                                                        ThroughputController throughputController,
                                                        User user)
                                                 throws IOException

Throws:

IOException

commitWriter

protected abstract List<org.apache.hadoop.fs.Path> commitWriter(T writer,
                                                                Compactor.FileDetails fd,
                                                                CompactionRequestImpl request)
                                                         throws IOException

Throws:

IOException

abortWriter

protected abstract void abortWriter(T writer)
                             throws IOException

Throws:

IOException

performCompaction

protected boolean performCompaction(Compactor.FileDetails fd,
                                    InternalScanner scanner,
                                    CellSink writer,
                                    long smallestReadPoint,
                                    boolean cleanSeqId,
                                    ThroughputController throughputController,
                                    boolean major,
                                    int numofFilesToCompact)
                             throws IOException

Performs the compaction.

Parameters:

fd - FileDetails of cell sink writer

scanner - Where to read from.

writer - Where to write to.

smallestReadPoint - Smallest read point.

cleanSeqId - When true, remove seqId(used to be mvcc) value which is <= smallestReadPoint

major - Is a major compaction.

numofFilesToCompact - the number of files to compact

Returns:

Whether compaction ended; false if it was interrupted for some reason.

Throws:

IOException

createScanner

protected InternalScanner createScanner(HStore store,
                                        ScanInfo scanInfo,
                                        List<StoreFileScanner> scanners,
                                        ScanType scanType,
                                        long smallestReadPoint,
                                        long earliestPutTs)
                                 throws IOException

Parameters:

store - store

scanners - Store file scanners.

scanType - Scan type.

smallestReadPoint - Smallest MVCC read point.

earliestPutTs - Earliest put across all files.

Returns:

A compaction scanner.

Throws:

IOException

createScanner

protected InternalScanner createScanner(HStore store,
                                        ScanInfo scanInfo,
                                        List<StoreFileScanner> scanners,
                                        long smallestReadPoint,
                                        long earliestPutTs,
                                        byte[] dropDeletesFromRow,
                                        byte[] dropDeletesToRow)
                                 throws IOException

Parameters:

store - The store.

scanners - Store file scanners.

smallestReadPoint - Smallest MVCC read point.

earliestPutTs - Earliest put across all files.

dropDeletesFromRow - Drop deletes starting with this row, inclusive. Can be null.

dropDeletesToRow - Drop deletes ending with this row, exclusive. Can be null.

Returns:

A compaction scanner.

Throws:

IOException