public class ValidCompactorWriteIdList extends ValidReaderWriteIdList
ValidWriteIdList
for use by the compactor.
Compaction should only include txns up to smallest open txn (exclussive).
There may be aborted write ids in the snapshot represented by this ValidCompactorWriteIdList.
Thus isWriteIdRangeValid(long, long)
returns NONE for any range that includes any unresolved
write ids. Any write id above highWatermark
is unresolved.
These produce the logic we need to assure that the compactor only sees records less than the lowest
open write ids when choosing which files to compact, but that it still ignores aborted
records when compacting.
See org.apache.hadoop.hive.metastore.txn.TxnUtils#createValidCompactTxnList() for proper
way to construct this.ValidWriteIdList.RangeResponse
abortedBits, exceptions, highWatermark
VALID_WRITEIDS_KEY
Constructor and Description |
---|
ValidCompactorWriteIdList() |
ValidCompactorWriteIdList(String value) |
ValidCompactorWriteIdList(String tableName,
long[] abortedWriteIdList,
BitSet abortedBits,
long highWatermark) |
ValidCompactorWriteIdList(String tableName,
long[] abortedWriteIdList,
BitSet abortedBits,
long highWatermark,
long minOpenWriteId) |
Modifier and Type | Method and Description |
---|---|
boolean |
isWriteIdAborted(long writeId)
Indicates whether a given write maps to aborted transaction.
|
ValidWriteIdList.RangeResponse |
isWriteIdRangeValid(long minWriteId,
long maxWriteId)
Returns org.apache.hadoop.hive.common.ValidWriteIdList.RangeResponse.ALL if all write ids in
the range are resolved and RangeResponse.NONE otherwise
Streaming ingest may create something like delta_11_20.
|
getHighWatermark, getInvalidWriteIds, getMinOpenWriteId, getTableName, isValidBase, isWriteIdRangeAborted, isWriteIdValid, readFromString, toString, updateHighWatermark, writeToString
public ValidCompactorWriteIdList()
public ValidCompactorWriteIdList(String tableName, long[] abortedWriteIdList, BitSet abortedBits, long highWatermark)
public ValidCompactorWriteIdList(String tableName, long[] abortedWriteIdList, BitSet abortedBits, long highWatermark, long minOpenWriteId)
tableName
- table which is under compaction. Full name of format <db_name>.<table_name>abortedWriteIdList
- list of all aborted write idsabortedBits
- bitset marking whether the corresponding write id is abortedhighWatermark
- highest committed write id to be considered for compaction,
equivalently (lowest_open_write_id - 1).minOpenWriteId
- minimum write ID which maps to a open transactionpublic ValidCompactorWriteIdList(String value)
public ValidWriteIdList.RangeResponse isWriteIdRangeValid(long minWriteId, long maxWriteId)
isWriteIdRangeValid
in interface ValidWriteIdList
isWriteIdRangeValid
in class ValidReaderWriteIdList
minWriteId
- minimum write ID to look for, inclusivemaxWriteId
- maximum write ID to look for, inclusivepublic boolean isWriteIdAborted(long writeId)
ValidWriteIdList
isWriteIdAborted
in interface ValidWriteIdList
isWriteIdAborted
in class ValidReaderWriteIdList
writeId
- write id to be validatedCopyright © 2021 The Apache Software Foundation. All rights reserved.