org.apache.cassandra.index.sasi.disk

Class StaticTokenTreeBuilder

java.lang.Object

org.apache.cassandra.index.sasi.disk.AbstractTokenTreeBuilder

org.apache.cassandra.index.sasi.disk.StaticTokenTreeBuilder

All Implemented Interfaces:

java.lang.Iterable<Pair<java.lang.Long,com.carrotsearch.hppc.LongSet>>, TokenTreeBuilder

public class StaticTokenTreeBuilder
extends AbstractTokenTreeBuilder

Intended usage of this class is to be used in place of DynamicTokenTreeBuilder when multiple index segments produced by PerSSTableIndexWriter are stitched together by PerSSTableIndexWriter.complete(). This class uses the RangeIterator, now provided by CombinedTerm.getTokenIterator(), to iterate the data twice. The first iteration builds the tree with leaves that contain only enough information to build the upper layers -- these leaves do not store more than their minimum and maximum tokens plus their total size, which makes them un-serializable. When the tree is written to disk the final layer is not written. Its at this point the data is iterated once again to write the leaves to disk. This (logarithmically) reduces copying of the token values while building and writing upper layers of the tree, removes the use of SortedMap when combining SAs, and relies on the memory mapped SAs otherwise, greatly improving performance and no longer causing OOMs when TokenTree sizes are big. See https://issues.apache.org/jira/browse/CASSANDRA-11383 for more details.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.cassandra.index.sasi.disk.AbstractTokenTreeBuilder

AbstractTokenTreeBuilder.InteriorNode, AbstractTokenTreeBuilder.Leaf, AbstractTokenTreeBuilder.LevelIterator, AbstractTokenTreeBuilder.Node

Nested classes/interfaces inherited from interface org.apache.cassandra.index.sasi.disk.TokenTreeBuilder

TokenTreeBuilder.EntryType

Field Summary

Fields inherited from class org.apache.cassandra.index.sasi.disk.AbstractTokenTreeBuilder

leftmostLeaf, numBlocks, rightmostLeaf, rightmostParent, root, tokenCount, treeMaxToken, treeMinToken

Fields inherited from interface org.apache.cassandra.index.sasi.disk.TokenTreeBuilder

AB_MAGIC, BLOCK_BYTES, BLOCK_ENTRY_BYTES, BLOCK_HEADER_BYTES, ENTRY_TYPE_MASK, LAST_LEAF_SHIFT, MAX_OFFSET, OVERFLOW_ENTRY_BYTES, OVERFLOW_TRAILER_BYTES, OVERFLOW_TRAILER_CAPACITY, SHARED_HEADER_BYTES, TOKENS_PER_BLOCK

Constructor Summary

Constructors

Constructor and Description
StaticTokenTreeBuilder(CombinedTerm term)

Method Summary

Modifier and Type	Method and Description
void	add(java.util.Iterator<Pair<java.lang.Long,com.carrotsearch.hppc.LongSet>> data)
void	add(java.lang.Long token, long keyPosition)
void	add(java.util.SortedMap<java.lang.Long,com.carrotsearch.hppc.LongSet> data)
protected void	constructTree()
long	getTokenCount()
boolean	isEmpty()
java.util.Iterator<Pair<java.lang.Long,com.carrotsearch.hppc.LongSet>>	iterator()
void	write(DataOutputPlus out)

Methods inherited from class org.apache.cassandra.index.sasi.disk.AbstractTokenTreeBuilder

add, alignBuffer, finish, flushBuffer, serializedSize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Constructor Detail

StaticTokenTreeBuilder

public StaticTokenTreeBuilder(CombinedTerm term)

Method Detail

add

public void add(java.lang.Long token,
                long keyPosition)

add

public void add(java.util.SortedMap<java.lang.Long,com.carrotsearch.hppc.LongSet> data)

add

public void add(java.util.Iterator<Pair<java.lang.Long,com.carrotsearch.hppc.LongSet>> data)

isEmpty

public boolean isEmpty()

iterator

public java.util.Iterator<Pair<java.lang.Long,com.carrotsearch.hppc.LongSet>> iterator()

getTokenCount

public long getTokenCount()

Specified by:

getTokenCount in interface TokenTreeBuilder

Overrides:

getTokenCount in class AbstractTokenTreeBuilder

write

public void write(DataOutputPlus out)
           throws java.io.IOException

Specified by:

write in interface TokenTreeBuilder

Overrides:

write in class AbstractTokenTreeBuilder

Throws:

java.io.IOException

constructTree

protected void constructTree()

Specified by:

constructTree in class AbstractTokenTreeBuilder