Package org.apache.druid.segment.index.semantic

Interface ValueSetIndexes

All Known Implementing Classes:

IndexedUtf8ValueIndexes

public interface ValueSetIndexes

Construct a BitmapColumnIndex for a set of values which might be present in the column.

Nested Class Summary

Nested Classes

Modifier and Type	Interface	Description
static class	ValueSetIndexes.BaseValueSetIndexesFromIterable

Field Summary

Fields

Modifier and Type	Field	Description
static int	SIZE_WORTH_CHECKING_MIN	minimum sorted match value iterator size to trim the initial values from the iterator to seek to the start of the value dictionary when using buildBitmapColumnIndexFromSortedIteratorScan(org.apache.druid.collections.bitmap.BitmapFactory, java.util.Comparator<T>, java.lang.Iterable<T>, int, org.apache.druid.segment.data.Indexed<T>, org.apache.druid.segment.data.Indexed<org.apache.druid.collections.bitmap.ImmutableBitmap>, com.google.common.base.Supplier<org.apache.druid.collections.bitmap.ImmutableBitmap>) or buildBitmapColumnIndexFromSortedIteratorBinarySearch(org.apache.druid.collections.bitmap.BitmapFactory, java.lang.Iterable<T>, int, org.apache.druid.segment.data.Indexed<T>, org.apache.druid.segment.data.Indexed<org.apache.druid.collections.bitmap.ImmutableBitmap>, com.google.common.base.Supplier<org.apache.druid.collections.bitmap.ImmutableBitmap>).
static double	SORTED_SCAN_RATIO_THRESHOLD	threshold of sorted match value iterator size compared to dictionary size to use buildBitmapColumnIndexFromSortedIteratorScan(org.apache.druid.collections.bitmap.BitmapFactory, java.util.Comparator<T>, java.lang.Iterable<T>, int, org.apache.druid.segment.data.Indexed<T>, org.apache.druid.segment.data.Indexed<org.apache.druid.collections.bitmap.ImmutableBitmap>, com.google.common.base.Supplier<org.apache.druid.collections.bitmap.ImmutableBitmap>) instead of buildBitmapColumnIndexFromSortedIteratorBinarySearch(org.apache.druid.collections.bitmap.BitmapFactory, java.lang.Iterable<T>, int, org.apache.druid.segment.data.Indexed<T>, org.apache.druid.segment.data.Indexed<org.apache.druid.collections.bitmap.ImmutableBitmap>, com.google.common.base.Supplier<org.apache.druid.collections.bitmap.ImmutableBitmap>).

Method Summary

Modifier and Type	Method	Description
static <T> BitmapColumnIndex	buildBitmapColumnIndexFromIteratorBinarySearch(BitmapFactory bitmapFactory, Iterable<T> values, int size, Indexed<T> dictionary, Indexed<ImmutableBitmap> bitmaps, com.google.common.base.Supplier<ImmutableBitmap> getUnknownsIndex)	Helper method for implementing forSortedValues(java.util.List<?>, org.apache.druid.segment.column.TypeSignature<org.apache.druid.segment.column.ValueType>) for a value set that is NOT sorted the same as the column dictionary.
static <T> BitmapColumnIndex	buildBitmapColumnIndexFromSortedIteratorBinarySearch(BitmapFactory bitmapFactory, Iterable<T> values, int size, Indexed<T> dictionary, Indexed<ImmutableBitmap> bitmaps, com.google.common.base.Supplier<ImmutableBitmap> getUnknownsIndex)	Helper method for implementing forSortedValues(java.util.List<?>, org.apache.druid.segment.column.TypeSignature<org.apache.druid.segment.column.ValueType>) for a value set that is sorted the same as the column dictionary.
static <T> BitmapColumnIndex	buildBitmapColumnIndexFromSortedIteratorScan(BitmapFactory bitmapFactory, Comparator<T> comparator, Iterable<T> values, int size, Indexed<T> dictionary, Indexed<ImmutableBitmap> bitmaps, com.google.common.base.Supplier<ImmutableBitmap> unknownsBitmap)	Helper method for implementing forSortedValues(java.util.List<?>, org.apache.druid.segment.column.TypeSignature<org.apache.druid.segment.column.ValueType>) for a value set that is sorted the same as the column dictionary.
BitmapColumnIndex	forSortedValues(List<?> sortedValues, TypeSignature<ValueType> matchValueType)	Get the wrapped ImmutableBitmap corresponding to the specified set of values (if they are contained in the underlying column).

Field Detail

SORTED_SCAN_RATIO_THRESHOLD

static final double SORTED_SCAN_RATIO_THRESHOLD

threshold of sorted match value iterator size compared to dictionary size to use buildBitmapColumnIndexFromSortedIteratorScan(org.apache.druid.collections.bitmap.BitmapFactory, java.util.Comparator<T>, java.lang.Iterable<T>, int, org.apache.druid.segment.data.Indexed<T>, org.apache.druid.segment.data.Indexed<org.apache.druid.collections.bitmap.ImmutableBitmap>, com.google.common.base.Supplier<org.apache.druid.collections.bitmap.ImmutableBitmap>) instead of buildBitmapColumnIndexFromSortedIteratorBinarySearch(org.apache.druid.collections.bitmap.BitmapFactory, java.lang.Iterable<T>, int, org.apache.druid.segment.data.Indexed<T>, org.apache.druid.segment.data.Indexed<org.apache.druid.collections.bitmap.ImmutableBitmap>, com.google.common.base.Supplier<org.apache.druid.collections.bitmap.ImmutableBitmap>).

See Also:

Constant Field Values

SIZE_WORTH_CHECKING_MIN

static final int SIZE_WORTH_CHECKING_MIN

minimum sorted match value iterator size to trim the initial values from the iterator to seek to the start of the value dictionary when using buildBitmapColumnIndexFromSortedIteratorScan(org.apache.druid.collections.bitmap.BitmapFactory, java.util.Comparator<T>, java.lang.Iterable<T>, int, org.apache.druid.segment.data.Indexed<T>, org.apache.druid.segment.data.Indexed<org.apache.druid.collections.bitmap.ImmutableBitmap>, com.google.common.base.Supplier<org.apache.druid.collections.bitmap.ImmutableBitmap>) or buildBitmapColumnIndexFromSortedIteratorBinarySearch(org.apache.druid.collections.bitmap.BitmapFactory, java.lang.Iterable<T>, int, org.apache.druid.segment.data.Indexed<T>, org.apache.druid.segment.data.Indexed<org.apache.druid.collections.bitmap.ImmutableBitmap>, com.google.common.base.Supplier<org.apache.druid.collections.bitmap.ImmutableBitmap>).

See Also:

Constant Field Values

Method Detail

forSortedValues

@Nullable
BitmapColumnIndex forSortedValues(@Nonnull
                                  List<?> sortedValues,
                                  TypeSignature<ValueType> matchValueType)

Get the wrapped ImmutableBitmap corresponding to the specified set of values (if they are contained in the underlying column). The set must be sorted using the comparator of the supplied matchValueType.

Parameters:

sortedValues - values to match, sorted in matchValueType order

matchValueType - type of the value to match, used to assist conversion from the match value type to the column value type

Returns:

ImmutableBitmap corresponding to the rows which match the values, or null if an index connot be computed for the supplied value type

buildBitmapColumnIndexFromSortedIteratorScan

static <T> BitmapColumnIndex buildBitmapColumnIndexFromSortedIteratorScan(BitmapFactory bitmapFactory,
                                                                          Comparator<T> comparator,
                                                                          Iterable<T> values,
                                                                          int size,
                                                                          Indexed<T> dictionary,
                                                                          Indexed<ImmutableBitmap> bitmaps,
                                                                          com.google.common.base.Supplier<ImmutableBitmap> unknownsBitmap)

Helper method for implementing forSortedValues(java.util.List<?>, org.apache.druid.segment.column.TypeSignature<org.apache.druid.segment.column.ValueType>) for a value set that is sorted the same as the column dictionary.

Builds a BitmapColumnIndex from an Iterable that is sorted the same as the columns Indexed value dictionary. Uses a strategy that does zipping similar to the merge step of a sort-merge, where we step forward on both the iterator and the dictionary to find matches to build a Iterable.

If sorted match value iterator size is greater than (dictionary size * SORTED_SCAN_RATIO_THRESHOLD), consider using this method instead of buildBitmapColumnIndexFromSortedIteratorBinarySearch(org.apache.druid.collections.bitmap.BitmapFactory, java.lang.Iterable<T>, int, org.apache.druid.segment.data.Indexed<T>, org.apache.druid.segment.data.Indexed<org.apache.druid.collections.bitmap.ImmutableBitmap>, com.google.common.base.Supplier<org.apache.druid.collections.bitmap.ImmutableBitmap>).

If the values in the iterator are NOT sorted the same as the dictionary, do NOT use this method, use buildBitmapColumnIndexFromIteratorBinarySearch(org.apache.druid.collections.bitmap.BitmapFactory, java.lang.Iterable<T>, int, org.apache.druid.segment.data.Indexed<T>, org.apache.druid.segment.data.Indexed<org.apache.druid.collections.bitmap.ImmutableBitmap>, com.google.common.base.Supplier<org.apache.druid.collections.bitmap.ImmutableBitmap>) instead.

buildBitmapColumnIndexFromSortedIteratorBinarySearch

static <T> BitmapColumnIndex buildBitmapColumnIndexFromSortedIteratorBinarySearch(BitmapFactory bitmapFactory,
                                                                                  Iterable<T> values,
                                                                                  int size,
                                                                                  Indexed<T> dictionary,
                                                                                  Indexed<ImmutableBitmap> bitmaps,
                                                                                  com.google.common.base.Supplier<ImmutableBitmap> getUnknownsIndex)

Helper method for implementing forSortedValues(java.util.List<?>, org.apache.druid.segment.column.TypeSignature<org.apache.druid.segment.column.ValueType>) for a value set that is sorted the same as the column dictionary.

Builds a BitmapColumnIndex from an Iterable that is sorted the same as the columns Indexed value dictionary. This algorithm iterates the values to match and does a binary search for matching values using Indexed.indexOf(Object) to build a Iterable short-circuiting the iteration if we reach the end of the Indexed before the values to match are exhausted.

If sorted match value iterator size is less than (dictionary size * SORTED_SCAN_RATIO_THRESHOLD), consider using this method instead of buildBitmapColumnIndexFromSortedIteratorScan(org.apache.druid.collections.bitmap.BitmapFactory, java.util.Comparator<T>, java.lang.Iterable<T>, int, org.apache.druid.segment.data.Indexed<T>, org.apache.druid.segment.data.Indexed<org.apache.druid.collections.bitmap.ImmutableBitmap>, com.google.common.base.Supplier<org.apache.druid.collections.bitmap.ImmutableBitmap>).

If the values in the iterator are not sorted the same as the dictionary, do not use this method, use buildBitmapColumnIndexFromIteratorBinarySearch(org.apache.druid.collections.bitmap.BitmapFactory, java.lang.Iterable<T>, int, org.apache.druid.segment.data.Indexed<T>, org.apache.druid.segment.data.Indexed<org.apache.druid.collections.bitmap.ImmutableBitmap>, com.google.common.base.Supplier<org.apache.druid.collections.bitmap.ImmutableBitmap>) instead.

buildBitmapColumnIndexFromIteratorBinarySearch

static <T> BitmapColumnIndex buildBitmapColumnIndexFromIteratorBinarySearch(BitmapFactory bitmapFactory,
                                                                            Iterable<T> values,
                                                                            int size,
                                                                            Indexed<T> dictionary,
                                                                            Indexed<ImmutableBitmap> bitmaps,
                                                                            com.google.common.base.Supplier<ImmutableBitmap> getUnknownsIndex)

Helper method for implementing forSortedValues(java.util.List<?>, org.apache.druid.segment.column.TypeSignature<org.apache.druid.segment.column.ValueType>) for a value set that is NOT sorted the same as the column dictionary.

Builds a BitmapColumnIndex from an Iterable that is NOT sorted the same as the columns Indexed value dictionary. This algorithm iterates the values to match and does a binary search for matching values using Indexed.indexOf(Object) to build a Iterable until the match values iterator is exhausted.

If values of the iterator are sorted the same as the dictionary, use buildBitmapColumnIndexFromSortedIteratorScan(org.apache.druid.collections.bitmap.BitmapFactory, java.util.Comparator<T>, java.lang.Iterable<T>, int, org.apache.druid.segment.data.Indexed<T>, org.apache.druid.segment.data.Indexed<org.apache.druid.collections.bitmap.ImmutableBitmap>, com.google.common.base.Supplier<org.apache.druid.collections.bitmap.ImmutableBitmap>) or buildBitmapColumnIndexFromSortedIteratorBinarySearch(org.apache.druid.collections.bitmap.BitmapFactory, java.lang.Iterable<T>, int, org.apache.druid.segment.data.Indexed<T>, org.apache.druid.segment.data.Indexed<org.apache.druid.collections.bitmap.ImmutableBitmap>, com.google.common.base.Supplier<org.apache.druid.collections.bitmap.ImmutableBitmap>) instead.