Package com.twilio.kudu.sql

Class KuduEnumerable

java.lang.Object

org.apache.calcite.linq4j.DefaultEnumerable<T>

org.apache.calcite.linq4j.AbstractEnumerable<Object>

com.twilio.kudu.sql.KuduEnumerable

All Implemented Interfaces:

Iterable<Object>, org.apache.calcite.linq4j.Enumerable<Object>, org.apache.calcite.linq4j.ExtendedEnumerable<Object>, org.apache.calcite.linq4j.ExtendedOrderedEnumerable<Object>, org.apache.calcite.linq4j.OrderedEnumerable<Object>, org.apache.calcite.linq4j.RawEnumerable<Object>

public final class KuduEnumerable
extends org.apache.calcite.linq4j.AbstractEnumerable<Object>

An Enumerable that *can* returns Kudu records in Ascending order on their primary key. It does so by wrapping a List of CalciteKuduEnumerable, querying each one for their next row and comparing those rows. This requires each CalciteKuduEnumerable scan only one Partition within the Kudu Table. This guarantees the first rows coming out of the AsyncKuduScanner will return rows sorted by primary key.

Field Summary

Fields

Modifier and Type	Field	Description
boolean	groupBySorted
long	limit
long	offset
KuduScanStats	scanStats
boolean	sort

Constructor Summary

Constructors

Constructor

Description

KuduEnumerable(List<List<CalciteKuduPredicate>> predicates, List<Integer> columnIndices, org.apache.kudu.client.AsyncKuduClient client, CalciteKuduTable calciteKuduTable, long limit, long offset, boolean sort, boolean groupBySorted, KuduScanStats scanStats, AtomicBoolean cancelFlag, org.apache.calcite.linq4j.function.Function1<Object,Object> projection, org.apache.calcite.linq4j.function.Predicate1<Object> filterFunction, boolean isSingleObject)

A KuduEnumerable is an Enumerable for Kudu that can be configured to be sorted.

Method Summary

Modifier and Type	Method	Description
KuduEnumerable	clone(List<List<CalciteKuduPredicate>> conjunctions)
org.apache.calcite.linq4j.Enumerator<Object>	enumerator()
org.apache.kudu.Schema	getTableSchema()
<TKey,TAccumulate,TResult> org.apache.calcite.linq4j.Enumerable<TResult>	groupBy(org.apache.calcite.linq4j.function.Function1<Object,TKey> keySelector, org.apache.calcite.linq4j.function.Function0<TAccumulate> accumulatorInitializer, org.apache.calcite.linq4j.function.Function2<TAccumulate,Object,TAccumulate> accumulatorAdder, org.apache.calcite.linq4j.function.Function2<TKey,TAccumulate,TResult> resultSelector)
org.apache.calcite.linq4j.function.Function1<List<Object>,org.apache.calcite.linq4j.Enumerable<Object>>	nestedJoinPredicates(org.apache.calcite.rel.core.Join joinNode)	Return a function that accepts the Left hand sides rows and creates a new SortableEnumerable that will match the batch of rows.
org.apache.calcite.linq4j.Enumerator<Object>	sortedEnumerator(List<org.apache.kudu.client.AsyncKuduScanner> scanners, List<org.apache.calcite.linq4j.Enumerator<CalciteRow>> subEnumerables)
org.apache.calcite.linq4j.Enumerator<Object>	unsortedEnumerator(List<org.apache.kudu.client.AsyncKuduScanner> scanners, BlockingQueue<CalciteScannerMessage<CalciteRow>> messages)

Methods inherited from class org.apache.calcite.linq4j.AbstractEnumerable

iterator

Methods inherited from class org.apache.calcite.linq4j.DefaultEnumerable

aggregate, aggregate, aggregate, all, any, any, asEnumerable, asOrderedQueryable, asQueryable, average, average, average, average, average, average, average, average, average, average, cast, concat, contains, contains, correlateJoin, count, count, createOrderedEnumerable, defaultIfEmpty, defaultIfEmpty, distinct, distinct, elementAt, elementAtOrDefault, except, except, except, except, first, first, firstOrDefault, firstOrDefault, foreach, getThis, getThisOrdered, groupBy, groupBy, groupBy, groupBy, groupBy, groupBy, groupBy, groupBy, groupBy, groupJoin, groupJoin, hashJoin, hashJoin, hashJoin, hashJoin, intersect, intersect, intersect, intersect, into, last, last, lastOrDefault, lastOrDefault, longCount, longCount, max, max, max, max, max, max, max, max, max, max, max, max, min, min, min, min, min, min, min, min, min, min, min, min, ofType, orderBy, orderBy, orderByDescending, orderByDescending, removeAll, reverse, select, select, selectMany, selectMany, selectMany, selectMany, sequenceEqual, sequenceEqual, single, single, singleOrDefault, singleOrDefault, skip, skipWhile, skipWhile, sortedGroupBy, sum, sum, sum, sum, sum, sum, sum, sum, sum, sum, take, takeWhile, takeWhile, thenBy, thenBy, thenByDescending, thenByDescending, toList, toLookup, toLookup, toLookup, toLookup, toMap, toMap, toMap, toMap, union, union, where, where, zip

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Field Detail

sort

public final boolean sort

groupBySorted

public final boolean groupBySorted

limit

public final long limit

offset

public final long offset

scanStats

public final KuduScanStats scanStats

Constructor Detail

KuduEnumerable

public KuduEnumerable(List<List<CalciteKuduPredicate>> predicates,
                      List<Integer> columnIndices,
                      org.apache.kudu.client.AsyncKuduClient client,
                      CalciteKuduTable calciteKuduTable,
                      long limit,
                      long offset,
                      boolean sort,
                      boolean groupBySorted,
                      KuduScanStats scanStats,
                      AtomicBoolean cancelFlag,
                      org.apache.calcite.linq4j.function.Function1<Object,Object> projection,
                      org.apache.calcite.linq4j.function.Predicate1<Object> filterFunction,
                      boolean isSingleObject)

A KuduEnumerable is an Enumerable for Kudu that can be configured to be sorted.

Parameters:

predicates - list of the filters for each disjoint Kudu Scan

columnIndices - the column indexes to fetch from the table

client - Kudu client that will execute the scans

calciteKuduTable - table metadata for the scan

limit - the number of rows this should return. -1 indicates no limit

offset - the number of rows from kudu to skip prior to returning rows

sort - whether or not have Kudu RPCs come back in sorted by primary key

groupBySorted - when sorted, and ExtendedEnumerable.groupBy(Function1, Function0, Function2, Function2)

scanStats - a container of scan stats that should be updated as the scan executes.

cancelFlag - boolean indicating the end process has asked the query to finish.

projection - function to translate RowResult into Calcite

filterFunction - filter applied to every RowResult

isSingleObject - whether or not Calcite object is an Object or an Object[]

Method Detail

unsortedEnumerator

public org.apache.calcite.linq4j.Enumerator<Object> unsortedEnumerator(List<org.apache.kudu.client.AsyncKuduScanner> scanners,
                                                                       BlockingQueue<CalciteScannerMessage<CalciteRow>> messages)

sortedEnumerator

public org.apache.calcite.linq4j.Enumerator<Object> sortedEnumerator(List<org.apache.kudu.client.AsyncKuduScanner> scanners,
                                                                     List<org.apache.calcite.linq4j.Enumerator<CalciteRow>> subEnumerables)

getTableSchema

public org.apache.kudu.Schema getTableSchema()

enumerator

public org.apache.calcite.linq4j.Enumerator<Object> enumerator()

groupBy

public <TKey,TAccumulate,TResult> org.apache.calcite.linq4j.Enumerable<TResult> groupBy(org.apache.calcite.linq4j.function.Function1<Object,TKey> keySelector,
                                                                                                    org.apache.calcite.linq4j.function.Function0<TAccumulate> accumulatorInitializer,
                                                                                                    org.apache.calcite.linq4j.function.Function2<TAccumulate,Object,TAccumulate> accumulatorAdder,
                                                                                                    org.apache.calcite.linq4j.function.Function2<TKey,TAccumulate,TResult> resultSelector)

Specified by:

groupBy in interface org.apache.calcite.linq4j.ExtendedEnumerable<Object>

Overrides:

groupBy in class org.apache.calcite.linq4j.DefaultEnumerable<Object>

clone

public KuduEnumerable clone(List<List<CalciteKuduPredicate>> conjunctions)

nestedJoinPredicates

public org.apache.calcite.linq4j.function.Function1<List<Object>,org.apache.calcite.linq4j.Enumerable<Object>> nestedJoinPredicates(org.apache.calcite.rel.core.Join joinNode)

Return a function that accepts the Left hand sides rows and creates a new SortableEnumerable that will match the batch of rows.

Parameters:

joinNode - The Join relation for this nest join.

Returns:

a function that produces another SortableEnumerable that matches the batches passed in.